# Date format for tine-tuning AI models

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 20, 2026, 5:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116 "2026-05-20T05:38:28Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kegintheai](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kegintheai/32/59166_2.png) [@kegintheai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkegintheai)%5C
**Post date:** [May 20, 2026, 5:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116%2F1 "2026-05-20T05:38:28Z")

</div>

Hi. I’m struggling with the date format to use in my datasets to fine-tune QwenX.X and Gemma-X.X models. The models is always inferring the wrong dates after fine-tuning. I tried several formats as shown in the example below to no avail. I would appreciate any guidance regarding the right date format to use or how to discover it for a given model.

Examples of date formats I test but none of them worked:

user: When did XYZ work for ABC. Assistant: From January 2021 till December 2021.

user: When did XYZ work for ABC. Assistant: From 01, 2021 till 12, 2021.

user: When did XYZ work for ABC. Assistant: From 01-01-2021 till 12-01- 2021.

user: When did XYZ work for ABC. Assistant: From 01-01-2021 till 12-01- 2021 .

user: When did XYZ work for ABC. Assistant: From January 2021 01-01-2021 till December 2021 12-01- 2021 .

---

<div class="post-metadata">

**Author:** ![Vultieris](https://avatars.discourse-cdn.com/v4/letter/v/f19dbf/32.png) [@Vultieris](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FVultieris)%5C
**Post date:** [May 20, 2026, 9:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116%2F2 "2026-05-20T09:21:54Z")

</div>

The issue is usually consistency and how the training examples are formatted. Avoid mixing natural language, commas, spaces, and different date styles in the same dataset. Also avoid adding fake exact days if your source data only has month/year.

I’d use one clear format everywhere, for example:

From 2021-01 to 2021-12

or, if you need exact dates:

From 2021-01-01 to 2021-12-01

---

<div class="post-metadata">

**Author:** ![kegintheai](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kegintheai/32/59166_2.png) [@kegintheai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkegintheai)%5C
**Post date:** [May 20, 2026, 11:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116%2F4 "2026-05-20T11:18:39Z")

</div>

Thanks @Vultieris I appreciate your valuable feedback. I’ll test it. Cheers!

---

<div class="post-metadata">

**Author:** ![kegintheai](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kegintheai/32/59166_2.png) [@kegintheai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkegintheai)%5C
**Post date:** [May 21, 2026, 3:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116%2F5 "2026-05-21T15:36:19Z")

</div>

Unfortunately it didn’t work out. My fine-tuned model is still making up invalid dates. I’m beginning to think the issue lies elsewhere outside the scope of date formats. I’ll keep digging!

I should add that I have witnessed this issue across the board while fine-tuning several QwenX.X and LlamaX.X models using SFT. I have traced the tokenizing of the dates and displayed the trainer’s batch input-ids right before starting training. They matched the dates found in the dataset.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [May 22, 2026, 1:35pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116%2F6 "2026-05-22T13:35:48Z")

</div>

Are you perhaps trying to have the LLM handle databases and strict data management on its own? This is generally a task that LLMs aren’t well-suited for. While it may be unavoidable if there are specific constraints, it’s generally more reliable to divide these tasks among different systems:

* * *

## Date format for fine-tuning Qwen/Gemma/Llama models: likely not just a date-format issue

I do **not** think there is a hidden “correct date format” that must be discovered separately for Qwen, Gemma, or Llama.

A consistent date format is still important. But if you already tried several formats, standardized the data, and still see the fine-tuned model inventing wrong or invalid dates, then the problem is probably **outside date formatting**.

The likely issue is one or more of these:

1. You are asking SFT to teach **closed-book factual recall** :  
`\<person\> + \<company\> -> exact start/end dates`.

2. The model may learn the **answer shape** but not the **exact factual mapping** :  
it learns “answer with a date range,” but not the correct date range.

3. The correct date may appear in `input_ids`, but not in the supervised `labels`.

4. The chat template / assistant mask may be wrong for the model family.

5. The output is unconstrained, so invalid dates such as `2021-13` are still possible.

6. The actual dates may need to live in a database, search index, RAG system, or structured record store rather than only in the model weights.

So my practical recommendation is:

> Use one clean date format, but do not rely on fine-tuning alone to memorize exact dates.  
> Use structured lookup or RAG for the facts.  
> Use SFT to teach the model how to use provided records, normalize dates, return JSON, and abstain when no record exists.  
> Use validation or constrained decoding to prevent invalid dates.

Useful background links:

- [Hugging Face TRL SFTTrainer docs](https://huggingface.co/docs/trl/sft_trainer)
- [Hugging Face TRL chat templates docs](https://huggingface.co/docs/trl/chat_templates)
- [Hugging Face Transformers chat templates docs](https://huggingface.co/docs/transformers/en/chat_templating)
- [Hugging Face RAG docs](https://huggingface.co/docs/transformers/en/model_doc/rag)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://aclanthology.org/2024.emnlp-main.444/)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://arxiv.org/abs/2312.05934)
- [Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge](https://arxiv.org/abs/2403.01432)
- [vLLM structured outputs](https://docs.vllm.ai/en/latest/features/structured_outputs/)
- [Outlines structured generation](https://dottxt-ai.github.io/outlines/latest/)
- [Ragas faithfulness metric](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/)
- [Ragas context precision metric](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/)

* * *

## 1. The date format I would use

If the source data only has month and year, use month precision:

```plaintext
From 2021-01 to 2021-12

```

or, better, structured JSON:

```json
{
  "start_date": "2021-01",
  "start_precision": "month",
  "end_date": "2021-12",
  "end_precision": "month"
}

```

If the source data truly has exact days, use exact ISO dates:

```plaintext
From 2021-01-01 to 2021-12-01

```

or:

```json
{
  "start_date": "2021-01-01",
  "start_precision": "day",
  "end_date": "2021-12-01",
  "end_precision": "day"
}

```

Do **not** silently convert this:

```plaintext
January 2021 to December 2021

```

into this:

```plaintext
2021-01-01 to 2021-12-01

```

unless those exact days are actually known.

That would add fake precision. The model may learn that unknown days should be invented as `01`, which is usually not what you want.

For employment histories, resumes, HR-style records, contracts, and timelines, I would usually use:

```plaintext
YYYY-MM

```

for month-level records, and keep a separate precision field.

* * *

## 2. Why the format changes did not fix it

The tested formats mix several patterns:

```plaintext
From January 2021 till December 2021.
From 01, 2021 till 12, 2021.
From 01-01-2021 till 12-01-2021.
From January 2021 01-01-2021 till December 2021 12-01-2021.

```

That inconsistency is worth fixing. It gives the model multiple competing surface patterns.

However, once you standardized to something like:

```plaintext
From 2021-01 to 2021-12

```

and the model still invented wrong or invalid dates, that strongly suggests that date format was not the main bottleneck.

There are two separate tasks here:

```plaintext
Task A:
Normalize "January 2021" to "2021-01".

Task B:
Know that <person> worked for <company> from 2021-01 to 2021-12.

```

Task A is date normalization. Fine-tuning can handle that.

Task B is factual recall. Fine-tuning is much less reliable for that.

The model may learn:

```plaintext
When asked about employment dates, answer with:
"From YYYY-MM to YYYY-MM."

```

but fail to learn:

```plaintext
<person> + <company> = 2021-01 to 2021-12

```

That distinction is the core of the problem.

* * *

## 3. This is probably a closed-book factual-recall problem

Your examples have this structure:

```plaintext
User:
When did XYZ work for ABC?

Assistant:
From 2021-01 to 2021-12.

```

At inference time, the model sees only:

```plaintext
When did XYZ work for ABC?

```

It does **not** see the employment record. Therefore, it must recover the fact from model weights:

```plaintext
XYZ + ABC -> 2021-01 to 2021-12

```

That is **closed-book factual recall**.

Closed-book means the answer is not provided in the prompt, not retrieved from an external source, and not looked up from a database visible to the model. The model must answer from memory.

This is fragile for exact dates because dates are high-entropy factual values. A model can often learn that an answer should look like a date range, but the exact dates are not predictable from the words `XYZ` and `ABC`.

This matches existing research:

- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://aclanthology.org/2024.emnlp-main.444/) reports that LLMs struggle to acquire new factual knowledge through fine-tuning and that fine-tuning new knowledge can increase hallucination tendency.
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://arxiv.org/abs/2312.05934) finds that RAG consistently outperforms unsupervised fine-tuning on knowledge-intensive tasks, including entirely new knowledge.
- [Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge](https://arxiv.org/abs/2403.01432) is especially relevant if your facts are private, rare, internal, or low-frequency.

In other words, you may not be debugging a date-string issue. You may be seeing the limits of using SFT as a factual database.

* * *

## 4. `input_ids` matching the dataset is not enough

Checking `input_ids` is useful. It tells you that tokenization did not completely destroy the date string.

But it does **not** prove the model is learning those dates.

In causal LM SFT, the answer text often appears in `input_ids`. That is normal. The real question is whether the answer tokens also appear in `labels` where loss is applied.

Many SFT pipelines mask tokens by setting labels to `-100`. Tokens with label `-100` are ignored by the loss.

So this can happen:

```plaintext
input_ids:
... From 2021-01 to 2021-12 ...

labels:
... -100 -100 -100 -100 ...

```

In that case, the date is visible in the batch, but the model is not trained to generate it.

This is particularly important when using:

- `assistant_only_loss`
- `completion_only_loss`
- `DataCollatorForCompletionOnlyLM`
- packing
- custom chat templates
- Qwen/Gemma/Llama-specific templates

See:

- [TRL SFTTrainer docs](https://huggingface.co/docs/trl/sft_trainer)
- [TRL chat templates docs](https://huggingface.co/docs/trl/chat_templates)
- [TRL issue on generation markers for assistant-only loss](https://github.com/huggingface/trl/issues/5471)
- [Transformers chat templates docs](https://huggingface.co/docs/transformers/en/chat_templating)

Run this check before training:

```python
batch = next(iter(trainer.get_train_dataloader()))

input_ids = batch["input_ids"][0]
labels = batch["labels"][0]

print("FULL INPUT")
print(tokenizer.decode(input_ids, skip_special_tokens=False))

print("\nSUPERVISED TOKENS ONLY")
supervised_ids = labels[labels != -100]
print(tokenizer.decode(supervised_ids, skip_special_tokens=False))

```

You want to see the assistant answer in the supervised region:

```plaintext
From 2021-01 to 2021-12.

```

or, if using JSON:

```json
{"start_date":"2021-01","end_date":"2021-12"}

```

If the date is not present in `labels[labels != -100]`, the model is not being trained to generate that date.

* * *

## 5. Chat templates are another likely cause

Qwen, Gemma, and Llama-style instruction models do not all expect the same prompt format.

A raw text format like this:

```plaintext
user: When did XYZ work for ABC.
Assistant: From 2021-01 to 2021-12.

```

may not match the actual chat template expected by the model.

Hugging Face’s [chat template documentation](https://huggingface.co/docs/transformers/en/chat_templating) explains that chat models use model-specific control tokens. The same user/assistant conversation can be rendered differently for different model families.

So training examples should usually be represented structurally:

```python
messages = [
    {"role": "user", "content": "When did XYZ work for ABC?"},
    {"role": "assistant", "content": "From 2021-01 to 2021-12."},
]

```

Then render them using the model tokenizer:

```python
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=False,
)

print(text)

```

At inference time:

```python
messages = [
    {"role": "user", "content": "When did XYZ work for ABC?"},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

print(prompt)

```

Training and inference must be structurally compatible.

A common failure is:

```plaintext
Training:
manual "user:" / "Assistant:" strings

Inference:
tokenizer.apply_chat_template(...)

```

or:

```plaintext
Training:
one model family's chat template

Inference:
another model family's chat template

```

This can produce poor generations even when the data itself is correct.

* * *

## 6. Assistant-only loss and generation markers can matter

If you use `assistant_only_loss=True`, the trainer needs to know which tokens belong to the assistant response.

That often depends on the chat template producing an assistant-token mask. TRL documents that SFT with assistant-only loss requires `{% generation %}` and `{% endgeneration %}` markers around assistant output so that the loss mask can target only assistant tokens.

Useful links:

- [TRL SFTTrainer docs](https://huggingface.co/docs/trl/sft_trainer)
- [TRL chat templates docs](https://huggingface.co/docs/trl/chat_templates)
- [TRL issue #5471: generation markers for common model families](https://github.com/huggingface/trl/issues/5471)

For Qwen, Gemma, or Llama models, verify:

```plaintext
Does the tokenizer chat template support assistant masks?
Does the trainer actually supervise assistant tokens?
Does the supervised region include the date?
Does this still work if packing is enabled?

```

For debugging, temporarily disable packing:

```python
packing = False

```

Packing is useful later, but it makes it harder to inspect example boundaries, EOS behavior, truncation, and loss masks.

* * *

## 7. Fine-tuning alone can work only in narrow conditions

Fine-tuning alone may work if all of these are true:

- The number of facts is small.
- The facts are static.
- Each fact appears many times.
- You use many paraphrases per fact.
- The questions are highly repetitive.
- The use case can tolerate occasional wrong answers.
- The deployment is low-risk.
- You evaluate exact-match behavior carefully.

Example of a narrow case where no-RAG SFT might be acceptable:

```plaintext
20 fictional people
20 fictional companies
many paraphrases per fact
internal demo
low consequence if an answer is occasionally wrong

```

But if you have:

- hundreds or thousands of people,
- many companies,
- similar names,
- changing records,
- private facts,
- audit requirements,
- exact-date requirements,

then fine-tuning alone is the wrong default.

For exact employment dates, the model should not be the only database.

* * *

## 8. Better approach: structured lookup or RAG

For employment dates, store the facts outside the model.

A record should look something like this:

```json
{
  "record_id": "employment_000123",
  "person_id": "person_xyz",
  "person_name": "XYZ",
  "employer_id": "org_abc",
  "employer_name": "ABC",
  "start_date": "2021-01",
  "start_precision": "month",
  "end_date": "2021-12",
  "end_precision": "month",
  "is_current": false
}

```

Then, at inference time:

1. Parse the question.
2. Resolve the person and employer.
3. Retrieve the matching employment record.
4. Give the record to the model.
5. Ask the model to answer using only that record.
6. Validate the output.

This changes the task from:

```plaintext
The model must remember the date.

```

to:

```plaintext
The model must read the date from the retrieved record and format it correctly.

```

That is a much better use of an LLM.

See:

- [Hugging Face RAG docs](https://huggingface.co/docs/transformers/en/model_doc/rag)
- [Hugging Face Advanced RAG cookbook](https://huggingface.co/learn/cookbook/en/advanced_rag)
- [Code a simple RAG from scratch](https://huggingface.co/blog/ngxson/make-your-own-rag)

If your data is already structured, start with **structured lookup** , not pure vector search.

For example:

```sql
SELECT start_date, end_date
FROM employment_records
WHERE person_id = 'person_xyz'
AND employer_id = 'org_abc';

```

Pure vector RAG is useful for unstructured text. But if you already have person IDs, employer IDs, start dates, and end dates, structured lookup is more reliable.

* * *

## 9. What SFT should do in the better design

SFT is still useful. It is just useful for a different job.

Do **not** primarily fine-tune the model to memorize this:

```plaintext
XYZ worked for ABC from 2021-01 to 2021-12.

```

Instead, fine-tune it to do this:

```plaintext
Given retrieved records + question -> return the correct structured answer using only the records.

```

A better SFT example:

```json
{
  "messages": [
    {
      "role": "system",
      "content": "Use only the provided employment records. Return JSON only. Do not invent dates."
    },
    {
      "role": "user",
      "content": "Records:\n[{\"person\":\"XYZ\",\"employer\":\"ABC\",\"start\":\"January 2021\",\"end\":\"December 2021\"}]\n\nQuestion: When did XYZ work for ABC?"
    },
    {
      "role": "assistant",
      "content": "{\"record_found\":true,\"start_date\":\"2021-01\",\"start_precision\":\"month\",\"end_date\":\"2021-12\",\"end_precision\":\"month\"}"
    }
  ]
}

```

This teaches:

- use context,
- normalize dates,
- preserve precision,
- return schema-compatible JSON,
- do not invent dates,
- abstain when no record exists.

That is a much better SFT target than closed-book memorization.

* * *

## 10. Add no-answer and distractor examples

A common mistake is training only examples where every question has an answer.

That teaches the model:

```plaintext
Always produce a date.

```

Then when it does not know, it still invents one.

You need examples where the correct output is null.

Example:

```json
{
  "messages": [
    {
      "role": "system",
      "content": "Use only the provided employment records. Return JSON only."
    },
    {
      "role": "user",
      "content": "Records:\n[]\n\nQuestion: When did XYZ work for ABC?"
    },
    {
      "role": "assistant",
      "content": "{\"record_found\":false,\"start_date\":null,\"end_date\":null,\"reason\":\"No matching employment record was provided.\"}"
    }
  ]
}

```

You also need distractor examples:

```json
{
  "records": [
    {
      "person": "XYZ",
      "employer": "ABD",
      "start_date": "2021-01",
      "end_date": "2021-12"
    },
    {
      "person": "XYX",
      "employer": "ABC",
      "start_date": "2020-01",
      "end_date": "2020-12"
    }
  ],
  "question": "When did XYZ work for ABC?",
  "answer": {
    "record_found": false,
    "start_date": null,
    "end_date": null
  }
}

```

This teaches the model not to combine the right person from one record with the right company from another.

That matters a lot for employment data, where names and organizations can be similar.

* * *

## 11. Invalid dates need validation or constrained decoding

Even after SFT, the model can still produce:

```plaintext
2021-13
2021-00
2021-02-31
2021/01
January 2021 2021-01

```

For machine-readable outputs, prompting is not enough.

Use one or more of:

- JSON Schema,
- regex-constrained decoding,
- grammar-constrained decoding,
- post-generation validation,
- retry on invalid output.

Useful links:

- [vLLM structured outputs](https://docs.vllm.ai/en/latest/features/structured_outputs/)
- [Outlines structured generation](https://dottxt-ai.github.io/outlines/latest/)
- [LM Format Enforcer](https://github.com/noamgat/lm-format-enforcer)
- [llguidance](https://github.com/guidance-ai/llguidance)

For month-level dates, validate with:

```python
import re

MONTH_RE = re.compile(r"^\d{4}-(0[1-9]|1[0-2])$")

def valid_month(value: str) -> bool:
    return bool(MONTH_RE.fullmatch(value))

```

For exact ISO dates:

```python
from datetime import date

def valid_iso_date(value: str) -> bool:
    try:
        date.fromisoformat(value)
        return True
    except ValueError:
        return False

```

Also validate date ordering:

```python
def month_key(value: str) -> tuple[int, int]:
    year, month = value.split("-")
    return int(year), int(month)

assert month_key(start_date) <= month_key(end_date)

```

Important limitation:

```plaintext
Validation can reject invalid dates.
It cannot prove that a valid date is factually correct.

```

For factual correctness, you need the retrieved record or database.

* * *

## 12. Evaluation should be exact, not semantic

For this task, do not evaluate only by reading a few answers manually.

Use exact metrics:

- `start_date` exact match
- `end_date` exact match
- `start_precision` exact match
- `end_precision` exact match
- `record_id` exact match
- invalid date rate
- malformed JSON rate
- false answer rate when no record exists
- wrong-person rate
- wrong-employer rate

If you use RAG, evaluate retrieval separately from generation.

Useful RAG evaluation links:

- [Ragas context precision](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/)
- [Ragas faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/)
- [Hugging Face RAG evaluation cookbook](https://huggingface.co/learn/cookbook/en/rag_evaluation)

For this task:

```plaintext
Context precision:
Did retrieval find the correct employment record?

Faithfulness:
Did the model answer using only the retrieved record?

Exact match:
Did the final start_date and end_date match the expected dates?

```

All three matter.

* * *

## 13. Debug checklist for the current SFT setup

### Check 1: print the rendered chat template

```python
messages = [
    {"role": "user", "content": "When did XYZ work for ABC?"},
    {"role": "assistant", "content": "From 2021-01 to 2021-12."},
]

rendered = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=False,
)

print(rendered)

```

Verify:

- Is this the model’s expected template?
- Does it include the assistant answer?
- Does it include duplicated special tokens?
- Does it match the inference template?

* * *

### Check 2: print supervised tokens

```python
batch = next(iter(trainer.get_train_dataloader()))

input_ids = batch["input_ids"][0]
labels = batch["labels"][0]

print("FULL INPUT")
print(tokenizer.decode(input_ids, skip_special_tokens=False))

print("\nSUPERVISED TOKENS ONLY")
print(tokenizer.decode(labels[labels != -100], skip_special_tokens=False))

```

Verify that the supervised tokens include:

```plaintext
From 2021-01 to 2021-12.

```

or the expected JSON answer.

* * *

### Check 3: count supervised tokens

```python
num_total = labels.numel()
num_supervised = (labels != -100).sum().item()

print("total tokens:", num_total)
print("supervised tokens:", num_supervised)
print("supervised ratio:", num_supervised / num_total)

```

Red flags:

- `supervised tokens = 0`
- only special tokens are supervised
- prompt tokens are supervised but answer tokens are ignored
- answer date is missing from supervised tokens

* * *

### Check 4: disable packing temporarily

```python
packing = False

```

Inspect one example at a time.

Packing is useful later, but it makes debugging masks, boundaries, EOS, and truncation harder.

* * *

### Check 5: run a tiny overfit test

Create one synthetic example with unusual dates:

```plaintext
Person_AAA worked for Company_BBB from 2091-03 to 2092-07.

```

Train on that one example.

Ask:

```plaintext
When did Person_AAA work for Company_BBB?

```

Expected answer:

```plaintext
From 2091-03 to 2092-07.

```

If the model cannot memorize one example, the problem is probably not date format. It is likely one of:

- labels,
- masking,
- chat template,
- adapter loading,
- learning rate,
- LoRA target modules,
- EOS/truncation,
- inference prompt mismatch.

* * *

### Check 6: run a ten-example overfit test

Train on 10 synthetic examples with unusual dates.

Evaluate on the same exact prompts.

If it fails, your training pipeline is probably broken.

If it succeeds on exact prompts but fails on paraphrases, the model memorized the prompt surface rather than robustly learning entity-date associations.

* * *

### Check 7: run a context-present test

Compare these two prompts.

Closed-book:

```plaintext
When did XYZ work for ABC?

```

Context-present:

```plaintext
Record:
XYZ worked for ABC from 2021-01 to 2021-12.

Question:
When did XYZ work for ABC?

```

If context-present works and closed-book fails, the solution is retrieval or structured lookup, not more date-format changes.

* * *

## 14. Recommended target format

Instead of this as the main target:

```plaintext
user: When did XYZ work for ABC.
assistant: From 2021-01 to 2021-12.

```

use a context-grounded format:

```json
{
  "messages": [
    {
      "role": "system",
      "content": "Answer employment-date questions using only the provided records. Return JSON only. Do not invent dates."
    },
    {
      "role": "user",
      "content": "Records:\n[{\"record_id\":\"employment_000123\",\"person\":\"XYZ\",\"employer\":\"ABC\",\"start_date\":\"2021-01\",\"start_precision\":\"month\",\"end_date\":\"2021-12\",\"end_precision\":\"month\"}]\n\nQuestion: When did XYZ work for ABC?"
    },
    {
      "role": "assistant",
      "content": "{\"record_found\":true,\"record_id\":\"employment_000123\",\"start_date\":\"2021-01\",\"start_precision\":\"month\",\"end_date\":\"2021-12\",\"end_precision\":\"month\"}"
    }
  ]
}

```

For a missing record:

```json
{
  "messages": [
    {
      "role": "system",
      "content": "Answer employment-date questions using only the provided records. Return JSON only. Do not invent dates."
    },
    {
      "role": "user",
      "content": "Records:\n[]\n\nQuestion: When did XYZ work for ABC?"
    },
    {
      "role": "assistant",
      "content": "{\"record_found\":false,\"record_id\":null,\"start_date\":null,\"end_date\":null,\"reason\":\"No matching employment record was provided.\"}"
    }
  ]
}

```

This trains the right behavior:

- use context,
- do not invent,
- return structured output,
- preserve precision,
- abstain when the record is absent.

It does not try to turn the model into the only source of truth.

* * *

## 15. Can fine-tuning alone solve it?

### In theory

Yes, sometimes.

Fine-tuning alone can memorize a small number of static facts if:

- the dataset is small,
- facts are repeated many times,
- the same facts appear in many paraphrases,
- the evaluation questions are similar to training questions,
- occasional errors are acceptable.

### In this case

Probably not reliably.

The symptoms suggest that the model is either:

- not actually being supervised on the answer tokens,
- not receiving the correct chat template,
- learning the answer pattern but not the date mapping,
- or being asked to do a task that should use retrieval or structured lookup.

Even if the SFT pipeline is fixed, fine-tuning alone is still not the best design if exact dates matter.

For employment dates, I would not trust model weights as the only source of truth.

* * *

## 16. Better architecture

I would use this design:

```plaintext
User question
  ↓
Entity extraction / normalization
  ↓
Person and employer resolution
  ↓
Structured lookup or RAG
  ↓
Retrieved employment record(s)
  ↓
LLM answers using only retrieved records
  ↓
Strict JSON output
  ↓
Date/schema validator
  ↓
Final natural-language answer

```

Example final prompt:

```plaintext
You answer employment-date questions using only the provided records.

Rules:
- Do not invent dates.
- If no exact matching record is provided, return null.
- Preserve date precision.
- Use YYYY-MM for month precision.
- Use YYYY-MM-DD for day precision.
- Return JSON only.

Records:
[
  {
    "record_id": "employment_000123",
    "person_name": "XYZ",
    "employer_name": "ABC",
    "start_date": "2021-01",
    "start_precision": "month",
    "end_date": "2021-12",
    "end_precision": "month"
  }
]

Question:
When did XYZ work for ABC?

```

Expected output:

```json
{
  "record_found": true,
  "record_id": "employment_000123",
  "person_name": "XYZ",
  "employer_name": "ABC",
  "start_date": "2021-01",
  "start_precision": "month",
  "end_date": "2021-12",
  "end_precision": "month",
  "answer": "XYZ worked for ABC from 2021-01 to 2021-12."
}

```

This system is easier to debug because you can inspect:

- Was the right record retrieved?
- Was the right record passed to the model?
- Did the model copy the right dates?
- Did validation pass?

With fine-tuning alone, a wrong date is much harder to diagnose.

* * *

## 17. Final recommendation

Use this division of labor:

| Component | Responsibility |
| --- | --- |
| Structured database / RAG | Store and retrieve actual employment dates |
| SFT | Teach context use, date normalization, abstention, schema-following |
| Chat template | Ensure the model sees the correct conversation format |
| Labels/masks | Ensure assistant answer tokens receive loss |
| Validator | Reject invalid date strings |
| Evaluator | Measure exact date correctness |

Do not keep searching for a model-specific date format. Use a clean format, but move the factual burden out of the model weights.

The best answer is:

```plaintext
Use YYYY-MM for month-level employment periods.
Use YYYY-MM-DD only for real day-level dates.
Verify labels, not only input_ids.
Use model-specific chat templates.
Do not expect SFT alone to reliably memorize arbitrary exact dates.
Use structured lookup or RAG for the facts.
Use SFT for behavior.
Use validation or constrained decoding for date validity.

```

* * *

## Short summary

- The date format should be consistent, preferably `YYYY-MM` for month-level data.
- If date-format cleanup did not fix the issue, the problem is likely not the date format.
- Seeing dates in `input_ids` does not prove the model is trained on them; inspect `labels != -100`.
- Qwen, Gemma, and Llama-style models need correct chat templates.
- Fine-tuning alone is weak for exact new/private factual knowledge.
- Use retrieval or structured lookup for actual employment dates.
- Fine-tune the model to use provided records, not to memorize all records.
- Add no-answer and distractor examples.
- Use JSON/schema/regex validation to prevent invalid dates.
- Evaluate with exact date match, not semantic similarity.

---

<div class="post-metadata">

**Author:** ![kegintheai](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kegintheai/32/59166_2.png) [@kegintheai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkegintheai)%5C
**Post date:** [May 22, 2026, 2:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdate-format-for-tine-tuning-ai-models%2F176116%2F7 "2026-05-22T14:48:55Z")

</div>

Brilliant pointers! Thank you so much @John6666! I’ll test them. It does indeed look like a closed-book factual-recall problem. ❤ 🙏
