Learn2Zinc Models

Learn2Zinc is a a family of small language models fine-tuned with LoRA for translating natural-language optimization problems into executable MiniZinc code. All models were trained on the learn2zinc_augmented dataset using the Unsloth library.

Models

Model	Base Model	Parameters	Chat Format
learn2zinc-GPT-oss-20B	GPT-OSS-20B	20B	Harmony (custom)
learn2zinc-Gemma-2-9B	Gemma 2 9B	9B	Standard chat template
learn2zinc-Llama-3.2-3B	Llama 3.2 3B	3B	Standard chat template
learn2zinc-Llama-3.2-1B	Llama 3.2 1B	1B	Standard chat template
learn2zinc-Qwen3-0.6B	Qwen3 0.6B	0.6B	Standard chat template

Shared Training Configuration

All models share the same LoRA and training setup:

Hyperparameter	Value
Fine-tuning method	LoRA (rank 64, alpha 64)
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning rate	2e-4 (cosine schedule, 50 warmup steps)
Epochs	3
Optimizer	AdamW 8-bit
Weight decay	0.01
Precision	bf16
Quantization during training	4-bit
Max sequence length	4096
Training	Response-only (SFTTrainer)

Evaluation

Models were evaluated on the IndustryOR subset of the learn2zinc benchmark. Generated MiniZinc code was executed with the HiGHS solver (120 s timeout). All generations used temperature = 0 for reproducibility.

Metrics: Execution Success Rate (code compiles and runs) and Solution Correctness (objective matches expected value within 1e-6).

For full evaluation details, see the learn2zinc GitHub repo.

Datasets

All models were trained on datasets from the Learn2Zinc collection:

Dataset	Strategy	Examples
learn2zinc-base	Direct generation	8,014
learn2zinc-cot	Chain-of-thought + generation	8,014
learn2zinc-augmented	Generation + correction	15,649

Framework

• Unsloth
• PEFT / LoRA
• TRL SFTTrainer

Citation

@misc{kadioglu2026modelingcopilotstexttomodeltranslation,
      title={Modeling Copilots for Text-to-Model Translation}, 
      author={Serdar Kadioglu and Karthik Uppuluri and Akash Singirikonda},
      year={2026},
      eprint={2604.12955},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2604.12955}, 
}