Qwen2.5-7B-ODA-Mixture-100k

Qwen2.5-7B-ODA-Mixture-100k is a supervised fine-tuned (SFT) model built on top of Qwen2.5-7B-Base, trained with ODA-Mixture-100k. This training set is curated by mixing top-performing open corpora selected via the OpenDataArena leaderboard, and refined through deduplication and benchmark decontamination, aiming to improve the model’s general capabilities across General, Math, Code, and Reasoning domains under a compact ~100K data budget.

---

🧠 Model Summary

• Base Model: Qwen/Qwen2.5-7B-Base
• Training Data: OpenDataArena/ODA-Mixture-100k
• Domain Coverage: General, Math, Code, Reasoning
• Scale (selected training set): ~100K samples
• Goal: Achieve significant general-purpose gains with a compact curated dataset, improving multi-domain reasoning and problem-solving ability.

---

⚙️ Training Data Curation Pipeline

ODA-Mixture-100k is built by following a single rule: trust the OpenDataArena leaderboard.

1️⃣ Data Collection

We chose LIMO as our foundation because it achieves a high ranking on the ODA overall leaderboard with very few samples. This efficiency allows us to establish a strong reasoning baseline. We then augment this core with AM-Thinking-v1-Distilled-math and AM-Thinking-v1-Distilled-code, the top-performing and efficient datasets on the ODA Math and Code leaderboards, to enhance specialized domain capabilities.

2️⃣ Deduplication & Decontamination

We first perform exact deduplication over all questions to remove identical items, and then run benchmark decontamination to reduce evaluation leakage by removing overlaps with standard and competition benchmarks.

3️⃣ Data Selection

To adhere to our ~100K data budget while maximizing the impact of each sample, we employ semantic clustering to map the overall data distribution. Within each cluster, we preferentially sample the most challenging instances, using sequence length as a practical proxy for reasoning complexity and problem difficulty.

---

📚 Training Data Source Composition

Source	Count	Percentage
LIMO	817	0.81%
AM-Thinking-Distilled-math	50,244	49.59%
AM-Thinking-Distilled-code	50,245	49.60%

---

🧩 Data Format

The training data sample format is as follows (aligned with the dataset schema):

{
  "id": "unique_identifier",
  "source": "data source",
  "question": "textual question or instruction",
  "response": "textual response"
}

---

📈 Performance

Qwen2.5-7B-ODA-Mixture-100k is evaluated as an SFT model built on Qwen2.5-7B-Base across the full ODA benchmark suite spanning four domains:

• General (DROP, IFEVAL, AGIEVAL, MMLU-Pro)
• Math (GSM8K, MATH500, Omni-Math, OlympiadBench, AIME2024)
• Code (HumanEval, MBPP, LCB (V5), HumanEval+)
• Reasoning (ARC-C, BBH, CALM, KOR-BENCH).

We observe consistent improvements over the base checkpoint, with particularly strong gains on several benchmarks.

Leaderboard Performance Comparison. Best scores in bold, second-best underlined. Eff. denotes Data Efficiency.

Model / Training Data	Size	Eff.	General	Math	Code	Reasoning	AVG
Qwen2.5-7B-Base
Qwen2.5-7B-Base	-	-	51.4	39.8	50.1	42.7	46.0
OpenThoughts3-1.2M	1.2M	+0.011	45.5	71.8	67.0	54.3	59.6
OmniThought-0528	365k	+0.027	47.1	71.2	47.6	57.2	55.8
SYNTHETIC-2-SFT-verified	105k	+0.086	51.3	69.8	40.1	58.9	55.0
AM-Thinking-v1-Distilled-math	558k	+0.016	57.7	77.4	39.5	44.8	54.8
LIMO	817	+9.920	60.7	44.0	57.9	53.8	54.1
MiroMind-M1-SFT-719K	719k	+0.006	52.0	71.0	26.3	51.5	50.2
AM-Thinking-v1-Distilled-code	324k	+0.024	49.9	52.3	68.7	44.4	53.8
Light-R1-SFTData	79k	+0.084	55.5	64.4	38.8	51.9	52.7
ODA-Mixture-500k	500k	+0.039	63.4	72.8	66.7	59.6	65.6
ODA-Mixture-100k	100k	+0.149	56.8	71.2	64.4	51.5	61.0

---

🌐 About OpenDataArena

OpenDataArena is an open research platform dedicated to discovering, evaluating, and advancing high-quality datasets for AI post-training. It provides a transparent, data-centric ecosystem to support reproducible dataset evaluation and sharing.

Key Features:

• 🏆 Dataset Leaderboard — helps researchers identify the most valuable and high-quality datasets across different domains
• 📊 Detailed Evaluation Scores — provides comprehensive metrics to assess data quality, complexity, difficulty, etc.
• 🧰 Data Processing Toolkit — OpenDataArena-Tool offers an open-source pipeline for dataset curation and scoring.

---

🚀 Usage

Model repo: OpenDataArena/Qwen2.5-7B-ODA-Mixture-100k. Below is a minimal runnable example for loading and inference:

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "OpenDataArena/Qwen2.5-7B-ODA-Mixture-100k"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, device_map="auto", trust_remote_code=True)

messages = [
    {"role": "user", "content": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📚 Citation

If you use this model or its training data (ODA-Mixture-100k), please cite:

@article{cai2025opendataarena,
  title={OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value},
  author={Cai, Mengzhang and Gao, Xin and Li, Yu and Lin, Honglin and Liu, Zheng and Pan, Zhuoshi and Pei, Qizhi and Shang, Xiaoran and Sun, Mengyuan and Tang, Zinan and others},
  journal={arXiv preprint arXiv:2512.14051},
  year={2025}
}