Qwen3-0.6B: Technical Overview and Usage

Model Highlights

• Type: Causal Language Model with 0.6B parameters (0.44B non-embedding).
• Architecture: 28 layers, 16 GQA attention heads (8 for KV), 32,768 context length.
• Key Features:
  • Seamless switch between thinking (complex reasoning) and non-thinking (efficient dialogue) modes.
  • Enhanced reasoning in math, coding, and logic, surpassing QwQ and Qwen2.5.
  • Superior human preference alignment in creative and multi-turn dialogues.
  • Expertise in agent capabilities for tool integration.
  • Supports 100+ languages with strong multilingual instruction following.

Performance and Comparisons

• Outperforms QwQ in thinking mode and Qwen2.5 in non-thinking mode.
• Excels in creative writing, role-playing, and instruction following.
• Leading performance in agent-based tasks among open-source models.

Quickstart Instructions

Installation: Ensure transformers>=4.51.0 to avoid KeyError: 'qwen3'.

Code Example:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen3-0.6B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

prompt = "Explain large language models."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=32768)
output = tokenizer.decode(generated_ids[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(output)

Deployment:

• SGLang: python -m sglang.launch_server --model-path Qwen/Qwen3-0.6B --reasoning-parser qwen3
• vLLM: vllm serve Qwen/Qwen3-0.6B --enable-reasoning --reasoning-parser qwen3

Thinking Mode Control:

• Default: enable_thinking=True (generates <think>...</think> blocks).
• Disable: enable_thinking=False for non-thinking mode.
• Soft switch: Use /think or /no_think in prompts for dynamic control.

Best Practices:

• Thinking Mode: Temperature=0.6, TopP=0.95, TopK=20, MinP=0. Avoid greedy decoding.
• Non-Thinking Mode: Temperature=0.7, TopP=0.8, TopK=20, MinP=0.
• Set presence_penalty=1.5 to reduce repetitions.
• Use max_new_tokens=32768 for most queries; 38,912 for complex tasks.

Agentic Use:
Integrate with Qwen-Agent for tool-calling capabilities. Example:

from qwen_agent.agents import Assistant

llm_cfg = {'model': 'Qwen3-0.6B', 'model_server': 'http://localhost:8000/v1', 'api_key': 'EMPTY'}
tools = [{'code_interpreter'}]
bot = Assistant(llm=llm_cfg, function_list=tools)
messages = [{'role': 'user', 'content': 'Analyze this data: ...'}]
response = bot.run(messages)

Resources:

• Blog
• GitHub
• Documentation

This summary provides essential technical details, performance comparisons, and practical instructions for using Qwen3-0.6B effectively.