Instructions to use He-Tag/smallm-125m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use He-Tag/smallm-125m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="He-Tag/smallm-125m", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("He-Tag/smallm-125m", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use He-Tag/smallm-125m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "He-Tag/smallm-125m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "He-Tag/smallm-125m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/He-Tag/smallm-125m

SGLang

How to use He-Tag/smallm-125m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "He-Tag/smallm-125m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "He-Tag/smallm-125m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "He-Tag/smallm-125m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "He-Tag/smallm-125m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use He-Tag/smallm-125m with Docker Model Runner:
```
docker model run hf.co/He-Tag/smallm-125m
```

smallm-125m / README.md

He-Tag

Update README.md

8515e6e verified about 1 month ago

preview code

Raw History Blame Contribute Delete

6.37 kB

---
license: apache-2.0
language: [en, de]
pipeline_tag: text-generation
library_name: transformers
datasets:
- HuggingFaceFW/fineweb-edu
- HuggingFaceTB/smollm-corpus
- epfml/FineWeb2-HQ
- HuggingFaceTB/smol-smoltalk
- HuggingFaceTB/smoltalk2
- mayflowergmbh/alpaca-gpt4_de
- mayflowergmbh/oasst_de
- DiscoResearch/germanrag
tags:
- small
- 125m
- bilingual
- english
- german
---
# smallm-125m

smallm-125m is a bilingual LLM trained on approximately 3.5B tokens of which 60% were English and 40% German.
The model was completely trained from scratch on 2 Kaggle T4 GPUs over the span of 28 hours.

Since all of the training was done on Kaggle, the model training was completely free.
You can check the training code out yourself at: https://github.com/He-Tag/smallm

Training was two stages: 3.49B tokens of pretraining, then instruction tuning on
162M tokens of English and German conversations. The context window is **1024 tokens**
for prompt and answer together.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("He-Tag/smallm-125m", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("He-Tag/smallm-125m", trust_remote_code=True)
messages = [{"role": "user", "content": "What is the capital city of France?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
                                       return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

Note that the model can and will hallucinate when asked for facts that are not common knowledge.

`trust_remote_code=True` is needed for **both** the model and the tokenizer: the
architecture is not one of the built-in `transformers` models. For the same reason the
inference widget on this page does not work — the Hub does not execute custom code.

The defaults in `generation_config.json` (temperature 0.3, top-p 0.9,
`repetition_penalty` 1.15, `no_repeat_ngram_size` 4) come from the evaluation rather
than from habit: at temperature 0.7 the model invents facts it gets right at 0.3, and
without n-gram blocking it loops on the prompts beyond its reach.

**Write German with proper umlauts.** "Österreich" is answered correctly with Vienna;
"Oesterreich" is read as "Osteria" and answered with Paris. The tokenizer was trained
on real German text, where that transliteration barely occurs.

## Architecture

12 layers, 12 heads, width 768, vocabulary 32,768 (byte-level BPE trained on the
training mixture), 135.3M parameters including embeddings.

Follows [modded-nanogpt](https://github.com/KellerJordan/modded-nanogpt): RMSNorm
without a learnable gain, no biases, rotary embeddings, QK-norm, ReLU² MLP,
value-residual learning, U-net skip connections between the layer halves, a per-block
embedding shortcut, zero-initialised output projections and a tanh logit softcap.
The model does **not** use bf16, instead it uses fp16 with a gradient scaler due to the T4 having no native bf16 support.

## What it can do

Measured on 22 prompts, greedy, no repetition penalty:

- Short facts with a clean ending: "Die Hauptstadt von Österreich ist Wien." in eight
  tokens, then it stops. 17 of 22 answers end on their own.
- Explanations that are correct where the topic is common — photosynthesis comes back
  with chloroplasts, chlorophyll and oxygen as a byproduct.
- Formats: numbered lists, email skeletons, summaries.
- Multi-turn: a reference back to an earlier turn holds for two or three rounds.

On Apple silicon via MPS it runs at around 90 tokens/s, and is usable on CPU.

Measured on text typed by hand, which no corpus can contain:
**0.817 bits/char for German, 0.988 for English.**

## Limitations

- **Arithmetic fails.** "3 apples plus 2" produces "3 × 2 = 6".
- **Translation does not exist.** Asked to translate a German sentence, it returns the
  German sentence.
- **Counting instructions fail.** "Count from 1 to 10" turns into an essay about the
  number 1.
- **Facts collapse beyond the most common ones.** Vienna, Berlin and Shakespeare are
  right; the largest ocean comes back as the Great Barrier Reef.
- **It hallucinates confidently** instead of declining to answer.
- **Long answers drift.** Holding an intent across several paragraphs is where it
  breaks: asked for a polite cancellation email, it writes an enthusiastic acceptance.
- Numbers in multi-turn answers are usually wrong.

These are limits of scale, not of data quality. The model saw 3.5B tokens; comparable
models of this size see several hundred billion.

Not suitable for: unverified factual lookup, translation, arithmetic, or anything with
legal, medical or financial consequences.

## License and data provenance

Weights and code are released under the Apache License 2.0.

The model was trained on the datasets below. None of them is copyleft; all require
attribution, which this section provides.

**Pretraining**
- [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) — ODC-BY 1.0
- [HuggingFaceTB/smollm-corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) (cosmopedia-v2) — ODC-BY 1.0
- [epfml/FineWeb2-HQ](https://huggingface.co/datasets/epfml/FineWeb2-HQ) (deu_Latn) — ODC-BY 1.0
**Fine-tuning**
- [HuggingFaceTB/smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) — Apache-2.0
- [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) — no license stated on the dataset card
- [mayflowergmbh/alpaca-gpt4_de](https://huggingface.co/datasets/mayflowergmbh/alpaca-gpt4_de) — none stated; derived from alpaca-gpt4 (CC BY-NC 4.0)
- [mayflowergmbh/oasst_de](https://huggingface.co/datasets/mayflowergmbh/oasst_de) — none stated; derived from OASST1 (Apache-2.0)
- [DiscoResearch/germanrag](https://huggingface.co/datasets/DiscoResearch/germanrag) — CC BY 4.0
Two things for anyone building on this:
- The English fine-tuning data contains answers generated with
  **Llama-3.1-405B-Instruct** (Smol-Magpie-Ultra, part of SmolTalk).
- One German source derives from data under **CC BY-NC 4.0**. If you need a
  commercially clean lineage, repeat stage 1 without it.
Released for research and education, without any warranty.

	---
	license: apache-2.0
	language: [en, de]
	pipeline_tag: text-generation
	library_name: transformers
	datasets:
	- HuggingFaceFW/fineweb-edu
	- HuggingFaceTB/smollm-corpus
	- epfml/FineWeb2-HQ
	- HuggingFaceTB/smol-smoltalk
	- HuggingFaceTB/smoltalk2
	- mayflowergmbh/alpaca-gpt4_de
	- mayflowergmbh/oasst_de
	- DiscoResearch/germanrag
	tags:
	- small
	- 125m
	- bilingual
	- english
	- german
	---

	# smallm-125m

	smallm-125m is a bilingual LLM trained on approximately 3.5B tokens of which 60% were English and 40% German.
	The model was completely trained from scratch on 2 Kaggle T4 GPUs over the span of 28 hours.

	Since all of the training was done on Kaggle, the model training was completely free.
	You can check the training code out yourself at: https://github.com/He-Tag/smallm

	Training was two stages: 3.49B tokens of pretraining, then instruction tuning on
	162M tokens of English and German conversations. The context window is 1024 tokens
	for prompt and answer together.

	## Usage

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model = AutoModelForCausalLM.from_pretrained("He-Tag/smallm-125m", trust_remote_code=True)
	tokenizer = AutoTokenizer.from_pretrained("He-Tag/smallm-125m", trust_remote_code=True)

	messages = [{"role": "user", "content": "What is the capital city of France?"}]
	inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
	return_tensors="pt", return_dict=True)
	out = model.generate(**inputs, max_new_tokens=200)
	print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
	```

	Note that the model can and will hallucinate when asked for facts that are not common knowledge.

	`trust_remote_code=True` is needed for both the model and the tokenizer: the
	architecture is not one of the built-in `transformers` models. For the same reason the
	inference widget on this page does not work — the Hub does not execute custom code.

	The defaults in `generation_config.json` (temperature 0.3, top-p 0.9,
	`repetition_penalty` 1.15, `no_repeat_ngram_size` 4) come from the evaluation rather
	than from habit: at temperature 0.7 the model invents facts it gets right at 0.3, and
	without n-gram blocking it loops on the prompts beyond its reach.

	Write German with proper umlauts. "Österreich" is answered correctly with Vienna;
	"Oesterreich" is read as "Osteria" and answered with Paris. The tokenizer was trained
	on real German text, where that transliteration barely occurs.

	## Architecture

	12 layers, 12 heads, width 768, vocabulary 32,768 (byte-level BPE trained on the
	training mixture), 135.3M parameters including embeddings.

	Follows [modded-nanogpt](https://github.com/KellerJordan/modded-nanogpt): RMSNorm
	without a learnable gain, no biases, rotary embeddings, QK-norm, ReLU² MLP,
	value-residual learning, U-net skip connections between the layer halves, a per-block
	embedding shortcut, zero-initialised output projections and a tanh logit softcap.
	The model does not use bf16, instead it uses fp16 with a gradient scaler due to the T4 having no native bf16 support.

	## What it can do

	Measured on 22 prompts, greedy, no repetition penalty:

	- Short facts with a clean ending: "Die Hauptstadt von Österreich ist Wien." in eight
	tokens, then it stops. 17 of 22 answers end on their own.
	- Explanations that are correct where the topic is common — photosynthesis comes back
	with chloroplasts, chlorophyll and oxygen as a byproduct.
	- Formats: numbered lists, email skeletons, summaries.
	- Multi-turn: a reference back to an earlier turn holds for two or three rounds.

	On Apple silicon via MPS it runs at around 90 tokens/s, and is usable on CPU.

	Measured on text typed by hand, which no corpus can contain:
	0.817 bits/char for German, 0.988 for English.

	## Limitations

	- Arithmetic fails. "3 apples plus 2" produces "3 × 2 = 6".
	- Translation does not exist. Asked to translate a German sentence, it returns the
	German sentence.
	- Counting instructions fail. "Count from 1 to 10" turns into an essay about the
	number 1.
	- Facts collapse beyond the most common ones. Vienna, Berlin and Shakespeare are
	right; the largest ocean comes back as the Great Barrier Reef.
	- It hallucinates confidently instead of declining to answer.
	- Long answers drift. Holding an intent across several paragraphs is where it
	breaks: asked for a polite cancellation email, it writes an enthusiastic acceptance.
	- Numbers in multi-turn answers are usually wrong.

	These are limits of scale, not of data quality. The model saw 3.5B tokens; comparable
	models of this size see several hundred billion.

	Not suitable for: unverified factual lookup, translation, arithmetic, or anything with
	legal, medical or financial consequences.

	## License and data provenance

	Weights and code are released under the Apache License 2.0.

	The model was trained on the datasets below. None of them is copyleft; all require
	attribution, which this section provides.

	Pretraining
	- [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) — ODC-BY 1.0
	- [HuggingFaceTB/smollm-corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) (cosmopedia-v2) — ODC-BY 1.0
	- [epfml/FineWeb2-HQ](https://huggingface.co/datasets/epfml/FineWeb2-HQ) (deu_Latn) — ODC-BY 1.0

	Fine-tuning
	- [HuggingFaceTB/smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) — Apache-2.0
	- [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) — no license stated on the dataset card
	- [mayflowergmbh/alpaca-gpt4_de](https://huggingface.co/datasets/mayflowergmbh/alpaca-gpt4_de) — none stated; derived from alpaca-gpt4 (CC BY-NC 4.0)
	- [mayflowergmbh/oasst_de](https://huggingface.co/datasets/mayflowergmbh/oasst_de) — none stated; derived from OASST1 (Apache-2.0)
	- [DiscoResearch/germanrag](https://huggingface.co/datasets/DiscoResearch/germanrag) — CC BY 4.0

	Two things for anyone building on this:

	- The English fine-tuning data contains answers generated with
	Llama-3.1-405B-Instruct (Smol-Magpie-Ultra, part of SmolTalk).
	- One German source derives from data under CC BY-NC 4.0. If you need a
	commercially clean lineage, repeat stage 1 without it.

	Released for research and education, without any warranty.