M2-2B

1.93B-parameter AwareLiquid M2 preset (2080 width × 34 layers × 16 heads), the M2 experimental line's 2b configuration trained via m2_training. Byte-level tokenizer (UTF-8 bytes, vocab 256).

Status at upload: research milestone — 200,000-step FINAL checkpoint (val PPL 2.4106). model.safetensors holds the 200K weights; the raw resumable PyTorch checkpoint is t2b_200k.pt. Training status (2026-09-28): the baseline run COMPLETED at 200,000 steps, final val PPL 2.4106. Full milestone table below; the 200K weights live in the M2 repo (docs/RESULTS_2B_30K.md) and on the training server (/root/M2/runs/t2b_30k.pt, 3.86GB bf16). The 200K raw checkpoint is now in this repo as t2b_200k.pt (PyTorch bf16, weight-tied; the matching 200K-weight model.safetensors export is in this repo since 2026-09-29).

Measured

Milestone	val PPL (byte-level, WikiText-103)
10K steps (feasibility)	4.53
30K steps	2.73
60K steps	2.556
90K steps	2.482
150K steps	2.413
200K steps (final, 09-28)	2.4106

Convergence: 30K→60K −6.4%, 60K→90K −2.9%, 90K→150K −2.8%, 150K→200K −0.1% (saturated).

Context-flat: PPL 2.72 / 2.72 / 2.68 at seq 128 / 256 / 512 (no length degradation). Training: bf16 model, AdamW8bit, lr 1e-4, seq 128, batch 8 × grad_accum 4, on a single A100-PCIE-40GB.

Boundaries (honest)

• Byte-level, not a subword model — PPL is on UTF-8 byte tokens; NOT comparable to gpt2-vocab models. Serve with byte tokenization.
• Not an instruct/chat model — raw next-token pretraining only.
• Research checkpoint — bf16 safetensors (weight-tied embedding; the tied lm_head.weight is omitted, standard HF re-tie on load). No optimizer state: not exactly resumable.
• The M2 line's verdict so far is honest-null on reasoning mechanisms; this checkpoint is the language-line scaling artifact, not a capability claim.

SFT checkpoints (post-training experiments, 2026-10)

file	recipe	val PPL	JevBench public (official harness)
t2b_sft_v1.pt	alpaca-short 5.4k pairs (<=128B), 3 epochs, lr 1e-4	3.95	-
t2b_sft_v2.pt	alpaca-short512 19.6k pairs (<=512B), 1+5k steps, lr 2e-5, rope_scale 4	3.20	raw 31.3% (48 easy + 72 original decisions)

Boundaries: instruction-SFT on short pairs; held-out generation went from 0/8 (base) to 4/8 format-correct, but decision-task accuracy remains the open problem (best family configuration: 2B frozen core + decision head, 39.6% easy-tier raw / ECE 0.20 / 0.168 s - see github.com/AwareLiquid/O1-Flash). Serving interface for all of the above: byte-level, max_seq_len per config.

Config

config.json carries the full MTLNNConfig (d_model 2080, 34 layers, 16 heads, gwtb_n_heads=1, vocab 256, max_seq_len 129).