Publish Q16.48 tiny-GPT reference model and validation card
Browse filesAdds the 115,576-byte model.mgw reference weights, full model card, and Apache-2.0 license.
The uploaded model is byte-identical to the file committed in nmicic/int-llm (SHA-256: 466cfe9dba7b888cdaa23dedf4b10351826795793448c8e95dcb0f7a61ed33eb).
Full training reproduced the same file on arm64 macOS, AMD and Intel x86-64 Linux, and a 32-bit ARMv6 Raspberry Pi 1.
The model card documents the custom MGW format, training-data provenance, intended use, limitations, and validation across seven 32-bit MCU boards in eight
ISA-mode configurations, plus an 8-bit AVR determinism-only result.
This is a custom C artifact for the small character GPT, not a Transformers/safetensors checkpoint or TinyLlama weights.
README.md
CHANGED
|
@@ -11,6 +11,7 @@ tags:
|
|
| 11 |
- rp2040
|
| 12 |
- esp32
|
| 13 |
- nrf52840
|
|
|
|
| 14 |
- fixed-point
|
| 15 |
- integer-only
|
| 16 |
- deterministic
|
|
@@ -21,7 +22,7 @@ tags:
|
|
| 21 |
- custom-code
|
| 22 |
---
|
| 23 |
|
| 24 |
-
# int-llm `model.mgw` β an integer-only GPT that runs bit-identically from x86 servers down to 32-bit microcontrollers
|
| 25 |
|
| 26 |
This repo hosts the committed reference weights of
|
| 27 |
**[int-llm](https://github.com/nmicic/int-llm)**: a tiny character-level GPT
|
|
@@ -47,7 +48,7 @@ Background and motivation: [the int-llm blog post](https://huggingface.co/blog/n
|
|
| 47 |
| Model | character-level GPT: `N_EMBD=32, N_HEAD=4, N_LAYER=1`, 14,272 parameters (9 weight tensors) |
|
| 48 |
| Weights | MGW v1 indexed container; 9 `int64_t` Q16.48 weight tensors plus tokenizer alphabet and RNG state |
|
| 49 |
| Training | 5000 steps on the public *makemore* names dataset, integer-only (`./gpt_int --save model.mgw`) |
|
| 50 |
-
| Math contract | determinism-gate golden hash `c0d933ea340452ec` β reproduced by every backend and every validated target below |
|
| 51 |
| License | Apache-2.0 |
|
| 52 |
|
| 53 |
The `.mgw` format (MGW v1) is a small indexed container: a 64-byte header
|
|
@@ -89,7 +90,9 @@ floating-point tolerance to hide behind.
|
|
| 89 |
The train-big/run-small loop closes on microcontrollers: train on a laptop,
|
| 90 |
run inference-only from this weight file on a $5 board. Every target below
|
| 91 |
passed both checks (determinism-grid golden hash + byte-identical 20-sample
|
| 92 |
-
inference)
|
|
|
|
|
|
|
| 93 |
[`validation/cpu/`](https://github.com/nmicic/int-llm/tree/main/validation/cpu).
|
| 94 |
|
| 95 |
| target | ISA | determinism | 20 samples |
|
|
@@ -101,7 +104,9 @@ inference); raw provenance-stamped transcripts live in
|
|
| 101 |
| Heltec V3 (ESP32-S3, LX7 @ 240 MHz) | Xtensa | 3.9 s | 1.1 s |
|
| 102 |
| LILYGO T-Beam (ESP32, LX6 @ 240 MHz) | Xtensa | 4.2 s | 2.6 s |
|
| 103 |
| XIAO nRF52840 (Cortex-M4F @ 64 MHz) | Armv7E-M | 13.9 s | 3.4 s |
|
| 104 |
-
|
|
|
|
|
|
|
|
| 105 |
| AMD Ryzen 7 7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
|
| 106 |
| Intel Core i7-7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
|
| 107 |
|
|
@@ -109,11 +114,13 @@ Same die, two ISAs: the Pico 2 reproduces the identical output in both its
|
|
| 109 |
ARM and RISC-V boot modes. The Linux rows additionally rerun the **full
|
| 110 |
training** and reproduce this repo's `model.mgw` byte-for-byte.
|
| 111 |
|
| 112 |
-
**Footprint** (XIAO RP2040
|
| 113 |
187,932 B of flash β *including* the whole 115 KB weight file baked into
|
| 114 |
rodata β and 15,268 B of static RAM (5.8% of the RP2040's 264 KB). Weights
|
| 115 |
are read in place over XIP flash; RAM holds only KV cache, activations, and
|
| 116 |
-
the USB stack.
|
|
|
|
|
|
|
| 117 |
|
| 118 |
## Run it
|
| 119 |
|
|
|
|
| 11 |
- rp2040
|
| 12 |
- esp32
|
| 13 |
- nrf52840
|
| 14 |
+
- avr
|
| 15 |
- fixed-point
|
| 16 |
- integer-only
|
| 17 |
- deterministic
|
|
|
|
| 22 |
- custom-code
|
| 23 |
---
|
| 24 |
|
| 25 |
+
# int-llm `model.mgw` β an integer-only GPT that runs bit-identically from x86 servers down to 32-bit microcontrollers, its arithmetic verified down to an 8-bit AVR
|
| 26 |
|
| 27 |
This repo hosts the committed reference weights of
|
| 28 |
**[int-llm](https://github.com/nmicic/int-llm)**: a tiny character-level GPT
|
|
|
|
| 48 |
| Model | character-level GPT: `N_EMBD=32, N_HEAD=4, N_LAYER=1`, 14,272 parameters (9 weight tensors) |
|
| 49 |
| Weights | MGW v1 indexed container; 9 `int64_t` Q16.48 weight tensors plus tokenizer alphabet and RNG state |
|
| 50 |
| Training | 5000 steps on the public *makemore* names dataset, integer-only (`./gpt_int --save model.mgw`) |
|
| 51 |
+
| Math contract | determinism-gate golden hash `c0d933ea340452ec` β reproduced by every backend and every validated target below, down to an 8-bit AVR |
|
| 52 |
| License | Apache-2.0 |
|
| 53 |
|
| 54 |
The `.mgw` format (MGW v1) is a small indexed container: a 64-byte header
|
|
|
|
| 90 |
The train-big/run-small loop closes on microcontrollers: train on a laptop,
|
| 91 |
run inference-only from this weight file on a $5 board. Every target below
|
| 92 |
passed both checks (determinism-grid golden hash + byte-identical 20-sample
|
| 93 |
+
inference), except the 8-bit Mega 2560 β a board over a decade old β which
|
| 94 |
+
runs the determinism gate only (8 KB RAM), computing the same 64-bit hash
|
| 95 |
+
from 8-bit ALU instructions; raw provenance-stamped transcripts live in
|
| 96 |
[`validation/cpu/`](https://github.com/nmicic/int-llm/tree/main/validation/cpu).
|
| 97 |
|
| 98 |
| target | ISA | determinism | 20 samples |
|
|
|
|
| 104 |
| Heltec V3 (ESP32-S3, LX7 @ 240 MHz) | Xtensa | 3.9 s | 1.1 s |
|
| 105 |
| LILYGO T-Beam (ESP32, LX6 @ 240 MHz) | Xtensa | 4.2 s | 2.6 s |
|
| 106 |
| XIAO nRF52840 (Cortex-M4F @ 64 MHz) | Armv7E-M | 13.9 s | 3.4 s |
|
| 107 |
+
| Arduino MKR Zero (SAMD21, Cortex-M0+ @ 48 MHz, 256 KB flash / 32 KB RAM) | Armv6-M | 37.2 s | 26.4 s |
|
| 108 |
+
| Arduino Mega 2560 (ATmega2560, 8-bit AVR @ 16 MHz, 8 KB RAM) | AVR | 747.9 s | n/a (8 KB RAM) |
|
| 109 |
+
| Raspberry Pi 1 B+ (2014, ARMv6, 32-bit Linux) | ARMv6 | 0.7 s | 0.2 s |
|
| 110 |
| AMD Ryzen 7 7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
|
| 111 |
| Intel Core i7-7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
|
| 112 |
|
|
|
|
| 114 |
ARM and RISC-V boot modes. The Linux rows additionally rerun the **full
|
| 115 |
training** and reproduce this repo's `model.mgw` byte-for-byte.
|
| 116 |
|
| 117 |
+
**Footprint** (XIAO RP2040): the complete firmware is
|
| 118 |
187,932 B of flash β *including* the whole 115 KB weight file baked into
|
| 119 |
rodata β and 15,268 B of static RAM (5.8% of the RP2040's 264 KB). Weights
|
| 120 |
are read in place over XIP flash; RAM holds only KV cache, activations, and
|
| 121 |
+
the USB stack. The 256 KB claim is measured, not extrapolated: on the
|
| 122 |
+
Arduino MKR Zero's SAMD21 (256 KB flash / 32 KB RAM), code plus the whole
|
| 123 |
+
weight file build to 133,472 B β 50.9% of flash β with 13,124 B of RAM.
|
| 124 |
|
| 125 |
## Run it
|
| 126 |
|