nmicic commited on
Commit
9d480ec
Β·
verified Β·
1 Parent(s): 49a685e

Publish Q16.48 tiny-GPT reference model and validation card

Browse files

Adds the 115,576-byte model.mgw reference weights, full model card, and Apache-2.0 license.

The uploaded model is byte-identical to the file committed in nmicic/int-llm (SHA-256: 466cfe9dba7b888cdaa23dedf4b10351826795793448c8e95dcb0f7a61ed33eb).
Full training reproduced the same file on arm64 macOS, AMD and Intel x86-64 Linux, and a 32-bit ARMv6 Raspberry Pi 1.

The model card documents the custom MGW format, training-data provenance, intended use, limitations, and validation across seven 32-bit MCU boards in eight
ISA-mode configurations, plus an 8-bit AVR determinism-only result.

This is a custom C artifact for the small character GPT, not a Transformers/safetensors checkpoint or TinyLlama weights.

Files changed (1) hide show
  1. README.md +13 -6
README.md CHANGED
@@ -11,6 +11,7 @@ tags:
11
  - rp2040
12
  - esp32
13
  - nrf52840
 
14
  - fixed-point
15
  - integer-only
16
  - deterministic
@@ -21,7 +22,7 @@ tags:
21
  - custom-code
22
  ---
23
 
24
- # int-llm `model.mgw` β€” an integer-only GPT that runs bit-identically from x86 servers down to 32-bit microcontrollers
25
 
26
  This repo hosts the committed reference weights of
27
  **[int-llm](https://github.com/nmicic/int-llm)**: a tiny character-level GPT
@@ -47,7 +48,7 @@ Background and motivation: [the int-llm blog post](https://huggingface.co/blog/n
47
  | Model | character-level GPT: `N_EMBD=32, N_HEAD=4, N_LAYER=1`, 14,272 parameters (9 weight tensors) |
48
  | Weights | MGW v1 indexed container; 9 `int64_t` Q16.48 weight tensors plus tokenizer alphabet and RNG state |
49
  | Training | 5000 steps on the public *makemore* names dataset, integer-only (`./gpt_int --save model.mgw`) |
50
- | Math contract | determinism-gate golden hash `c0d933ea340452ec` β€” reproduced by every backend and every validated target below |
51
  | License | Apache-2.0 |
52
 
53
  The `.mgw` format (MGW v1) is a small indexed container: a 64-byte header
@@ -89,7 +90,9 @@ floating-point tolerance to hide behind.
89
  The train-big/run-small loop closes on microcontrollers: train on a laptop,
90
  run inference-only from this weight file on a $5 board. Every target below
91
  passed both checks (determinism-grid golden hash + byte-identical 20-sample
92
- inference); raw provenance-stamped transcripts live in
 
 
93
  [`validation/cpu/`](https://github.com/nmicic/int-llm/tree/main/validation/cpu).
94
 
95
  | target | ISA | determinism | 20 samples |
@@ -101,7 +104,9 @@ inference); raw provenance-stamped transcripts live in
101
  | Heltec V3 (ESP32-S3, LX7 @ 240 MHz) | Xtensa | 3.9 s | 1.1 s |
102
  | LILYGO T-Beam (ESP32, LX6 @ 240 MHz) | Xtensa | 4.2 s | 2.6 s |
103
  | XIAO nRF52840 (Cortex-M4F @ 64 MHz) | Armv7E-M | 13.9 s | 3.4 s |
104
- | Raspberry Pi 1 B+ (ARMv6, 32-bit Linux) | ARMv6 | 0.7 s | 0.2 s |
 
 
105
  | AMD Ryzen 7 7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
106
  | Intel Core i7-7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
107
 
@@ -109,11 +114,13 @@ Same die, two ISAs: the Pico 2 reproduces the identical output in both its
109
  ARM and RISC-V boot modes. The Linux rows additionally rerun the **full
110
  training** and reproduce this repo's `model.mgw` byte-for-byte.
111
 
112
- **Footprint** (XIAO RP2040, the smallest target): the complete firmware is
113
  187,932 B of flash β€” *including* the whole 115 KB weight file baked into
114
  rodata β€” and 15,268 B of static RAM (5.8% of the RP2040's 264 KB). Weights
115
  are read in place over XIP flash; RAM holds only KV cache, activations, and
116
- the USB stack. Code + weights comfortably fit 256 KB-flash-class parts.
 
 
117
 
118
  ## Run it
119
 
 
11
  - rp2040
12
  - esp32
13
  - nrf52840
14
+ - avr
15
  - fixed-point
16
  - integer-only
17
  - deterministic
 
22
  - custom-code
23
  ---
24
 
25
+ # int-llm `model.mgw` β€” an integer-only GPT that runs bit-identically from x86 servers down to 32-bit microcontrollers, its arithmetic verified down to an 8-bit AVR
26
 
27
  This repo hosts the committed reference weights of
28
  **[int-llm](https://github.com/nmicic/int-llm)**: a tiny character-level GPT
 
48
  | Model | character-level GPT: `N_EMBD=32, N_HEAD=4, N_LAYER=1`, 14,272 parameters (9 weight tensors) |
49
  | Weights | MGW v1 indexed container; 9 `int64_t` Q16.48 weight tensors plus tokenizer alphabet and RNG state |
50
  | Training | 5000 steps on the public *makemore* names dataset, integer-only (`./gpt_int --save model.mgw`) |
51
+ | Math contract | determinism-gate golden hash `c0d933ea340452ec` β€” reproduced by every backend and every validated target below, down to an 8-bit AVR |
52
  | License | Apache-2.0 |
53
 
54
  The `.mgw` format (MGW v1) is a small indexed container: a 64-byte header
 
90
  The train-big/run-small loop closes on microcontrollers: train on a laptop,
91
  run inference-only from this weight file on a $5 board. Every target below
92
  passed both checks (determinism-grid golden hash + byte-identical 20-sample
93
+ inference), except the 8-bit Mega 2560 β€” a board over a decade old β€” which
94
+ runs the determinism gate only (8 KB RAM), computing the same 64-bit hash
95
+ from 8-bit ALU instructions; raw provenance-stamped transcripts live in
96
  [`validation/cpu/`](https://github.com/nmicic/int-llm/tree/main/validation/cpu).
97
 
98
  | target | ISA | determinism | 20 samples |
 
104
  | Heltec V3 (ESP32-S3, LX7 @ 240 MHz) | Xtensa | 3.9 s | 1.1 s |
105
  | LILYGO T-Beam (ESP32, LX6 @ 240 MHz) | Xtensa | 4.2 s | 2.6 s |
106
  | XIAO nRF52840 (Cortex-M4F @ 64 MHz) | Armv7E-M | 13.9 s | 3.4 s |
107
+ | Arduino MKR Zero (SAMD21, Cortex-M0+ @ 48 MHz, 256 KB flash / 32 KB RAM) | Armv6-M | 37.2 s | 26.4 s |
108
+ | Arduino Mega 2560 (ATmega2560, 8-bit AVR @ 16 MHz, 8 KB RAM) | AVR | 747.9 s | n/a (8 KB RAM) |
109
+ | Raspberry Pi 1 B+ (2014, ARMv6, 32-bit Linux) | ARMv6 | 0.7 s | 0.2 s |
110
  | AMD Ryzen 7 7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
111
  | Intel Core i7-7700 (Linux, gcc) | x86-64 | native + portable | byte-identical |
112
 
 
114
  ARM and RISC-V boot modes. The Linux rows additionally rerun the **full
115
  training** and reproduce this repo's `model.mgw` byte-for-byte.
116
 
117
+ **Footprint** (XIAO RP2040): the complete firmware is
118
  187,932 B of flash β€” *including* the whole 115 KB weight file baked into
119
  rodata β€” and 15,268 B of static RAM (5.8% of the RP2040's 264 KB). Weights
120
  are read in place over XIP flash; RAM holds only KV cache, activations, and
121
+ the USB stack. The 256 KB claim is measured, not extrapolated: on the
122
+ Arduino MKR Zero's SAMD21 (256 KB flash / 32 KB RAM), code plus the whole
123
+ weight file build to 133,472 B β€” 50.9% of flash β€” with 13,124 B of RAM.
124
 
125
  ## Run it
126