-
v2.0: profile-guided VectorE reduction (1.30x at S=1024)
Applied 5 spec changes from the mamba2-ssd-optimize project:
- Change 3: delete redundant B_f32 tensor_copy (pre-existing dead code)
- Change 5: pass triu as constant from framework; drop per-head nc_transpose
- Change 1: cumsum via triangular matmul (headline). Deletes tensor_tensor_scan
and its transpose tail; adds one PE matmul per chunk. Hoists A_Q broadcast
outside chunk loop. Uses Option B (ones_QQ matmul + activation) for
exp_cs_last broadcast.
- Change 2: fuse PSUM reads into scale/mask for CB_T, Y_intra, Y_off,
chunk_state (4 sub-changes, deletes one tensor_copy per site per chunk)
- Change 4: exp(-cs) via activation(scale=-1.0); deletes tensor_scalar(-1)
Also added:
- C1: assert n_groups == 1 in NeuronMamba2MixerForward.forward
- C2: factored-decay dynamic-range constraint documented in kernel + wrapper
Performance (Granite dims, trn2.3xlarge, Beta 3, isolated ssd_kernel):
| Seqlen | v1 p50 | v2.0 p50 | Speedup |
|--------|----------|----------|---------|
| 128 …
-
Upload folder using huggingface_hub
-
initial commit
