DustinEwan/muse-glimmer-30b-nvfp4-dflash-fp4

Muse-Glimmer-30B NVFP4 + DFlash FP4 — radiance container

nvidia/Muse-Glimmer-30B-NVFP4 as a single .rad container for the radiance inference engine, with Meta's Muse-Glimmer-30B-assistant DFlash drafter merged in for speculative decoding. The target is the NVIDIA ModelOpt NVFP4 mixed-precision quant of Meta's Muse-Glimmer-30B.

File	muse-glimmer-30b-nvfp4-dflash-fp4.rad — 23.2 GiB
Weights	Target: mixed W4A16 NVFP4 / FP8 / BF16 fallback as shipped by nvidia/Muse-Glimmer-30B-NVFP4. Perception encoder and unmatched modules remain BF16; KV cache unquantized at serve time.
Speculator	DFlash, drafting 15 tokens a step. Drafter linears and head are MXFP4 / FP8 as per the container recipe; the drafter only proposes, the model verifies every token it emits.
Context	131,072 trained, served with --max-model-len 524288
Modalities	Input: text + image, Output: text
Needs	radiance 1.17.0 or newer, museglimmer architecture plugin

Serve

radiance --model muse-glimmer-30b-nvfp4-dflash-fp4.rad \
  --served-model-name museglimmer \
  --radiance-home /build/radiance-home-tp2 \
  --tp 2 --max-model-len 524288 --max-num-seqs 4 \
  --kv-cache-dtype fp8 --num-speculative-tokens 15 --draft-short-from 2 \
  --gpu-headroom-mib 120 --host 0.0.0.0 --port 8086 \
  --temp 0.7 --top-p 0.95

The container is built for AMD RDNA4 + ROCm. The server speaks the OpenAI API, with tool calls and structured output.

How this file was made

rad-convert nvidia/Muse-Glimmer-30B-NVFP4 \
  --draft-model meta-models/Muse-Glimmer-30B-assistant \
  --recipe muse-glimmer-30b-nvfp4-dflash-fp4.recipe \
  -o muse-glimmer-30b-nvfp4-dflash-fp4.rad

The recipe quantises the language-model projections and lm_head to W4A16 NVFP4 / FP8 where AutoQuant selected, keeps the perception encoder BF16, and merges the DFlash drafter under draft. with MXFP4 / FP8 weights. The resulting container contains both target and drafter weights, so speculative decoding runs from a single file.

Source models

• Target: nvidia/Muse-Glimmer-30B-NVFP4 — ModelOpt AutoQuant mixed W4A16 NVFP4 / FP8 / BF16 of meta-models/Muse-Glimmer-30B
• Drafter: meta-models/Muse-Glimmer-30B-assistant — DFlash block drafter used by Muse Glimmer

License: Apache-2.0 for both source checkpoints. This container inherits Apache-2.0.