A practical breakdown of how memory, MTP, output-head precision, and runtime stack affect real inference speed across NVFP4 model variants.
Adapted from @Oluwaphilemon1I spent some time comparing Qwen3.8-27B NVFP4 variants, and the interesting part is that accuracy isn’t really separating them. The bigger differences show up in memory usage and inference speed. If you’re VRAM-constrained, minima-ai/mnma_qwen3.8_27b_nvfp4 is an interesting option. The tradeoff is that it doesn’t come with MTP, so you miss out on speculative decoding for potentially faster generation. NVIDIA’s NVFP4 build does include MTP. But in my long-context coding tests, MTP-4 was only around 2× faster than running without MTP. That’s decent, but it still left NVIDIA’s implementation roughly 2.5–3× behind Unsloth on an RTX Pro 6000. So the quantization format itself isn’t telling the whole story. The inference stack matters. One possible explanation is the treatment of the model’s output head. NVIDIA quantizes the lm_head to NVFP4, while Unsloth keeps it at FP8. That’s potentially important for MTP because the speculative tokens still have to be evaluated against the target model, and MTP shares that target model lm_head. If the lower-precision output head makes predictions less reliable, the speculative draft can get rejected more often. Lower acceptance means less of the theoretical MTP speedup actually reaches the final decode rate. That’s still a hypothesis, not something I’ve proven experimentally. But it would explain why two builds that look very similar on paper can behave very differently once you put them through long-context coding. For me, the takeaway is pretty simple: Don’t choose a quant solely from the model name or advertised precision. Look at the entire inference path. Weights. Output head. MTP. KV cache. Runtime. Context length. Acceptance rate. All of these can matter. For my current setup, I’m sticking with the Unsloth NVFP4 build. Now I’m moving on to the more interesting test: How does it actually hold up when you give it long-horizon agentic coding tasks? That’s where I expect the differences between these builds to become much easier to see.
