Sebastian Raschka • 3/12/2026

Nemotron 3 Super Throughput Notes

This article discusses NVIDIA's Nemotron 3 Super 120B-A12B, an open-weight AI model designed for optimal accuracy-throughput trade-off. It highlights key efficiency choices including Mamba-2 layers, Latent MoE layers, shared-weight multi-token prediction, and GQA layers. The model is benchmarked against GPT-OSS 120B and Qwen3.5, showing competitive accuracy with stronger throughput. The article emphasizes its relevance for local agentic applications where latency and cost are critical.

0 comments

#Mixture Of Experts #Speculative Decoding #Gqa