Gemma 4 Architecture and Benchmark Notes

This article provides a detailed technical analysis of Google's Gemma 4 language model, focusing on its architecture and benchmark performance. It notes that the 31B dense model retains the Gemma-style local-global attention mechanism with a 5:1 sliding-window ratio, grouped-query attention, QK-Norm, and RMSNorm blocks, supporting a 256k-token context and 262k-token vocabulary. The performance improvement over Gemma 3 is attributed more to training data and recipe than architectural changes. Benchmark comparisons place Gemma 4 31B close to Qwen3.5-27B. The article also mentions a sparse MoE variant (26B-A4B) and highlights the favorable Apache 2.0 licensing.