Grouped Query Attention Articles

Page 1 of 1 (3 articles)

8/11/2026 • EN

Muse Glimmer 30B Architecture Notes

Analysis of Meta's new open-weight Muse Glimmer 30B LLM, focusing on its architecture, KV-cache efficiency, and comparison to similar models.

Grouped Query Attention Kv Cache LLM Architecture Meta Muse Glimmer Sliding Window Attention

Sebastian Raschka

3/22/2026 • EN

A Visual Guide to Attention Variants in Modern LLMs

A visual guide to attention variants in modern LLMs, covering MHA, GQA, MLA, sparse attention, and hybrid architectures.

Grouped Query Attention LLM Architectures Multi Head Attention Multi Query Attention Sparse Attention

Sebastian Raschka

8/9/2025 • EN

From GPT-2 to gpt-oss: Analyzing the Architectural Advances

Analysis of OpenAI's new gpt-oss models, comparing architectural improvements from GPT-2 and examining optimizations like MXFP4 and Mixture-of-Experts.

Grouped Query Attention LLM Optimization Mixture Of Experts Rope Embeddings Transformer Architecture

Sebastian Raschka

Grouped Query Attention Articles

Muse Glimmer 30B Architecture Notes

A Visual Guide to Attention Variants in Modern LLMs

From GPT-2 to gpt-oss: Analyzing the Architectural Advances

Select Language

We use cookies