A workflow for understanding open-weight LLM architectures using config files and code from Hugging Face.
A workflow for understanding new open-weight LLM architectures using config files and code from Hugging Face.
A visual guide to attention variants in modern LLMs, covering MHA, GQA, MLA, sparse attention, and hybrid architectures.
An overview of alternative LLM architectures beyond standard transformers, including linear attention hybrids, text diffusion models, and code world models.
An overview of alternative LLM architectures beyond standard transformers, including linear attention hybrids, text diffusion models, and world models.
