SebastianRaschka.com is the personal blog of Sebastian Raschka, PhD, an LLM research engineer whose work bridges academia and industry in AI and machine learning. On his blog and notes section he publishes deep, well-documented articles on topics such as LLMs (large language models), reasoning models, machine learning in Python, neural networks, data science workflows, and deep learning architecture. Recent posts explore advanced themes like “reasoning LLMs”, comparisons of modern open-weight transformer architectures, and guides for building, training, or analyzing neural networks and model internals.
130 articles from this blog
Analysis of Kimi K3's open-weight architecture, focusing on LatentMoE, attention residuals, and NoPE innovations for inference efficiency.
Overview of recent notable open-weight AI models including Nanbeige, Laguna, Motif, Solar, and Antares with architectural details.
Thanks everyone for all the kind words and feedback. Super happy that you are enjoying Build a Reasoning Model (From Scratch)! Unfortunately, there’s
Explains how LLMs learn low, medium, and high-effort reasoning modes, covering training and inference techniques for controlling reasoning effort.
Analysis of Thinking Machines Lab's new open-weight 975B MoE LLM, Inkling, with benchmark comparisons and architectural surprises.
Analysis of GPT 5.6's 72 configurations, mapping model choices and reasoning effort to performance and cost.
Announcement of the release of 'Build a Reasoning Model (From Scratch)', a book on implementing modern reasoning techniques for AI.
Tutorial on setting up a local coding agent using open-weight LLMs and open-source tools as an alternative to Claude Code and Codex.
Testing local open-weight LLMs like Qwen-Code and Codex in coding harnesses, comparing token use and task success.
Analysis of GLM-5.2, a new open-weight model with IndexShare for efficient long-context sparse attention.
Analysis of VibeThinker-3B, a small AI model achieving strong coding and reasoning performance through advanced post-training techniques.
Analysis of Cohere's new North Mini Code model for agentic coding tasks, including architecture and benchmark performance.
A curated list of LLM research papers from January to May 2026, covering architecture, reasoning, RL, agents, and more.
Analysis of Nemotron 3 Ultra, a 550B parameter LLM using Latent MoE scaling and hybrid Mamba-Transformer architecture.
Analysis of MiniMax M2 LLM technical report: sparse MoE, full attention, agent training, and production-oriented model design.
DeepSeek Sparse Attention implementation from scratch, featuring a learned sparse pattern replacing fixed attention with a lightweight indexer and selector.
Analysis of recent LLM architecture innovations including KV sharing, compressed attention, and mHC for reducing long-context costs.
A talk on implementing LLM architectures from scratch in Python and PyTorch, covering workflow for analyzing new models.
A workflow for understanding open-weight LLM architectures using config files and code from Hugging Face.
An overview of coding agent components, including tools, memory, and repo context, and how they enhance LLM performance in practice.
