Mixture Of Experts Articles

Page 1 of 1 (4 articles)

9/6/2025 • EN

Understanding and Implementing Qwen3 From Scratch

A hands-on guide to understanding and implementing the Qwen3 large language model architecture from scratch using pure PyTorch.

llm Mixture Of Experts Pytorch Qwen3 Transformer

Sebastian Raschka

8/9/2025 • EN

From GPT-2 to gpt-oss: Analyzing the Architectural Advances

Analysis of OpenAI's new gpt-oss models, comparing architectural improvements from GPT-2 and examining optimizations like MXFP4 and Mixture-of-Experts.

Grouped Query Attention LLM Optimization Mixture Of Experts Rope Embeddings Transformer Architecture

Sebastian Raschka

7/19/2025 • EN

The Big LLM Architecture Comparison

A detailed comparison of architectural developments in major large language models (LLMs) released in 2024-2025, focusing on structural changes beyond benchmarks.

Attention Mechanisms LLM Architecture Mixture Of Experts Normalization Layers Transformer Models

Sebastian Raschka

1/23/2025 • EN

Noteworthy LLM Research Papers of 2024

A curated list of 12 influential LLM research papers from 2024, highlighting key advancements in AI and machine learning.

Fine Tuning llm Lora Mixture Of Experts Research Papers

Sebastian Raschka

Mixture Of Experts Articles

Understanding and Implementing Qwen3 From Scratch

From GPT-2 to gpt-oss: Analyzing the Architectural Advances

The Big LLM Architecture Comparison

Noteworthy LLM Research Papers of 2024

Select Language