Sebastian Raschka, PhD, is an LLM Research Engineer and AI expert bridging academia and industry, specializing in large language models, high-performance AI systems, and practical, code-driven machine learning.
97 articles from this blog
A curated list of 12 influential LLM research papers from each month of 2024, covering topics like Mixture of Experts, LoRA, and scaling laws.
A step-by-step educational guide to building a Byte Pair Encoding (BPE) tokenizer from scratch, as used in models like GPT and Llama.
Explains how multimodal LLMs work, reviews recent models like Llama 3.2, and compares different architectural approaches.
A guide to transforming pretrained LLMs into text classifiers, with insights from the author's new book on building LLMs from scratch.
A 3-hour coding workshop video covering the implementation, training, and use of Large Language Models (LLMs) from scratch.
A technical review of the latest pre-training and post-training methodologies used in state-of-the-art large language models (LLMs) like Qwen 2 and Llama 3.1.
Explores recent research on instruction finetuning for LLMs, including cost-effective data generation methods and an overview of new models like Gemma 2.
Explores new research on instruction masking and LoRA finetuning techniques for improving large language models (LLMs).
A 1-hour video presentation covering the full development cycle of Large Language Models, from architecture and pretraining to finetuning and evaluation.
A review and comparison of the latest open LLMs (Mixtral, Llama 3, Phi-3, OpenELM) and a study on DPO vs. PPO for LLM alignment.
Discusses strategies for continual pretraining of LLMs and evaluating reward models for RLHF, based on recent research papers.
A summary of key AI research papers from February 2024, focusing on new open-source LLMs, small fine-tuned models, and efficient fine-tuning techniques.
A technical guide implementing DoRA, a new low-rank adaptation method for efficient model finetuning, from scratch in PyTorch.
Explores dataset-centric strategies for fine-tuning LLMs, focusing on instruction datasets to improve model performance without altering architecture.
A guide to participating in the NeurIPS 2023 LLM Efficiency Challenge, covering setup, rules, and strategies for efficient LLM fine-tuning on limited hardware.
A guide to 9 PyTorch techniques for drastically reducing memory usage when training vision transformers and LLMs, enabling training on consumer hardware.
A guide to efficiently finetuning Falcon LLMs using parameter-efficient methods like LoRA and adapters to reduce compute costs.
Explores how mixed-precision training techniques can speed up large language model training and inference by up to 3x, reducing memory use.
Explains Low-Rank Adaptation (LoRA), a parameter-efficient technique for fine-tuning large language models to reduce computational costs.
Explains parameter-efficient finetuning methods for large language models, covering techniques like prefix tuning and LLaMA-Adapters.
