Simon Willison • 3/19/2026

Autoresearching Apple's "LLM in a Flash" to run Qwen 397B locally

Article details an experiment by Dan Woods to run the Qwen3.5-397B-A17B MoE model on a 48GB MacBook Pro M3 Max. It leverages Apple's 'LLM in a Flash' paper for efficient inference from flash storage, using AI-assisted coding (Claude) for optimization, quantization, and performance tuning to achieve several tokens per second.

0 comments

#memory optimization #Quantization #Mixture Of Experts