Simon Willison • 1/31/2026

Quoting Andrej Karpathy

The article quotes Andrej Karpathy comparing the 2019 training of GPT-2 (costing ~$43K over 7 days) to a modern, more efficient training run achieving a higher benchmark score in just 3.04 hours for ~$73. This demonstrates a 600x cost reduction over 7 years, equating to training costs falling by approximately 2.5x annually in AI/ML.

0 comments

#Machine Learning #hardware scaling #Gpt 2