Numerical Stability Articles

Page 1 of 1 (3 articles)

5/29/2026 • EN

Online (one-pass) algorithms

Explains online (one-pass) algorithms using Welford's algorithm for computing mean and variance, plus higher moments and regression.

Numerical Stability Online Algorithms Streaming Algorithms Variance Welfords Algorithm

John D. Cook

12/15/2022 • EN

Numerically Stable Softmax and Cross Entropy

Explains numerical instability in naive softmax and cross-entropy implementations and provides stable alternatives for deep learning.

Cross Entropy Deep Learning Machine Learning Numerical Stability Softmax

Jay Mody

4/4/2022 • EN

Losses Learned

A guide to correctly implementing cross-entropy loss in PyTorch for binary and multiclass classification, explaining common pitfalls and best practices.

Bceloss Binary Classification Cross Entropy Loss Numerical Stability Pytorch

Sebastian Raschka

Numerical Stability Articles

Online (one-pass) algorithms

Numerically Stable Softmax and Cross Entropy

Losses Learned

Select Language

We use cookies