John D. Cook • 5/28/2026

Turning K-L divergence into a metric

This article discusses the Kullback-Leibler divergence and its limitations as a metric, specifically its asymmetry and failure to satisfy the triangle inequality. It introduces Jeffreys divergence as a symmetric variant but shows it still violates the triangle inequality using Bernoulli random variables and Python code. The article then presents the Jensen-Shannon distance, which takes the square root of the Jensen-Shannon divergence, as a true metric that satisfies all distance properties, including the triangle inequality, demonstrated with code examples.

0 comments

#Python #Jensen Shannon Distance #Kullback Leibler Divergence