Alex Gaynor • 6/2/2025

What is a Benchmark?

This article argues that software benchmarks should be viewed as scientific hypotheses—falsifiable statements predicting how performance changes in a test correlate to changes in a production system. It discusses the need for benchmarks to accurately indicate the direction, magnitude, and absence of performance changes, and explains why complex systems require suites of benchmarks rather than a single test.

0 comments

#software development #performance #benchmarking