Some latency measurement pitfalls
A discussion of common pitfalls in measuring tail latency metrics in distributed systems, using examples from Twitter's infrastructure.
DanLuu.com is the personal blog of Dan Luu, known for long-form essays that mix systems thinking with careful measurement and clear writing. The topics range from computer latency and input lag, testing versus informal reasoning, and concurrency bugs, to industry pieces on developer compensation and curated lists of programming blogs worth reading. Many posts include data, historical context, and reproducible reasoning, which is why the site is often cited in courses and shared across the developer community. The design is intentionally minimal, which puts all attention on the ideas.
128 articles from this blog
A discussion of common pitfalls in measuring tail latency metrics in distributed systems, using examples from Twitter's infrastructure.
A blog author lists and corrects serious past errors in their articles, reflecting on their blogging journey and lessons learned.
The article argues against treating employees as interchangeable units in project planning and budgeting, emphasizing that individual skills matter.
The article argues that company culture, alongside incentives and processes, is a crucial tool for influencing employee behavior and performance.
The author argues that being willing to appear stupid by asking naive questions leads to a deeper understanding of complex topics over time.
Argues against learning everything, advocating for mastering a few core 'tricks' to achieve expertise, using examples from math and judo.
The author argues for the importance of increasing productivity and velocity, countering common objections and using a personal project as an example.
Explains why large tech companies like Twitter benefit from having in-house kernel teams to handle critical system issues and reduce costs.
The author argues that measurement and data analysis are undervalued but high-ROI activities, using the impact of Jepsen tests as a key example.
A critique of Fred Brooks' 'No Silver Bullet' essay, arguing against the idea that programmer productivity has a fixed upper limit.
Analyzes if car manufacturers optimize vehicles for specific crash tests by examining results from new, unpracticed IIHS safety tests.
A critical analysis of Star Trek: Voyager's premise and its failure to deliver compelling, character-driven stories due to the constraints of series television.
Explains how to get high value from distributed tracing with less effort, using a real-world implementation from Twitter as a case study.
How Twitter built LongTermMetrics (LTM), a system that identified and shipped millions in cost savings by analyzing JVM metrics.
An analysis of why many corporate engineering blogs fail to attract readers and the key processes used by successful ones like Cloudflare and Segment.
Analyzes the growth of command line options in Unix/Linux tools from 1979 to 2017, contrasting it with the original 'do one thing well' philosophy.
Explores how sharp income cutoffs in US tax and subsidy policies can create perverse incentives for people to intentionally lose money.
Argues that achieving the 95th percentile in many activities is not impressive, using examples from video games and real-world skills.
A software engineer critiques algorithms interviews, arguing they don't reflect real-world problem-solving, despite providing value by fixing such issues on the job.
A talk transcript explaining why files and filesystems are complex, error-prone abstractions for developers, using Dropbox as a case study.