Caching

In 2006, Amazon discovered something that would reshape how the industry thinks about performance: every 100 milliseconds of latency cost them 1% in sales. That same year, Google found that adding just 500 milliseconds of delay to search results caused a 20% drop in traffic. These weren’t hypothetical concerns—they were measured impacts on real revenue. The physics of the internet imposes hard constraints. Light travels through fiber optic cable at roughly two-thirds its speed in vacuum—approximately 200,000 kilometers per second. A round trip from New York to Singapore covers about 30,000 kilometers of fiber, which means a theoretical minimum latency of 150 milliseconds just for light to make the journey. Add network equipment, routing hops, and protocol overhead, and real-world latency easily exceeds 200 milliseconds. ...

A major e-commerce platform celebrated when their cache hit ratio hit 99.2%. Their dashboard showed beautiful green charts. Three days later, their database collapsed during a flash sale. The cache hit ratio never dropped below 98%. What went wrong? The team optimized for the wrong metric. While their cache served 99% of requests from memory, the 1% that missed were the most expensive queries—complex aggregations, joins across multiple tables, and expensive computations. A cache hit ratio tells you how often you avoid work, not how much work you’re avoiding. ...

When you type a URL into your browser, something invisible happens before a single byte of webpage content loads. Your computer must translate that human-readable name into a machine-readable IP address—a process that typically completes in under 100 milliseconds but involves traversing a global hierarchy of servers spanning multiple continents. The Domain Name System (DNS) is often called the “phonebook of the Internet,” but that analogy undersells its complexity. A phonebook is a static directory. DNS is a distributed database with over 1,500 root server instances worldwide, millions of domain records, and caching layers at every level—all designed to resolve billions of queries per day while maintaining consistency across a system that was architected in the 1980s. ...