Infrastructure

In April 2025, Meta’s Llama 4 Scout achieved something previously thought impossible: processing 10 million tokens in a single context window. To put this in perspective, that’s roughly 20 novels, 40 hours of video, or an entire mid-sized codebase—all in one prompt. The secret behind this breakthrough isn’t a revolutionary new model architecture or exotic hardware. It’s a clever distributed computing technique called Ring Attention that fundamentally rethinks how we compute attention across multiple GPUs. ...

On June 30, 2012, at 23:59:60 UTC, something unusual happened. A single extra second was added to the world’s clocks to account for the Earth’s gradually slowing rotation. Within minutes, Reddit went offline. LinkedIn stopped responding. Mozilla’s servers ground to a halt. Qantas Airways reported that their check-in systems had failed, stranding passengers across Australia. The culprit wasn’t a cyberattack or a hardware failure. It was a bug in how Linux handled leap seconds—a feature that had been tested only a handful of times in the previous decade. The Network Time Protocol (NTP) had warned servers about the incoming leap second, but the kernel’s high-resolution timer subsystem got confused. Applications that were “sleeping” suddenly woke up all at once, overwhelming CPUs. ...

In 2006, Amazon discovered something that would reshape how the industry thinks about performance: every 100 milliseconds of latency cost them 1% in sales. That same year, Google found that adding just 500 milliseconds of delay to search results caused a 20% drop in traffic. These weren’t hypothetical concerns—they were measured impacts on real revenue. The physics of the internet imposes hard constraints. Light travels through fiber optic cable at roughly two-thirds its speed in vacuum—approximately 200,000 kilometers per second. A round trip from New York to Singapore covers about 30,000 kilometers of fiber, which means a theoretical minimum latency of 150 milliseconds just for light to make the journey. Add network equipment, routing hops, and protocol overhead, and real-world latency easily exceeds 200 milliseconds. ...

When you type a URL into your browser, something invisible happens before a single byte of webpage content loads. Your computer must translate that human-readable name into a machine-readable IP address—a process that typically completes in under 100 milliseconds but involves traversing a global hierarchy of servers spanning multiple continents. The Domain Name System (DNS) is often called the “phonebook of the Internet,” but that analogy undersells its complexity. A phonebook is a static directory. DNS is a distributed database with over 1,500 root server instances worldwide, millions of domain records, and caching layers at every level—all designed to resolve billions of queries per day while maintaining consistency across a system that was architected in the 1980s. ...