Networks, Automation, Connectivity & Data Centres · Chapter 1 of 18
What actually happens when you open a web page
Eight distinct mechanisms, each with its own failure mode. Learn this once and every network conversation for the rest of your career becomes easier.
This is the single most useful thing in the book, and it is asked in some form in almost every infrastructure interview — usually as “what happens when you type a URL and press enter?”. Candidates who have not thought about it give a four-sentence answer about DNS and TCP. Candidates who have give a fifteen-minute answer and get asked to stop, which is the outcome you want. The reason it is asked is not trivia: almost every production network problem you will ever debug is one of these eight steps failing, and someone who cannot name the steps cannot bisect the problem.
Step 1 — the name has to become a number
Nothing on a network routes to api.example.com. Routing works on IP addresses, so the first thing that must happen is a translation, and it goes through a surprising number of places before it reaches a DNS server at all.
- The browser's own cache. Chrome keeps its own DNS cache with its own TTL handling, which is why
chrome://net-internals/#dnsexists and why a change can be live everywhere except in one person's browser. - The OS cache.
systemd-resolved,nscd, or the Windows DNS Client service. On macOS,dscacheutil -flushcache. - The hosts file.
/etc/hostsbeats DNS entirely. This is the cause of an enormous number of ‘works on my machine’ problems, because someone added an override during a migration eighteen months ago and forgot. - The stub resolver → recursive resolver. Your machine asks a resolver — your ISP's, or 8.8.8.8, or your corporate one. This is one UDP packet on port 53, and if it is over 512 bytes it may fall back to TCP, which some badly configured firewalls block, which produces the memorable failure where small DNS answers work and large ones do not.
- The recursive walk. Root → TLD (
.com) → authoritative nameserver for the zone.dig +traceshows you every step of this and is the single most under-used diagnostic command in the industry.
The TTL is a promise you cannot take back
When you publish a record with a TTL of 3600, you have told every resolver on earth that it may cache the answer for an hour. If you then need to move that name in an emergency, you cannot. There is no recall mechanism.
This is why every migration runbook starts, at least 48 hours ahead, with lower the TTL to 60 seconds. Do it early enough that the old long TTL has expired everywhere before you need the short one. Raise it back afterwards, because a 60-second TTL on a high-traffic name is a meaningful load on your authoritative servers and a meaningful latency cost on every cold client.
Step 2 — finding the way off your own network
Now you have an IP address. Your machine compares it against its own subnet mask to answer one question: is this address on my local network, or not? If it is local, the packet goes directly. If it is not — and it almost never is — the packet must go to the default gateway.
But Ethernet does not deliver to IP addresses; it delivers to MAC addresses. So your machine broadcasts an ARP request — literally “who has 10.20.30.1, tell me” — to every device in the broadcast domain, and the gateway replies with its MAC. That answer is cached, typically for a few minutes.
Two things follow from this that matter in production. First, ARP is unauthenticated: anything on the local network can answer, which is what ARP spoofing is and why network segmentation matters more than people assume. Second, a stale ARP entry after a failover — where a virtual IP has moved to a new physical machine but the old MAC is still cached — causes a failover that ‘completed successfully’ and yet nothing works for four minutes. The fix is gratuitous ARP, which the new holder broadcasts to force everyone's cache to update, and which not all appliances do reliably.
Step 3 — the packet crosses the local network
A switch receives your frame and looks up the destination MAC in its forwarding table, which it built by watching source addresses on every port. If the destination is unknown it floods the frame out of every port except the one it arrived on. This is normal and it is also why a switch with a full MAC table degrades into a hub — a real attack, called MAC flooding.
If you have VLANs, the switch also checks whether the frame is allowed on this VLAN, and a frame crossing between VLANs must go up to a router or a layer-3 switch. This is worth internalising because “two machines on the same switch cannot reach each other” is almost always a VLAN or a firewall, not a cable.
Step 4 — the router, and the route table
Your gateway does a longest prefix match against its routing table. If it has a route for 203.0.113.0/24 and also a default route 0.0.0.0/0, the more specific one wins — always, regardless of any other consideration. Longest prefix match is the most important single rule in IP routing and it explains a surprising amount of otherwise baffling behaviour, including route hijacks, which we come back to in chapter 4.
On the way out of most networks the packet also gets NATed: the source address is rewritten from a private address to a public one, and the router keeps a table entry so it can reverse the translation on the way back. That table is finite. On a busy NAT gateway it is one of the quietest and most confusing production limits you will meet — connections start failing not because of bandwidth, CPU or the application, but because a translation table filled up. On Linux this is nf_conntrack_max, and dmesg will say nf_conntrack: table full, dropping packet, which is one of the more useful error messages in computing because it tells you exactly what is wrong.
The AWS NAT gateway bill, which surprises everyone once
A managed NAT gateway charges per hour and per gigabyte processed. Traffic from a private subnet to S3 in the same region, routed through a NAT gateway, is billed per GB even though it never leaves AWS.
The fix is a VPC (Virtual Private Cloud) gateway endpoint for S3 and DynamoDB, which is free, and interface endpoints for other services, which are cheaper than NAT for high volume. Teams commonly find five-figure monthly savings here in an afternoon, and it is a very good thing to have done before an infrastructure interview.
Step 5 — twelve to twenty hops of someone else's network
From here the packet is out of your control. Each router along the path repeats the same operation — longest prefix match, decrement TTL, forward — and the path itself is decided by BGP, which is chapter 4. Two things about this segment matter.
MTU. Every link has a maximum frame size, typically 1500 bytes on Ethernet, less if there is a tunnel (VPN, VXLAN, PPPoE) in the path. If a packet is too large and has the ‘don't fragment’ bit set, the router is supposed to send back an ICMP “fragmentation needed” message so the sender reduces its size. If some middlebox blocks ICMP — and a great many corporate firewalls do, on the theory that ICMP is dangerous — that message never arrives, and you get a path MTU black hole: small requests work perfectly, large ones hang forever. The signature is unmistakable once you have seen it: curl of a small endpoint is fine, a large POST hangs, and ping -M do -s 1472 fails while ping succeeds.
Asymmetric routing. The path back is not necessarily the path out. This is normal on the internet and catastrophic through a stateful firewall, which sees a returning packet for a connection it never saw open and drops it. It is the standard explanation for “it works from the office but not from the VPN”.
Steps 6 to 8 — arrival, TCP, TLS, and the load balancer
The packet arrives at an edge, which for any large service is an anycast address — the same IP announced from dozens of locations, with BGP delivering you to the nearest one. Then TCP's three-way handshake (one round trip), then TLS (one more round trip in TLS 1.3, two in 1.2), then finally your HTTP request.
Count the round trips, because this is where perceived slowness usually lives. On a 180ms Mumbai–Virginia path: DNS may be one round trip if not cached, TCP is one, TLS 1.3 is one, and the request itself is one. That is 720ms before your application has done a single thing. This is the entire commercial argument for CDNs and edge termination — not bandwidth, but moving the handshakes closer so only the last, cacheable leg crosses the ocean.
Say it out loud, in order
The interview version of this answer should be structured, not exhaustive. Name the eight steps, then say “and the ones that actually break in production are DNS TTL, MTU, and certificate expiry — shall I go deeper on any of them?” That single sentence moves you from someone who memorised a sequence to someone who has debugged one.
Check yourself