Networks, Automation, Connectivity & Data Centres · Chapter 4 of 18
Routing, and how the internet decides where your packet goes
From a static route to BGP. Including why the internet occasionally routes half of Europe through a small ISP by accident.
Routing is the process of choosing the next hop. Inside your own network you control it; between networks it is a negotiation between around 75,000 independent organisations who mostly trust each other, which is a design decision from the 1980s that we have never managed to replace.
The route table, and the one rule that governs it
A router holds a table of prefixes and next hops. For each packet it finds every matching prefix and picks the longest one — the most specific. A /24 beats a /16 beats a /8 beats the default route 0.0.0.0/0. Nothing overrides this: not administrative distance, not metric, not protocol preference. Those only break ties between routes of the same prefix length.
Once you hold that firmly, several things become obvious. Why a single more-specific static route can black-hole traffic that everything else routes correctly. Why adding a /32 is the standard trick for pinning one host down a different path. And why route hijacking works.
Interior protocols: OSPF, in the depth an interview wants
- Static routes — you type them. Fine for small, stable, or deliberately-pinned paths. They do not react to failure, which is either the point or the problem.
- OSPF — a link-state protocol. Every router floods its local view, everyone builds an identical map of the area, and each runs Dijkstra against it. Converges fast, scales to a few hundred routers per area, cost defaults to a function of bandwidth. Area 0 is the backbone and all other areas must touch it.
- IS-IS — the same class of protocol, less common in enterprises, very common in large service providers because it scales better and is not tied to IP.
- EIGRP — Cisco's, now nominally open. Fast convergence, still widely deployed in Cisco-heavy enterprises.
The interview-useful contrast: link-state protocols share a map and each router computes its own path; distance-vector protocols share their conclusions and trust their neighbours. That difference is why link-state converges faster and why distance-vector protocols need loop prevention mechanisms like split horizon.
BGP, which is a different kind of thing entirely
BGP is not really a routing protocol in the same sense. It is a policy protocol. It does not try to find the fastest path — it has no idea what fast means, no latency measurement, no bandwidth awareness. It finds a path that everyone in the chain has agreed to carry, which is a commercial question as much as a technical one.
The decision process, in order, with the ones that matter in practice marked:
| # | Attribute | Who sets it | Note |
|---|---|---|---|
| 1 | Highest weight | You, locally (Cisco-specific) | Never leaves the router |
| 2 | Highest local preference | You, network-wide | The main tool for choosing your outbound path |
| 3 | Locally originated | — | Prefer your own routes |
| 4 | Shortest AS path | The path itself | The one everybody quotes; only the fourth criterion |
| 5 | Lowest origin type | Originator | IGP < EGP < incomplete |
| 6 | Lowest MED | Your neighbour | A hint about which of their entry points to use — often ignored |
| 7 | eBGP over iBGP | — | External beats internal |
| 8 | Lowest IGP cost to next hop | Your IGP | ‘Hot potato’ — hand it off as soon as possible |
The practical summary: local preference controls what leaves your network; AS path prepending is the crude tool you use to influence what enters it. If you announce the same prefix on two links and want most traffic on one, you prepend your own AS number two or three times on the other, making it look further away. It is inelegant and it is what everyone does.
Peering and transit — the commercial layer
- Transit — you pay someone to carry your traffic to the whole internet. Billed on the 95th percentile of measured throughput, which means you can burst for 36 hours a month for free and the shape of that billing drives real engineering decisions.
- Settlement-free peering — two networks exchange traffic directly, for free, because it saves both of them transit costs. Only covers each other's customers, never the whole internet.
- Paid peering — when the traffic ratio is lopsided enough that one side wants payment. This is what most large content-to-eyeball disputes are actually about.
- An internet exchange (IXP) — a shared switch fabric where hundreds of networks peer with each other over one port. You pay for the port, not per gigabit, which is why exchanges are the cheapest bandwidth on the internet.
The two failure modes worth being able to describe
Route leak. A network that buys transit from two providers accidentally re-announces one provider's routes to the other, advertising itself as a path between two much larger networks. Traffic that should never touch it suddenly does, and it has neither the capacity nor the intent. This has repeatedly taken large portions of the internet offline for tens of minutes. The defence is prefix filtering and maximum-prefix limits on every session — if a neighbour who normally announces 30 prefixes suddenly announces 200,000, drop the session automatically.
Prefix hijack. An AS announces a more specific prefix it does not hold. Because longest prefix match is evaluated before any policy attribute, it wins globally and instantly. This has been used to intercept cryptocurrency traffic and to issue fraudulent TLS certificates by hijacking the validation path. The defence is RPKI: the address holder publishes a signed Route Origin Authorisation, and validating networks drop announcements that contradict it. Adoption passed a majority of routes in the mid-2020s and is one of the genuinely good news stories in internet security.
What this looks like in a cloud interview
You will rarely run BGP yourself in a cloud role, but you will meet it constantly at the edges: Direct Connect and ExpressRoute are BGP sessions, anycast for a global load balancer is BGP, and your CDN's entire routing model is BGP.
The answer that shows understanding: “BGP chooses a policy-compliant path, not a fast one, which is why a packet from Mumbai to Singapore sometimes goes via Europe — and why you cannot fix latency by asking the network team nicely. You fix it by changing where you terminate, not by changing the path.”
Check yourself