Infrastructure & Reliability · Reliability, operations & platforms · Chapter 5 of 33
The physical layer, and why AI broke the datacentre
Power, cooling and space. Unfashionable, non-optional, and currently the binding constraint on the entire industry.
Cloud abstracted the physical layer without removing it, and AI has made it visible again. A quarter of the IT infrastructure postings in this atlas are AI cluster buildout, and the reason is that training workloads break the assumptions datacentres were designed around.
The arithmetic that decides everything
| Conventional hall | AI training | |
|---|---|---|
| Power per rack | 5–15 kW | 40–130 kW |
| Cooling | Air, comfortably | Rear-door heat exchangers or direct-to-chip liquid |
| What binds first | Space, sometimes power | Power distribution, then heat rejection |
| Failure impact | One node degrades a service | One node stops a synchronous job across thousands |
| Lead time | Weeks for servers | Quarters to years for transformers and switchgear |
The practical consequence: a hall with plenty of floor space frequently cannot host a single row of training racks, because the busway to that position cannot deliver the current and the building cannot reject the heat. Space is almost never the constraint, and saying so is the fastest way to demonstrate you have worked at this layer.
Redundancy notation, decoded
- N — exactly what is needed. Any failure is an outage.
- N+1 — one spare of each component. The common enterprise target.
- 2N — two complete independent systems.
- Concurrently maintainable — every component can be serviced without dropping load. This is what Tier III actually means and it is the phrase that matters in practice, because most real outages happen during maintenance rather than during failure.
PUE, and why it is often quoted misleadingly
Power usage effectiveness is total facility power divided by IT power. 1.0 would mean every watt reaches the equipment; 1.5 means half again is spent on cooling and losses.
It is quoted selectively — at design load, in winter, excluding certain draws. Ask what load it was measured at and over what period, because a PUE quoted at full design load says nothing about a hall running at 40%.
What the physical layer forces on your design
Most engineers never see a datacentre, and three physical facts still shape every availability decision they make.
- A rack shares power and a top-of-rack switch. Two instances in the same rack are one failure domain, however separate they look in your console. This is the reason placement and anti-affinity rules exist, and the reason they are worth setting rather than hoping.
- An availability zone is a distinct facility, and zones within a region are close enough for synchronous replication — typically a millisecond or two apart. That closeness is the entire product: it is what makes multi-zone the cheap, obvious, always-correct default.
- Regions are far enough apart that synchronous replication is not viable. Cross-region means asynchronous, which means a recovery point objective greater than zero — you will lose some recent writes in a regional failover. Any design claiming otherwise has not been tested.
- Power and cooling are the real capacity constraint, not floor space. A facility runs out of kilowatts and kilowatts of cooling long before it runs out of room, which is why rack density and efficiency dominate datacentre economics and why an unused reservation still costs someone money.
The correlated-failure question is the one to bring to any high-availability design review.
Three replicas are only three failures if they fail independently. Ask what they share: the same rack, the same zone, the same power feed, the same deployment pipeline, the same configuration, the same certificate, the same expiring credential.
The shared dependencies that actually cause multi-replica outages are almost never hardware. They are a bad configuration pushed everywhere at once, a certificate expiring simultaneously on all nodes, a DNS change, or a deploy that rolled out faster than anyone could stop it.
Which means your deployment pipeline is a failure domain, and usually the largest one you have. Staged rollout with a real bake time is not release ceremony; it is the control that keeps a software fault from becoming a correlated hardware-scale outage.
And the honest test of a failover design is whether it has been run — deliberately, on a normal working day, with the people who would actually be doing it. A failover that has never been exercised is a hypothesis.
Check yourself