Networks, Automation, Connectivity & Data Centres · Chapter 8 of 18
Cloud networking, and the bill it generates
VPCs, security groups, endpoints and egress. The concepts are the same as the physical world with different names and a meter running.
Cloud networking is not a different discipline — it is the same discipline with the cabling replaced by API calls and a bill attached to data movement. Everything from the earlier chapters still applies: subnets are subnets, route tables are route tables, longest prefix match still governs. What is new is that some choices cost money per gigabyte, and those choices are usually made once, early, by someone who did not know.
The building blocks, and their real-world equivalents
| Cloud construct | Physical equivalent | What people get wrong |
|---|---|---|
| VPC (Virtual Private Cloud) / VNet (Virtual Network) | Your own data centre network | Choosing 10.0.0.0/16 like everyone else, then failing to peer later |
| Subnet | A VLAN with an SVI | In AWS a subnet lives in one AZ — so a subnet is a failure domain, not just an address range |
| Route table | A router's route table | Public vs private subnet is defined by whether the route table has an internet gateway. Nothing else |
| Internet gateway | Your border router | Free. The NAT gateway next to it is not |
| NAT gateway | An outbound-only firewall | Charged per hour and per GB processed — the most common surprise line on a cloud bill |
| Security group | A stateful firewall on the instance | Stateful — allow inbound and the reply is automatically allowed. Allow-only; you cannot write a deny rule |
| Network ACL | A stateless router ACL on the subnet | Stateless — you must allow the return traffic explicitly, including ephemeral ports. This is why NACLs break things mysteriously |
| VPC endpoint | A private on-ramp to a service | Gateway endpoints (S3, DynamoDB) are free. Interface endpoints cost per hour per AZ plus per GB — but still beat NAT at volume |
| Transit gateway | A hub router | Charges per attachment per hour and per GB — it is a router with a meter |
| PrivateLink | A one-way private connection to someone else's service | Great security story, and each endpoint has an hourly cost that adds up across a large estate |
Security groups versus NACLs — the single most common cloud networking interview question
Security groups are stateful and allow-only. They attach to an instance or ENI. If you allow inbound 443, the response leaves automatically. You cannot express ‘deny’ — absence of an allow is the deny.
NACLs are stateless and support both allow and deny. They attach to a subnet. If you allow inbound 443, you must also allow outbound on ephemeral ports (1024–65535) or the response is dropped. This is why a NACL change frequently breaks something that looks unrelated.
Where each belongs. Use security groups for essentially all normal access control — and reference other security groups rather than IP ranges, so the rule reads ‘the web tier may reach the database tier’ and stays correct as instances change. Use NACLs sparingly, for coarse subnet-level blocks such as denying a known-bad range.
The egress bill, which is a design constraint and not an accounting detail
Data movement pricing is where cloud networking becomes an architecture question. The shape is consistent across providers even though the numbers differ:
- Inbound from the internet — free, everywhere. This is deliberate.
- Outbound to the internet — charged per GB, with the first tier being the most expensive and volume discounts arriving only at serious scale.
- Between availability zones — charged in both directions on AWS. A chatty microservice architecture spread randomly across three AZs can generate a very large bill purely from cross-AZ chatter, and the fix — topology-aware routing that prefers a same-zone endpoint — is usually a configuration change rather than a redesign.
- Between regions — charged, and more than cross-AZ.
- Through a NAT gateway — charged per GB on top of whatever the destination charge is. Traffic to S3 through a NAT gateway is paying twice for something a free gateway endpoint would have handled.
- Leaving the provider entirely — since 2024, all three major providers waive egress fees for customers migrating away, driven by the EU Data Act. This applies to a full exit, not to ordinary traffic, and it is worth knowing precisely because it is often quoted imprecisely.
A worked example, because the arithmetic is the point
A service in ap-south-1 serves 40 TB a month of API responses to the internet, and pulls 15 TB a month from S3 in the same region through a NAT gateway.
| Line | Volume | Approximate rate | Monthly |
|---|---|---|---|
| Internet egress | 40 TB | ~$0.109/GB at this tier | ~$4,460 |
| NAT gateway data processing | 15 TB | $0.045/GB | ~$690 |
| NAT gateway hours | 2 AZs × 730 h | $0.045/h | ~$66 |
| S3 → EC2 same region | 15 TB | free via gateway endpoint | $0 |
| Total as built | ~$5,216 | ||
| With an S3 gateway endpoint | ~$4,526 — a $690/month change that takes ten minutes |
Rates move and differ by region, so treat these as the shape rather than a quote. The point is the method: identify each leg of data movement, find its rate, multiply, and then ask which legs could be free. Being able to do this arithmetic out loud is unusual among candidates and extremely well received, because it is exactly what the job involves.
Hybrid connectivity: getting your own network in
- Site-to-site VPN — IPsec over the public internet. Cheap, quick, typically capped around 1.25 Gbps per tunnel, and subject to internet variability. Correct for most companies most of the time.
- Direct Connect / ExpressRoute / Cloud Interconnect — a dedicated circuit into the provider's network via a colocation facility. You pay a port fee plus reduced egress rates, and you get predictable latency. Worth it above roughly 5–10 TB a month of egress, or when jitter matters — and the break-even calculation is a genuine and impressive thing to be able to sketch.
- The gotcha — a single dedicated circuit is less reliable than the internet, because the internet reroutes and a cut fibre does not. Two circuits at two locations, or one circuit with VPN backup, is the design that actually improves availability.
Check yourself