Networks, Automation, Connectivity & Data Centres · Chapter 3 of 18
Switching, VLANs and the layer-2 world
The layer people stop learning about, and the layer that produces the worst outages. A broadcast storm can take down a hospital.
Layer 2 is where a lot of engineers' knowledge thins out, because in cloud environments it is abstracted away. But it is still there — under your VPC (Virtual Private Cloud), under your Kubernetes overlay, and very much present in any data centre, factory, campus or trading floor. It is also where the fastest and most total outages come from, because layer 2 has no TTL field and therefore no natural protection against loops.
What a switch actually does
A switch has one job: receive a frame, look at the destination MAC, and send it out of the right port. It learns which MAC lives on which port by watching the source address of every frame it receives, storing it in a CAM table with an ageing timer of typically five minutes.
- Known unicast — destination is in the table, so send it out that one port. This is the normal case and it is why a switch is not a hub.
- Unknown unicast — not in the table, so flood out every port except the ingress. Normal after a table ages out, abnormal if constant.
- Broadcast — destination ff:ff:ff:ff:ff:ff, flood everywhere. ARP is the main producer of these.
- Multicast — flooded unless IGMP snooping is on, which is why a single misconfigured multicast source can saturate a whole VLAN.
Why a loop at layer 2 is different from a loop at layer 3
An IP packet has a TTL that decrements at every hop, so a routing loop wastes bandwidth for a few dozen hops and then the packet dies.
An Ethernet frame has no TTL. A broadcast frame in a physical loop circulates forever, is duplicated at every switch, and doubles on each pass. Within seconds the link is saturated and every CPU on the segment is busy processing broadcasts — including the switches' own management planes, which is why you often cannot log in to fix it.
This is a broadcast storm, and one patch cable plugged into two wall ports by a well-meaning person is enough to cause it. It is the reason Spanning Tree exists and the reason you never disable it to ‘speed up port bring-up’ without also enabling PortFast with BPDU Guard.
Spanning Tree, in the amount of detail you actually need
STP builds a loop-free tree over a physically redundant topology by electing a root bridge and then blocking any port that would create a second path to it. The blocked link is idle — which is the design's main weakness, because you paid for that link.
- Root election is by lowest bridge ID (priority, then MAC). Left at defaults, the oldest switch in the building usually wins, because it has the lowest MAC. This is bad: your traffic now transits a ten-year-old access switch. Always set the root priority explicitly on the switch you want to be root. It is a one-line change and it is the single most common layer-2 design mistake.
- Classic STP convergence is 30–50 seconds. Rapid STP (802.1w) brings this to a few seconds and is the sane default. MSTP maps groups of VLANs to different trees so you can at least use the redundant links for some traffic.
- BPDU Guard on access ports. If a BPDU arrives on a port that should only have a laptop on it, someone has plugged in a switch — shut the port down immediately. This one setting prevents most accidental loops.
In modern data centres STP has largely been replaced. Leaf-spine fabrics with VXLAN and EVPN give you a layer-2 service over a layer-3 underlay, so every link is active and equal-cost multipath spreads traffic across all of them. If you are asked about data centre design, “I would not build a large layer-2 domain with spanning tree in 2026; I would use a routed leaf-spine underlay with VXLAN overlays where layer-2 adjacency is genuinely required” is the answer that lands.
VLANs: one wire, many networks
A VLAN divides a physical switch into multiple logical broadcast domains. Frames tagged with VLAN 10 never reach ports in VLAN 20 without passing through a router, which makes VLANs both a scaling tool (smaller broadcast domains) and a security tool (segmentation).
| Term | What it means | Where it bites |
|---|---|---|
| Access port | Belongs to exactly one VLAN; frames leave untagged | A device on the wrong access VLAN sees nothing and cannot be pinged — looks like a dead port |
| Trunk port | Carries many VLANs, tagged with 802.1Q | Forgetting to allow a VLAN on the trunk is the classic ‘it works on switch A but not switch B’ |
| Native VLAN | The one VLAN sent untagged on a trunk | A native VLAN mismatch between two switches silently merges two networks — a real security incident, not a cosmetic one |
| Voice VLAN | A second tagged VLAN on an access port for an IP phone | Phone works, PC behind it does not, or vice versa |
| SVI | A virtual layer-3 interface on the switch for a VLAN | This is the gateway address for that VLAN; if it is missing, the VLAN is an island |
Link aggregation, and the myth about it
LACP bonds several physical links into one logical one. The important and frequently misunderstood point: a single TCP flow does not get faster. The bond hashes each flow — usually on source/destination MAC, IP or port — to one member link and keeps it there, because reordering packets would be worse than the bandwidth gain. Four bonded 1G links give you 4G of aggregate across many flows and 1G for any one transfer.
This produces a genuinely puzzling symptom: aggregate throughput is fine, one large backup job is capped at exactly a quarter of it, and everything looks healthy. The fix is either a different hashing policy (layer 3+4 rather than layer 2) or more flows, not more links.
A scenario: the Monday morning outage
Symptom. At 08:40 on a Monday, an entire floor loses network. Ping to the gateway times out. The switch management interface is unreachable. Other floors are fine.
What most people do. Start looking at the router, the DHCP server, the firewall rules that changed on Friday.
What the evidence says. One floor, total, sudden, and the switch itself is unreachable. A routing or DHCP problem would not take out the switch's own management plane. Something is saturating the segment.
The diagnosis. Console in over serial — which is why console access exists and why you should know where the cable is. CPU at 100%, interface counters showing millions of broadcasts per second. A broadcast storm.
The cause. Someone tidying their desk over the weekend plugged both ends of a patch cable into two wall ports, creating a loop between two access ports on the same switch.
The fix now. Find the two ports with insane counters, shut them, and the network recovers in seconds.
The fix forever. BPDU Guard and storm control on every access port. This is a ten-minute configuration change that permanently removes an entire class of outage, and describing it is a very good answer to ‘tell me about a problem you fixed properly’.
Check yourself