A strong answer. “‘Utilisation looks fine’ is usually where the investigation goes wrong, because utilisation is an average and the problems that cause this are not visible in averages.

I would work top-down and cheap-first. Saturation, not utilisation — run queue length and pressure stall information tell you whether processes are waiting, which is what slow feels like. A host at 40% CPU with a run queue of eight is saturated and the graph says idle.

Then the specific asymmetries between otherwise identical machines. Thermal throttling — check current clock against nominal; a partially blocked heatsink or a failing fan produces exactly ‘identical hardware, one is slower’. A degraded memory channel — correctable ECC errors accumulating, or a DIMM that has been mapped out so the host is running with less interleaving than its siblings. NUMA — memory allocated on the wrong node, so every access crosses the interconnect. A degraded disk that has not failed but is retrying, visible in latency percentiles rather than in throughput. Network — a link negotiated at the wrong speed or duplex, or interface errors, which is embarrassingly common and invisible unless you look at the counters.

The single most useful thing is that we have identical siblings, so I would diff everything mechanically — firmware versions, BIOS settings, kernel parameters, NIC negotiated speed, DIMM population. The difference is almost always in that diff, and it is faster than reasoning about it.

My first guess given the symptom would be thermal or a mapped-out DIMM, because both produce exactly this and neither shows up in utilisation.”