Cloud Platform Engineering · Chapter 1 of 9
The compute spectrum, and the ceilings that decide it
Virtual machines to functions, with the limits that actually make the choice — and the one architectural difference that matters more than cost.
Cloud compute is usually taught as a ladder of abstraction, which is true and not useful. The decision is made by a small number of hard ceilings and one structural difference, and if you know those you can choose in a meeting rather than in a proof of concept.
The ceilings, and they are not negotiable
| Constraint | Functions | Serverless containers | Kubernetes / VMs |
|---|---|---|---|
| Maximum runtime | 15 minutes on the classic model. A newer microVM variant reaches 8 hours | Around 60 minutes | Unbounded |
| Memory | Up to ~10 GB. Note that one provider allocates CPU proportionally — 1,769 MB is exactly one vCPU | Up to ~32 GiB | Instance-sized |
| Request / response size | 6 MB synchronous on the largest provider — this alone rules functions out of a great deal of media work | 32 MiB on HTTP/1, unlimited on HTTP/2 | Yours |
| Concurrency per instance | One | Up to 1,000 configurable | Yours |
| HTTP response ceiling | 230 seconds on one provider, on every plan — a load-balancer idle timeout the function timeout cannot override | — | — |
That last row deserves attention because it catches people repeatedly. On one major cloud, an HTTP-triggered function cannot hold a response open beyond about 230 seconds no matter what you set the function timeout to, because the limit lives in the load balancer in front of it. The fix is the asynchronous pattern — accept, return a job identifier, and let the caller poll — and discovering this after building a long-running synchronous endpoint is a genuinely expensive lesson.
The architectural difference that matters more than any price
A function handles one request per execution environment. A container handles many.
Three consequences follow, and they shape the whole design:
Per-request memory is instance memory. You size the environment for one request, so a workload with occasional large requests pays for that size on every request.
A connection pool cannot pool. Each environment holds its own connections, and 200 concurrent requests means 200 database connections rather than a pool of ten. This is why managed connection proxies exist, and it is the single most common way a serverless application takes down its own database.
I/O-bound work is much more expensive. A container waiting on a downstream call can serve hundreds of other requests on the same instance. A function waiting on a downstream call is a whole environment doing nothing, billed for the wait.
Cold starts, and the snapshot trick
Cold start is the time to create an environment, load your code and run your initialisation before the first request is served. Every provider quotes a median and the number that matters is the tail.
The most interesting mitigation is snapshotting: the platform runs your initialisation once when you publish a version, takes a memory-and-disk snapshot of the microVM, and resumes future invocations from it. Initialisation effectively disappears.
And it carries a correctness trap that is a genuinely good interview question.
Anything unique generated during initialisation is baked into the snapshot and reused across every execution environment. A UUID generated at startup is now the same UUID everywhere. A seeded random number generator produces identical sequences. Cached credentials are frozen at their snapshot values and will expire. An open socket is restored into a connection that no longer exists.
The rule: generate anything that must be unique, and open anything that must be live, at invocation time rather than at initialisation time — which is the opposite of the usual advice to do expensive work once during init.
Also worth knowing: this feature supports only some runtimes, it works only on published versions rather than the latest code, and it is mutually exclusive with pre-provisioned concurrency — so you choose one warm-start strategy, not both.
The cost question, answered properly
“Serverless is cheaper” and “serverless is more expensive” are both true, and the variable that resolves them is duty cycle — the fraction of wall-clock time your compute is actually doing work.
- Below roughly 10% busy, serverless wins decisively. You pay for milliseconds of execution instead of hours of idle instance.
- Above roughly 50%, committed instances win decisively, and the gap widens with commitment discounts on top.
- Between the two, cold-start tolerance decides it, and so does operational cost — a small team that would otherwise be patching operating systems has a real reason to pay the premium.
- And the cheap lever people miss: ARM-based compute is materially cheaper on the same platform — on one provider's serverless containers it is exactly 20% less on both the CPU and the memory dimension. For anything that compiles cleanly to ARM, that is a one-line change for a fifth off.
What each is genuinely wrong for
- Functions are wrong for long-lived connections (WebSockets, database pools), large payloads, sustained steady load, and anything with large local state such as model weights — you pay the load cost on every cold start.
- Serverless containers are wrong for anything needing persistent local disk, very large single instances, or the full Kubernetes primitive set.
- Managed Kubernetes is wrong for a team of five with three services. The operational surface is real and it does not shrink because the control plane is managed.
- Virtual machines are wrong for spiky load, because you buy the peak and pay for it continuously — and for any team that would rather not be patching operating systems.
Check yourself