Infrastructure & Reliability · Reliability, operations & platforms · Chapter 17 of 33
The Kubernetes ecosystem, layer by layer
The controller in half of all clusters that is now unmaintained, the licence story everyone reports wrongly, and the mesh adoption number that halved.
The core has become boring in the good way — the last four releases are stability work, not new APIs. The interesting decisions are all in the layers around it, and several of them changed materially in the last twelve months.
Version and support, which is stricter than people assume
| Detail | |
|---|---|
| Cadence | Three minor releases a year, roughly every 3–4 months |
| Supported | The most recent three minors only. Anything older gets no security fixes |
| Patch support | About a year per minor |
| The deprecation rule that bites | A beta API is supported for 9 months OR 3 minor releases, whichever is longer. At three releases a year, the second clause usually wins — so ~12 months, not 9 |
| GA APIs | May be deprecated but not removed within a major version. This is why v1 never disappears |
The pre-upgrade check almost nobody runs: since v1.19 the API server increments a counter for deprecated API requests and stamps a deprecation marker on audit events. Scrape that metric and you get a precise list of what will break, from your own traffic, rather than reading release notes and guessing.
And a subtlety worth knowing: because objects already written to storage must remain decodable, ‘removed’ and ‘gone from etcd’ are different things. The REST endpoint — the ordinary web interface the cluster exposes — can be switched off while the API server still understands persisted data.
Dockershim — the most-mangled fact in this area
“Kubernetes dropped Docker, so Docker images stopped working.” That is wrong, and it is still repeated.
Images are OCI images. Something built with docker build runs identically under containerd or CRI-O. Nothing about image format changed.
What was removed was the shim — the kubelet's built-in translation layer to the Docker daemon. Deprecated in v1.20 (December 2020), removed from the kubelet in v1.24 (May 2022). Dating it to 2020 quotes the deprecation, not the removal.
If you genuinely need Docker Engine as the runtime, cri-dockerd is the adapter — maintained by Mirantis, not a Kubernetes component.
containerd is now the near-universal default across the managed services and most distributions; CRI-O is effectively the OpenShift runtime.
Ingress: the change most clusters have not acted on
The retirement statement is unusually blunt for the project: the component was maintained solely by one or two people working in their free time, with technical debt and design flaws judged unfixable even with new resources. Existing deployments keep running and artefacts stay available — but remaining on it, in the committees' own words, leaves you vulnerable.
Where people are going: Envoy Gateway (the Cloud Native Computing Foundation migrated its own internal cluster to it), kgateway, NGINX Gateway Fabric, or the Gateway API implementations built into Istio and Cilium.
The layers you must choose in
| Layer | The real choice | The honest read |
|---|---|---|
| Packaging | Helm, Kustomize, Timoni, cdk8s | Helm 4 (Nov 2025) replaced the three-way merge with Server-Side Apply, which fixes a large class of 'it didn't update the field' bugs. It still templates YAML as text, so indentation is semantic. Kustomize is right for your own manifests, wrong for redistributing to strangers — no parameter contract, no versioning |
| GitOps | Argo CD, Flux | Both are CNCF Graduated — the claim that one is incubating is wrong. Argo brings its own RBAC layer mapped to SSO; Flux impersonates Kubernetes ServiceAccounts, so there is only one authorization system. Argo if humans click sync; Flux if the platform is machine-driven |
| Service mesh | Istio, Linkerd, Cilium | Istio's ambient mode (GA Nov 2024) removes the per-Pod sidecar — a node-level L4 proxy, with a separate L7 proxy only where you need L7. That is the mesh's answer to its own biggest complaint |
| CNI | Cilium, Calico | Cilium graduated in Oct 2023 and is the second most active CNCF project after Kubernetes itself |
| Autoscaling | Cluster Autoscaler, Karpenter, KEDA | Cluster Autoscaler scales pre-defined node groups; Karpenter is groupless — it reads pending Pod constraints and provisions the cheapest instance that fits. KEDA is not an alternative to either: it scales Pods off external events and composes with them |
Two facts almost everyone reports wrongly
1 — Linkerd did not go closed-source and did not change its licence. It is Apache 2.0 and CNCF Graduated, with copyright held by the project's authors. What changed in February 2024 is the release model: the open-source project stopped publishing stable release artefacts. Weekly edge releases continue from the open repository, and the vendor's enterprise distribution is free including production for organisations under fifty people. “You can't run Linkerd for free in production” is false.
2 — Karpenter is not a standalone CNCF project. Its vendor-neutral core was contributed to Kubernetes SIG Autoscaling, where it sits alongside Cluster Autoscaler. Saying “Karpenter is a CNCF project like Argo” is wrong, and it matters because governance is what people are actually asking about.
And what eBPF actually replaced, since this is the third one people fumble: kube-proxy and the iptables datapath, not the service mesh. Service load balancing becomes a kernel hash-map lookup — constant time regardless of cluster size — instead of sequential iptables rule evaluation that degrades as services multiply.
Service mesh adoption halved, and that is a correction rather than a failure
Survey data shows service mesh adoption falling from roughly 18% of developers to 8% over two years, while a large share of organisations report consolidating microservices back into bigger deployable units — which removes the premise for a mesh entirely.
The honest reading: the category over-sold itself to teams whose real problem was ‘we need mutual TLS and retries’, which a modern CNI and a decent HTTP client solve. You need a mesh if you need per-service layer-7 policy, multi-cluster identity, or regulated mTLS everywhere. Otherwise you need a CNI and a gateway. The drop is the market correcting, and sidecar-free designs may well reverse it.
Check yourself