What the job actually is

Making training and inference fast and affordable at scale. Cluster scheduling, distributed training that survives node failure, serving stacks, batching strategy, quantisation, and the unglamorous arithmetic of how many tokens per second you get per dollar. This is the archetype whose skills age best — the specific models change yearly, the systems constraints do not.

Titles this hides behind

Who is hiring it

Baseten, Modal, Together, Fireworks, Databricks, Snowflake, plus the infrastructure organisation inside every frontier lab. In India, Sarvam is hiring both training infrastructure and inference performance, and Glance is hiring on-device inference — which is a distinctly Indian specialism driven by a mobile-first market.

How to recognise it in a JD

Look for throughput, latency, quantisation, KV cache, vLLM, Triton, CUDA, FSDP, multi-node, cost per token. These terms are rare corpus-wide — CUDA 4%, distributed training 6% — which is precisely why they are a clean signal when present.

The loop

Prepare like this

• Do the memory arithmetic until it is instant — parameters, optimiser state, activations, and KV cache growth as a function of batch and sequence length. This gets asked directly and answered badly.
• Know continuous batching, paged attention, speculative decoding, and tensor versus pipeline parallelism well enough to explain the trade-off, not just the name.
• Bring one measured win. A latency or cost-per-token improvement with a before, an after, and the profiling that found it.
• Serve a real model yourself on rented GPUs and hit an actual throughput target. The gap between reading and doing shows immediately here.

Be ready for

Red flags in the posting

Compensation