Artificial Intelligence
Agent / AI Product Engineer
Named in 64% of all job descriptions. The centre of gravity of the whole market.
What the job actually is
You build product features where a model takes actions — calling tools, holding state across turns, recovering from its own mistakes. The hard part is not the prompt. It is everything around it: what happens when the third tool call in a chain returns garbage, how you keep a multi-turn session inside a context budget, how you stop a retry loop from costing $400, and how you prove a change made things better rather than merely different.
Titles this hides behind
Who is hiring it
Effectively everyone — this is the archetype that spread furthest beyond AI-native companies. Sierra, Glean, Writer, Cursor, Notion, Linear, Vanta, Ramp, Postman, Atlan, plus the whole frontier-lab product surface. In India: Postman, Sarvam, Observe.AI, Tekion, Meesho.
How to recognise it in a JD
Look for agentic, tool use, function calling,
orchestration, multi-turn, guardrails, and
increasingly MCP. The tell that separates it from archetype 3 is that
the metrics named are product metrics — task completion, resolution rate, deflection —
rather than model metrics.
The loop
Prepare like this
- Ship one agent with real tool calling and a real eval harness. Not a notebook — something with retries, timeouts, structured logs, and a regression suite you can run before and after a prompt change.
- Break it deliberately and write down what happened. Kill a tool mid-chain. Return malformed JSON. Overflow the context. The interview is largely about failure modes, and you cannot narrate failures you have not seen.
- Learn MCP properly while it is still a differentiator at 8% JD penetration and climbing.
- Get fluent in cost and latency arithmetic — tokens per request, cost per resolved task, p95 versus p50 when a chain has four sequential calls.
- Have a position on prompt injection and what you actually do about it when the agent has write access to something.
Be ready for
- "Your agent calls a tool that fails silently and returns an empty result. How do you detect it?" The best answers reach for evals and tracing, not for a better prompt.
- "How do you evaluate a multi-turn agent, where there is no single correct output?" The archetype-defining question. See archetype 6.
- "The context window fills up mid-task. Now what?" Summarisation, retrieval, handoff, or task decomposition — and the cost of each.
- "You changed a prompt and the demo looks better. Do you ship it?" Correct answer involves a golden set and a regression run, not a vibe.
Red flags in the posting
No mention of evaluation anywhere. A team building agents without evals is shipping on vibes and will blame you when quality regresses invisibly.
A framework named as the job. "Experience with LangChain required" as a headline requirement usually means the team has confused a library with an architecture.
Compensation
A day that is actually representative
| Time | What you are actually doing |
|---|---|
| 09:00 | Look at last night's traces. Not metrics — actual conversations that went wrong, which is the only way to find the failure class you have not thought of. |
| 10:30 | Fix a tool. The retrieval tool returns an empty array when the upstream times out, and the agent has been confidently inventing answers on top of nothing. |
| 13:00 | Eval run. You changed the planner prompt; the golden set says overall flat, and three previously-passing cases now fail. That is the real result. |
| 15:00 | Cost. Someone noticed the bill. You are looking at how many turns a resolved task takes and where the retries come from. |
| 16:30 | A design conversation about whether a step should be an agent decision or plain code — and the answer is usually plain code. |
Real cases
An early conversational deployment produced erratic, occasionally hostile long-conversation behaviour that had not appeared in shorter testing.
Use it to say: “It is why I evaluate multi-turn trajectories rather than single responses. Failure modes that only appear at turn thirty are invisible to single-turn evals.”
An assistant with no constraint on what it could assert agreed to a one-dollar car sale.
Use it to say: “My standard example that the guardrail belongs on the action space, not the prompt.”
Liability for the chatbot's incorrect statement rested with the company.
Use it to say: “It changes the product conversation: model output is a company statement, so what it is allowed to assert is a design decision with legal weight.”
A three-week plan
Week 1 — ship one
Days 1–3: Build an agent with real tool calling, retries, timeouts and structured logs. Not a notebook.
Days 4–5: Break it deliberately: kill a tool mid-chain, return malformed JSON, overflow the context. Write down what each looked like from outside.
Week 2 — make it measurable
Days 1–3: Build the eval harness — golden set, baseline, regression run. Include the failure cases from week one.
Days 4–5: Instrument cost and latency per turn, and compute the gap between the sum of step medians and the turn p99.
Week 3 — rehearse
Days 1–2: The silent-tool-failure and context-overflow answers, out loud, on a timer.
Days 3–4: Learn MCP properly while it is still a differentiator at 8% JD penetration.
Day 5: Five stories: a failure mode you found, an eval that caught a regression, a cost reduction, a decision to use plain code instead of an agent, and something you got wrong.
The book for this field
LLM Systems Engineering
Seven chapters on the engineering underneath applied AI: the roofline and KV-cache arithmetic that explain every serving decision, quantisation with the evidence, serving stacks and the prompt-cache traps that cost real money, retrieval internals and whether long context replaced RAG, agents at the token level, evaluation with actual statistics, and what won in post-training.
Cross-cutting
Skills every archetype tests
These are shared across every field on this site — the same question asked in different vocabulary — so preparation here compounds rather than being spent once. The book above is specific to your field.
Practice the 22 AI questions → · Back to Artificial Intelligence →