Software Engineering · Chapter 30 of 31
APIs, webhooks and MCP
Three ways software talks to software, what each is genuinely for, and a worked Confluence chatbot using two of them.
These three come up constantly and get muddled, usually because people compare them as competing technologies. They are not alternatives. They answer different questions, and once you see which, the choice stops being a debate.
API — you ask
An API (Application Programming Interface) is a way for one program to ask another for something. You send a request, you wait, you get an answer. Everything on the internet is built on this.
- REST is the common style: a URL identifies a thing, and the HTTP method says what to do with it.
GET /pages/12345reads it;POSTcreates;PUTreplaces;DELETEremoves. - GraphQL is the main alternative: one endpoint, and the caller specifies exactly which fields it wants. Useful when clients need very different slices of the same data; more work to secure and cache.
- An OpenAPI specification is a machine-readable description of a REST API — every endpoint, parameter and response shape. Remember this one; it is what MCP is doing for models.
What an API cannot do is tell you something happened. To notice a change you must ask again — polling — and polling is a bad trade in both directions: poll often and you waste almost every request discovering nothing changed; poll rarely and you find out late. That gap is exactly what webhooks exist to fill.
Webhook — you are told
A webhook is an API call in reverse. You register a URL with the other system; when the event happens, it calls you. No polling, no delay.
The mental shift is that you are now running a server, with a publicly reachable endpoint, that receives requests from outside on a schedule you do not control. That brings four requirements, and skipping any of them is how webhook integrations fail:
- Verify the signature. Your endpoint is a public URL — anyone can post to it. Real senders sign each payload with a shared secret; recompute the signature and reject on mismatch. An unverified webhook endpoint is an unauthenticated write into your system, and it is startlingly common.
- Respond immediately, then do the work. Acknowledge with a 200 in milliseconds and queue the actual processing. Senders time out, and a slow endpoint gets retried — or disabled.
- Be idempotent. Every serious sender retries on failure, and retries sometimes arrive for messages you already processed. Deduplicate on the event ID. ‘At least once’ delivery is the norm, not an edge case.
- Expect them out of order, and to be dropped. Treat a webhook as a hint that something changed, not as the data itself — then fetch the current state via the API. And run a periodic reconciliation sweep anyway, because some events will be missed and you will not know which.
from fastapi import FastAPI, Request, HTTPException, BackgroundTasks
import hmac, hashlib, os
app = FastAPI()
SECRET = os.environ["WEBHOOK_SECRET"].encode()
seen = set() # in production: Redis with a TTL
@app.post("/webhooks/confluence")
async def receive(request: Request, background: BackgroundTasks):
raw = await request.body()
# 1. Verify -- compare_digest, not ==, to avoid a timing side channel
expected = hmac.new(SECRET, raw, hashlib.sha256).hexdigest()
if not hmac.compare_digest(expected, request.headers.get("x-hub-signature", "")):
raise HTTPException(401, "bad signature")
event = await request.json()
# 2. Deduplicate -- retries WILL arrive for events you already handled
if event["eventId"] in seen:
return {"ok": True, "duplicate": True}
seen.add(event["eventId"])
# 3. Acknowledge NOW, work later. A slow 200 gets you retried or disabled.
background.add_task(reindex_page, event["page"]["id"])
return {"ok": True}
def reindex_page(page_id: str):
# 4. The webhook is a HINT. Fetch current state -- the body may be stale,
# partial, or out of order relative to other events for the same page.
page = confluence_api.get_page(page_id, expand="body.storage,version")
upsert_chunks(page)MCP — the model asks
MCP (Model Context Protocol) is the newest of the three and the most misunderstood. It is not a replacement for APIs. It is a standard way to describe tools to a language model, so the model can decide for itself when to use them.
The problem it solves, stated concretely
Before MCP: to let a model search Confluence, you hand-wrote a tool definition — name, description, parameter schema — plus the code to call the API and format the result. Then you wrote it again for Jira. And again for your database.
And then again in a different shape for every agent framework you used, because each had its own tool format.
MCP standardises that description. Write one MCP server for Confluence and every client that speaks the protocol can use it — Claude Code, a chat assistant, an agent framework, an IDE.
The analogy that fits: OpenAPI describes a REST API to a programmer. MCP describes a capability to a model. Underneath, the MCP server is calling the ordinary REST API — it is a translation layer, not a new transport.
An MCP server exposes three kinds of thing: tools (actions the model may take — search, create, update), resources (content it may read), and prompts (pre-written templates a user can invoke). Tools are what people mean in practice.
from mcp.server.fastmcp import FastMCP
import httpx, os
mcp = FastMCP("confluence")
@mcp.tool()
def search_confluence(query: str, space: str | None = None, limit: int = 5) -> str:
"""Search Confluence pages by text. Returns titles, URLs and excerpts.
Use this when the user asks about internal documentation, policies or
runbooks. Prefer a specific `space` when the user names a team.
"""
# ^ THIS DOCSTRING IS THE PROMPT. It is what the model reads to decide
# whether to call the tool and with what arguments. Vague description,
# vague tool use -- this is the single highest-leverage thing you write.
cql = f'text ~ "{query}"' + (f' AND space = "{space}"' if space else "")
r = httpx.get(
f"{os.environ['CONFLUENCE_URL']}/wiki/rest/api/search",
params={"cql": cql, "limit": limit},
headers={"Authorization": f"Bearer {os.environ['CONFLUENCE_TOKEN']}"},
)
r.raise_for_status()
return "\n\n".join(
f"{h['content']['title']}\n{os.environ['CONFLUENCE_URL']}{h['url']}\n{h.get('excerpt','')}"
for h in r.json()["results"]
)
if __name__ == "__main__":
mcp.run()The tool description is a prompt, and it is the part people underinvest in.
The model has never seen your system. All it has is the tool's name, its description, and its parameter names.
Write when to use it and when not to. ‘Search Confluence’ gets you a tool called at random. ‘Use this when the user asks about internal documentation, policies or runbooks’ gets you one called correctly.
If an agent is using your tools badly, the description is the first place to look and almost the last place people check.
Worked example: a chatbot over Confluence
This is the request that motivated the chapter, and it is a good one because the two architectures pull in opposite directions.
Architecture A — index it. The retrieval pipeline from chapter 12. Pull pages through the REST API on a schedule, chunk them by section, embed, store the space and page permissions alongside, and search with a permission pre-filter.
# Incremental sync -- pull only what changed. The full crawl is for day one.
def sync_since(last_run_iso: str):
start = 0
while True:
r = httpx.get(
f"{BASE}/wiki/rest/api/content/search",
params={
"cql": f'type=page AND lastmodified > "{last_run_iso}"',
"expand": "body.storage,version,space,restrictions.read.restrictions.group",
"limit": 50, "start": start,
},
headers=AUTH,
).json()
for page in r["results"]:
upsert_chunks(
page_id = page["id"],
title = page["title"],
url = f"{BASE}/wiki{page['_links']['webui']}",
text = html_to_text(page["body"]["storage"]["value"]),
updated = page["version"]["when"], # for the staleness filter
acl = read_groups(page), # for the permission pre-filter
)
if len(r["results"]) < 50:
break
start += 50
# Add the webhook so edits land in seconds instead of waiting for the nightly run.
# Keep the scheduled sweep anyway -- webhooks get missed, and you will not know which.Architecture B — give the model a tool. The MCP server above. No index, always current, and the model can search again when its first attempt misses — which is a genuine advantage a fixed retrieval step does not have.
The permission question, which decides whether B can go live at all
Whose credentials does the MCP server use?
A shared service account is what almost every tutorial does, and it means every user of the chatbot effectively has the permissions of that account. If it can read the HR space, so can everyone. Acceptable only if the corpus is genuinely open to all staff, and you should say that out loud rather than assume it.
Per-user credentials — the server calls Confluence with the asking user's own token — is correct, and it is the harder build: you need an OAuth flow, token storage and refresh per user.
This is the same problem as the pre-filter versus post-filter decision in chapter 13, wearing different clothes. It is also the single most common reason an internal chatbot pilot cannot be promoted to production.
Most real systems end up with both, and the split is sensible: index for search, because it is fast and permission-filterable across thousands of pages; a live tool for freshness, so the model can fetch the current version of a specific page it has already found. And a webhook to keep the index honest between syncs. All three, each doing what it is good at.
Choosing, in one table
| Need | Use |
|---|---|
| Get something right now | API |
| React the moment something changes | Webhook — plus a reconciliation sweep, because some will be missed |
| Let a model decide when to fetch something | MCP |
| Search across thousands of documents fast, with permissions | Index it — retrieval, not a live tool call |
| Guarantee the answer reflects the current version | A live tool call — an index is always as stale as its last sync |
The sentence worth remembering: an API is you asking, a webhook is you being told, and MCP is the model asking. MCP does not replace the API — it wraps one so a model can use it without you hand-writing a tool definition for every client.
Check yourself