Start with one click. A customer opens the bank’s website and presses Log in. Before any business logic runs, that request crosses four or five pieces of middleware, each with a different job, and each able to fail in a way that looks, from the outside, exactly like every other failure: the page does not load.

• Name lookup. The browser asks the Domain Name System (DNS) which address serves the bank’s hostname. Covered in DNS.
• Load balancer. The address usually belongs to a load balancer, not a server. It picks a healthy web server and forwards the connection. Covered in load balancing.
• Web server. Terminates the encrypted connection, serves anything static, and passes everything else inward.
• Application server. Runs the bank’s code: checks the password, calls the authentication service, builds the page.
• Back-end systems. A database, a queue manager in front of the core banking system, or another service’s API.

The name undersells it. A modern web server does six jobs, and in a bank the first three matter most:

A reverse proxy is so called because it is the mirror of the proxy most people know: a forward proxy acts on behalf of clients going out to the internet, a reverse proxy acts on behalf of servers receiving traffic from it.

An application server is a runtime for business code. In banks that code is overwhelmingly Java, and the application server is a long-running Java process — a Java Virtual Machine (JVM) — that hosts many applications and gives them a set of services so each does not have to build its own:

• A request pipeline — the part that turns an HTTP request into a call to your code. In Java this is the servlet container; Apache Tomcat is only this part.
• Connection pools to databases and message brokers, so an application borrows an open connection rather than opening a new one per request.
• Transactions, including ones that span a database and a message queue and must succeed or fail together.
• Messaging through a standard Java interface, so code can put a message on an IBM MQ queue without being written for IBM MQ specifically.
• Security — logins, roles and single sign-on handled by the server, not re-implemented in every application.
• Clustering and administration — many copies of the server managed as one, with a console, deployment tooling and monitoring.

The full set of these services is a published specification. It was called Java EE (Java Enterprise Edition) under Sun and then Oracle. Oracle handed it to the Eclipse Foundation in 2017, and in 2018 it was renamed Jakarta EE. A server that implements all of it — IBM WebSphere, Oracle WebLogic, Red Hat JBoss — is a full application server. Tomcat implements only the web part. That distinction drives both architecture and licence cost, and later chapters return to it.

Every application server has a thread pool — the maximum number of requests it will work on at once — and a connection pool for each database or broker it talks to. Most web-tier incidents that are not certificate expiries come down to one of these filling up, and the arithmetic that predicts it fits on one line.

Little’s law says that the number of requests in flight equals the rate at which they arrive multiplied by the time each one spends in the system. It holds for any stable queue — a bank branch, a call centre, a thread pool — and needs no assumptions about the pattern of arrivals.

The defences follow directly from the arithmetic:

• Timeouts on every outbound call, set from the downstream system’s real latency, not a default of sixty seconds. If the database hangs rather than slows, each request's time in the system becomes the timeout: 200 a second × 60 seconds is 12,000 requests in flight; with a two-second timeout it is 400.
• Timeouts that compose. If the load balancer gives up after 30 seconds but the application waits 60 for the database, the customer has gone and the thread is still working. Each outer timeout must be longer than the inner one it wraps.
• Separate pools for separate dependencies (called bulkheads, after the watertight compartments in a ship), so one slow system cannot consume every thread.
• Fail fast when a dependency is sick — a circuit breaker stops calling a system that is timing out, and returns an immediate error or a cached answer instead.

HTTP is stateless: each request arrives with no memory of the last. A logged-in session is therefore an illusion maintained by the server — a cookie in the browser carries an identifier, and the server looks up what that identifier means. Where that lookup lives is an architectural decision with a visible consequence:

A bank that sees customers logged out during every monthly patch has chosen the first row, whether or not anyone decided to.

A load balancer needs to know which servers to use. Two kinds of check matter. A liveness check asks whether the process is running at all. A readiness check asks whether it should be sent traffic now — it may be alive but still starting, or alive but unable to reach its database. Conflating the two sends customers to a server that is up but useless.

Draining is the other half: before a server is patched, the load balancer stops sending it new requests and lets the in-flight ones finish. A cluster patched without draining drops every request that happened to be on the node at the moment it stopped.