Lakshya

Infrastructure & Reliability · Middleware — Start here · Chapter 19 of 33

Middleware: the layer between the application and everything else

What the word actually covers, the three families a bank runs, who owns them — and why a job corpus built from technology companies cannot see any of it.

10 min read1 diagramAll 33 chapters

Most large banks have a team, often literally called Middleware, that almost no interview-preparation material describes. It owns the software that sits between applications — the layer that lets one program hand a request, a message or a file to another without either of them knowing how the other works, where it runs, or whether it is awake.

The easiest way to picture it is a building. The applications are the offices. Middleware is the reception desk, the internal post room and the loading bay: nobody visits the building to admire them, the building does not function without them, and when one of them stops, every office discovers at once how much it depended on something it never thought about.

What our job corpus says, and why it is the wrong instrument here. Across the 7,086 unique job descriptions behind this site, Kafka appears in 317 (4.5%) and “event-driven” in 204 (2.9%). IBM MQ, TIBCO, Solace, WebSphere, WebLogic, IBM Sterling, MOVEit, GoAnywhere and the phrase “managed file transfer” appear in none of them.

That zero is a fact about the sample, not the market. The corpus is harvested from the public hiring boards of technology companies and a set of capability centres — Kyndryl, Salesforce, NVIDIA, Adobe, Caterpillar, Mastercard, eBay, Autodesk, Workday and PayPal. It contains no banks and no insurers, which is exactly where this stack lives. IBM’s own product blog says 98 of the top 100 global banks use IBM MQ; that is a vendor’s figure, undated and with no published method, but no one who has worked in a bank would dispute the direction of it.

So this section is written from vendor documentation, standards bodies, regulators’ own reports and public case studies, and it is not wired to any role’s reading list — the evidence standard the rest of the site uses cannot support that. The same disclosure appears on the India regulation chapter, for the same reason.

The three families

Everything a middleware team runs falls into one of three families, and the difference between them is not the technology — it is what moves, and whether anyone is waiting for it.

FamilyWhat movesTimingWho is waitingTypical products in a bank
Web and application servingA request, then a response (the web’s HTTP protocol)MillisecondsA person, or a partner’s system, holding the lineApache HTTP Server, IBM HTTP Server, Nginx, Microsoft IIS; IBM WebSphere, Oracle WebLogic, Red Hat JBoss, Apache Tomcat
MessagingA message or an event — one instruction or one factMilliseconds to seconds, asynchronousNobody directly; the sender has moved onIBM MQ, Apache Kafka, RabbitMQ, TIBCO EMS (Enterprise Message Service), Solace PubSub+
File transferA file — often thousands of instructions at onceMinutes to days, on a scheduleA cut-off timeIBM Sterling (File Gateway, Connect:Direct), Axway, Progress MOVEit, Fortra GoAnywhere

Asynchronous is the word that separates the second family from the first. In a synchronous call the caller stops and waits for an answer, like a phone call. In an asynchronous exchange the sender drops the message off and carries on, like posting a letter — which is only safe if something guarantees the letter arrives. Providing that guarantee is most of what messaging software is for.

ONE SALARY RUN TOUCHES ALL THREE FAMILIES — IN THIS ORDER FILE TRANSFERminutes to hoursscheduled batchesMESSAGINGmilliseconds to secondsasynchronousWEB & APPmilliseconds, synchronousa person is waiting STEP 1Employer uploads one payroll file4,000 payments, encrypted, over SFTPSTEP 2MFT server checks it, hands it ondecrypt · validate · log · routeSTEP 3Payments engine splits the fileone message per payment, 4,000 of themSTEP 4Queue manager delivers each onceto core banking and the payment schemeSTEP 5Later, the employee opens the appa login, then a balance querySTEP 6Web server, then application serveranswers in well under a second hours later THREE FAMILIES, THREE FAILURE CLOCKS. A web outage is noticed in seconds, by customers. A stuck queue in minutes, by operations. A missed file is noticed at the next cut-off — by the payment scheme, the regulator, or the press.
A single salary run passes through all three families, in order. The employer’s file arrives by managed file transfer (MFT); the payments engine turns it into thousands of individual messages; hours later an employee checks the balance through the web tier. Each family fails on a different clock, and is noticed by different people.

Two terms from the diagram, since they recur throughout the section. SFTP is the SSH File Transfer Protocol — file transfer carried inside an encrypted Secure Shell (SSH) connection. MFT, managed file transfer, is the software layer on top of protocols like SFTP that adds what a bank actually needs: guaranteed delivery, restart after failure, encryption at rest, an audit trail and scheduling.

The neighbours that are not in this section

Two things get called middleware and are covered elsewhere on this site. API gateways — the front door for application programming interfaces (APIs), which handle authentication and rate limits for other programs — are in APIs, webhooks and MCP (the Model Context Protocol, which lets AI agents call tools). Integration platforms, historically called an enterprise service bus (ESB), such as IBM App Connect or MuleSoft, transform and route data between systems; they usually sit on top of the messaging layer described here rather than replacing it.

Why banks have so much of it

  • Decades of systems that cannot call each other directly. A typical large bank runs a core ledger designed in the 1980s or 1990s, channel systems from the 2000s and mobile services from last year. Middleware is the translation and buffering layer that lets them coexist without each knowing the others’ formats, speeds or opening hours.
  • Guarantees the business requires. A payment instruction must arrive exactly once even if a network drops mid-transfer. That is a property of the middleware, not of either application — which is why the middleware is audited.
  • Vendors certify it. Core banking packages are certified against specific application servers and message brokers; Temenos’s published architecture material for its Transact core has long listed IBM WebSphere, Oracle WebLogic and Red Hat JBoss as supported application servers. The bank’s choice is often the vendor’s choice.
  • The networks the bank joins define the interface. SWIFT, the global interbank messaging network, documents an adaptor through which a bank’s back office exchanges messages with its SWIFT gateway over IBM MQ queues. Payment schemes such as the UK’s Bacs and the US ACH network accept bulk payments as files in fixed formats. The bank does not get to choose these.
  • Evidence. When a regulator or an auditor asks whether an instruction was received, when, and by whom, the answer usually comes from middleware logs.

Who runs it, and what the job is

The team’s work splits into five streams, and interviews probe all five:

StreamWhat it involvesWhat goes wrong if it is neglected
BuildInstalling and configuring servers, queue managers and transfer nodes — now increasingly as code, so every environment is built the same wayTwo ‘identical’ production nodes that behave differently under load
RunOn-call, monitoring queue depths and file arrivals, restarting stuck channels, handling messages that cannot be processedA payment queue quietly backing up for an hour before anyone notices
ChangeFix packs, version upgrades, Java updates — inside change windows and freezesRunning a version out of vendor support, which auditors and regulators flag
SecurityEncryption on every connection, certificates, authentication between systems, patching internet-facing file servers within hoursAn expired certificate, or an unpatched file server that becomes a breach
Capacity and licensingKnowing what each server needs, and what each server costs to licenseAn audit bill for software that was licensed on the wrong metric

Common job titles: Middleware Engineer, MQ Administrator, WebSphere or WebLogic Administrator, Integration Engineer, Kafka Platform Engineer, MFT Engineer. A great deal of this work is outsourced — the only middleware titles in our corpus are an “Integration Engineer” and a “MuleSoft Administrator” at Kyndryl, the infrastructure-services company that runs IT estates for large enterprises, banks among them.

The theme that runs through all three families: certificates

Every tier in this section encrypts its connections with Transport Layer Security (TLS), and every TLS connection depends on a certificate that expires on a known date. Certificate expiry is one of the most common self-inflicted middleware outages there is — and entirely preventable. The mechanics are in TLS; the machine-identity side is in machine identity.

How this section is organised

Each family runs from first principles to the level an interview for a senior role will test: what it is and why it exists, the real flows in a bank, the products on the market, the infrastructure under them, how licensing works and what it costs, and finally a comparison of the products. The prices are list prices observed in September 2026, dated as such — a bank pays less after negotiation, but the metric a product is charged on matters more than the number, and the metric is what interviewers ask about.

What interviewers ask about middleware

  • “Walk me through what happens when a customer’s payment leaves the mobile app.”
  • “Why would you put a queue between two systems rather than have one call the other?”
  • “What does ‘exactly once’ mean — and which component actually guarantees it?”
  • “A queue’s depth is climbing. What do you check, in what order?”
  • “How do you patch an application-server cluster without an outage?”
  • “A partner’s file did not arrive by the cut-off. What happens next, and who decides?”
  • “How is IBM MQ licensed, and what is the trap in sub-capacity licensing?”
  • “When would you choose Kafka over IBM MQ — and when would you not?”

Every one of those is answered somewhere in the chapters that follow.

Check yourself

What is middleware, in one sentence?The software between applications that lets one hand a request, a message or a file to another without either knowing how the other works, where it runs, or whether it is available.
What separates the three families?What moves and whether anyone is waiting: web serving moves a request and a person waits milliseconds; messaging moves individual instructions asynchronously; file transfer moves batches against a cut-off.
What does asynchronous mean?The sender hands the message over and carries on without waiting for an answer — like posting a letter. It is only safe if something guarantees delivery, which is most of what messaging software does.
Why does our job corpus show zero IBM MQ postings?It contains no banks or insurers — it is harvested from technology companies and capability centres. The zero describes the sample, not the market.
Name three reasons banks carry so much middleware.Decades of systems that cannot call each other directly; business guarantees such as exactly-once delivery; and interfaces fixed by vendors and networks — SWIFT's adaptor runs over IBM MQ, payment schemes take files.
What are the five streams of a middleware team's work?Build, run, change, security, and capacity and licensing.
What single preventable failure spans all three families?Certificate expiry — every tier encrypts with TLS and every certificate has a known end date.
Where do API gateways and integration platforms fit?They are neighbours: gateways front APIs, integration platforms transform and route data, usually on top of the messaging layer rather than replacing it.
← Managed Kubernetes, and whether you should run it at allWeb servers, reverse proxies and application servers, from zero →