Trading & Capital Markets · Chapter 7 of 8
Market data: what it is, what it costs, and who sells it
The largest line item in most trading technology budgets, and the source of the errors that quietly invalidate research.
Market data is deceptively simple — prices, in a file — and it is where an extraordinary share of trading-technology cost, effort and error lives. It is also a subject where being specific instantly marks you out, because most candidates have only ever used a free CSV.
The four levels of data, and what each is for
| Level | What you get | Typical use | Roughly what it costs |
|---|---|---|---|
| End of day | OHLCV per instrument per day | Research, portfolio analytics, most backtests | Free to modest. NSE and BSE publish bhavcopies openly |
| Level 1 / top of book | Best bid, best ask, last trade | Retail platforms, simple execution | Bundled with most broker APIs |
| Level 2 / depth | Several levels of the book aggregated by price | Execution algorithms, liquidity analysis | A real subscription — exchange fees plus vendor |
| Full depth / tick-by-tick | Every order add, modify, cancel and trade | Market making, microstructure research, surveillance | Expensive, and usually only sensible with colocation |
The step from level 2 to full tick-by-tick is not incremental. Aggregated depth tells you how much is resting at each price; tick-by-tick tells you the sequence of individual messages that produced it, which is what you need to reconstruct queue position, detect spoofing patterns, or run any genuine microstructure study. The volume difference is one to two orders of magnitude, and it changes your storage and processing architecture completely.
Where the money goes
- Exchange fees. Exchanges license their data and charge separately for display use (a human looking at it) versus non-display use (a machine trading on it), and non-display is dramatically more expensive. They also charge per user, per application, and audit compliance. This is a large and growing revenue line for every exchange group on earth and it is the subject of continuous industry dispute.
- Vendor fees on top, if you take the data through a Bloomberg, LSEG, FactSet or similar rather than direct.
- Connectivity — cross-connects, ports and lines into the exchange.
- Your own storage and processing, which at tick level is substantial.
- The licensing audit, which is a real operational function at any firm of size, because inadvertently letting non-display data reach a display application is an expensive discovery.
Display versus non-display, the distinction that catches new firms
If a human looks at a price on a screen, that is display use. If a machine consumes it to make a decision, that is non-display, and the licence is far more expensive — frequently by an order of magnitude.
A startup that builds a trading tool on a cheap display feed and then adds automation has changed licence category without noticing. Exchanges audit for this, and the settlements are retrospective.
Mentioning this in an interview for any market-data or trading-platform role is a strong signal, because it is a commercial reality that only people who have actually run a data function know about.
The vendor landscape
| Category | Who | What they are for |
|---|---|---|
| Institutional terminals | Bloomberg, LSEG Workspace, FactSet, S&P Capital IQ | Everything in one place, with the communications network being much of the actual value. Priced per seat per year, in the five figures in USD |
| API-first data vendors | Databento, Polygon, Tiingo, Alpaca, Nasdaq Data Link | Programmatic access to historical and live data without a terminal. Priced per subscription or per data volume, dramatically cheaper |
| Indian broker APIs | Zerodha Kite Connect, Dhan, Upstox, Angel One SmartAPI, Fyers, ICICI Breeze | Live quotes, historical candles and order placement. Monthly subscription in the low thousands of rupees, or free with some brokers |
| Exchange direct | NSE, BSE, MCX data products | The authoritative source, and the licensing conversation that comes with it |
| Free / public | NSE and BSE bhavcopies, Yahoo Finance, exchange websites | Genuinely useful for research. Yahoo in particular is adjusted, unversioned and occasionally wrong — fine for exploration, dangerous as a research foundation |
Prices in this market move and are frequently negotiated rather than published, so verify before quoting. What matters in an interview is knowing the categories and being able to reason about which one a given problem needs — a research backtest on daily bars does not need a colocated tick feed, and saying so is better than knowing a price.
The errors that quietly ruin research
Adjusted prices. A split- or dividend-adjusted series has had the adjustment applied backwards across the entire history. The price you 'see' on a date in 2019 therefore embeds a corporate action from 2023. Using adjusted close as an entry price in a backtest is lookahead, and it is invisible because the series looks perfectly clean. Use unadjusted prices for execution and handle actions explicitly.
Survivorship. A universe built from today's index members excludes everything that was delisted, merged or collapsed. Point-in-time membership is the fix and it is genuinely hard to source, which is why so much published research quietly skips it.
Symbol as identity. Symbols are reused, renamed and rolled. A futures symbol changes every month. Join on a stable instrument identifier with validity dates, never on a ticker string.
Timestamp semantics. Exchange time, vendor receipt time, or your capture time? And is a bar stamped with its open or its close? An off-by-one-bar error is the single most common cause of a spectacular and entirely fake backtest.
Holiday and half-day handling. Indian markets have special sessions, muhurat trading, and occasional segment-specific holidays. A naive calendar produces duplicate or missing bars, and duplicated bars silently double your apparent returns.
Bad prints. Erroneous trades happen and are sometimes cancelled after the fact. A single bad tick can create an apparent 8% intraday move that your strategy trades beautifully.
The systemic answer to all of these is a point-in-time data layer: every query takes an as-of timestamp and returns exactly what was knowable then, including reference data and universe membership, not just prices. Building that is the highest-value thing a market data engineer does, and describing it is the answer that gets the job.
Check yourself