Lakshya

Analytics & BI Tools · Chapter 15 of 18

Eight decisions, worked

Real situations, the answer, and — more usefully — why the obvious answer is wrong in each one.

10 min read0 diagramsAll 18 chapters

Tool selection is taught as a feature comparison and decided by something else entirely: the skills you have, the contracts you already hold, the freshness you genuinely need, and who will own the thing in two years. These eight scenarios are the shapes that recur. In each one the obvious answer is defensible and wrong.

1 · A 200-person company already on Microsoft 365

Situation. No BI tool today. Data in a cloud warehouse. Finance wants dashboards. Two analysts, no data engineers.

The obvious answer. Evaluate three vendors on features.

The actual answer. Power BI, and do not run an evaluation. The per-user licence is inexpensive, it may already be inside your existing agreement, your analysts almost certainly know Excel and the transition is short, and identity and sharing work with no additional setup.

Why the obvious answer is wrong. A three-vendor evaluation at this size costs more in people's time than several years of licences, and it will select on features that two analysts serving one finance team will never use. The cost of choosing correctly can exceed the cost of choosing wrong, and recognising when you are in that regime is a genuine skill.

What would change it. A strong existing investment in another tool, or an analyst team whose entire skill set is elsewhere. Skills beat features at this scale.

2 · A company with three BI tools and three different revenue numbers

Situation. Historic sprawl. Finance uses one tool, marketing another, product a third. Every quarterly review starts with reconciliation.

The obvious answer. Consolidate onto one tool.

The actual answer. Fix the definitions first, in the warehouse or a semantic layer, and consolidate tools second if at all.

Why the obvious answer is wrong. The three tools are a symptom. If you migrate everyone onto one and each team rebuilds its own model with its own definition of revenue, you have spent a year and a large amount of money and still have three numbers. This happens often enough to be predictable.

And the sequencing is the whole insight: shared definitions make consolidation optional. Once every tool queries the same certified metrics, having three tools is a licensing inefficiency rather than a correctness problem — which is a much smaller and more patient problem to solve.

The political point. Consolidation is popular with executives because it looks decisive and has a number attached. Definition work is unglamorous and has no launch date. Being the person who argues for the second is harder and correct.

3 · A product team that wants analytics inside their application

Situation. A SaaS product; customers want dashboards on their own data inside the product. Ten thousand end customers.

The obvious answer. Embed the BI tool the company already uses internally.

The actual answer. Treat this as a completely different problem with different economics, and evaluate embedded-specific options — or build it.

Why the obvious answer is wrong. Internal BI is priced per named user or per capacity for an audience of dozens. Ten thousand external customers breaks both models. And the requirements differ in kind: multi-tenant row-level isolation that must be provably correct, your own branding, your own authentication, and a latency budget set by your application rather than by an analyst's patience.

The isolation point deserves emphasis. Internally, a row-level security mistake means someone sees the wrong department's numbers. Externally, it means one customer sees another customer's data — a reportable breach. That difference should change how the thing is tested, and it argues for isolation enforced below the BI layer, in the database or the API, rather than in a dashboard filter.

The build-versus-buy line sits roughly where customisation demands do: a handful of standard charts is worth embedding, and a genuinely differentiated analytics experience that is part of your product's value proposition usually gets built.

4 · Finance wants real-time dashboards

Situation. The request is explicit: live data, refreshed continuously.

The obvious answer. Live query against the warehouse.

The actual answer. Ask one question first — what decision do you make differently if this number is an hour old? — and in a finance context the answer is almost always none.

Why the obvious answer is wrong. Live query makes every interaction a warehouse query. At forty finance users clicking filters, the warehouse bill rises materially and the dashboard becomes slower than an imported model would have been. You have paid more for a worse experience to satisfy a requirement nobody examined.

What ‘real-time’ usually means when you ask: not yesterday's. An hourly refresh satisfies it completely.

Where it is genuinely required: operational monitoring — a fraud queue, a fulfilment exception board, a live incident view. Those are real, and they are usually better served by a purpose-built operational display than by a BI dashboard, because their latency and concurrency profile is different from analytics.

The middle answer that satisfies almost everyone: an imported model refreshing hourly, with a visible ‘data as of’ timestamp. Most of the anxiety behind the request is not knowing how stale the number is.

5 · The analytics team is three data scientists and no analysts

Situation. A technical team. Everyone writes SQL and Python. They need to publish results to the business.

The obvious answer. Buy a traditional BI tool so the business can self-serve.

The actual answer. A notebook-first or SQL-first tool for the team, plus a small number of curated dashboards for the business.

Why the obvious answer is wrong. A drag-and-drop tool is friction for people who write code — it is slower for them than SQL and it hides what the query is doing. They will use it reluctantly, badly, and eventually route around it. Meanwhile the business self-service the tool was bought for does not materialise, because self-service requires a curated model that nobody has built.

The pattern that works: the technical team works in the medium that suits them, and publishes a deliberately small set of well-designed dashboards. Five dashboards everyone trusts beats a self-service platform nobody uses, and it is far less work.

What would change it. The business hiring its own analysts. At that point the self-service argument becomes real, and you should revisit it — but as a response to demand rather than in anticipation of it.

6 · A regulated institution with an on-premises mandate

Situation. Data cannot leave the country, or in some cases the building. Auditors need to trace every number to its source.

The obvious answer. Whichever cloud BI tool offers a regional deployment.

The actual answer. Start from the constraint rather than from the tool. Which options genuinely deploy where you need them, and which can produce the lineage evidence an auditor will accept?

Why the obvious answer is wrong. ‘Available in your region’ and ‘satisfies your data-residency obligation’ are different claims. Metadata, telemetry, query logs and support access frequently cross borders even when the data does not, and that is exactly what a regulator asks about. Get the specific answer in writing before the evaluation, not during the procurement.

The lineage requirement is the one that narrows the field, and it is usually underestimated. ‘Where did this number come from’ must be answerable through every transformation, and automatic lineage capture has real gaps — it typically does not survive object renames and does not follow logic hidden inside a user-defined function. If your evidence requirement is stronger than best-effort, you need the definitions in version control where the history is the audit trail.

And this reframes the semantic layer from a nice-to-have into the compliance artefact, which is frequently how these projects actually get funded.

7 · The warehouse bill tripled and nobody changed anything

Situation. Warehouse spend up sharply. Query volume roughly flat. No obvious culprit.

The obvious answer. Optimise the expensive queries.

The actual answer. Find out what is issuing the queries before optimising any of them. In a BI-heavy estate the answer is usually a dashboard.

The specific causes, in the order they turn out to be true. A popular dashboard moved to live query, so every filter click is now a warehouse query multiplied by the audience. A scheduled refresh interval was shortened from daily to hourly, quietly multiplying refresh cost twenty-four-fold. A dashboard with a default filter of ‘all time’ that scans three years on every open. Or dozens of models refreshing at exactly the same minute, creating a spike the warehouse must be sized for.

Why query optimisation is the wrong first move. The queries may each be fine. The problem is how many of them there are, and that is a design and scheduling question rather than a SQL one.

The fix that usually works: import instead of live query where freshness allows, pre-aggregate to the grain the dashboard shows, default filters to a narrow range, and stagger refresh schedules. None of these is query tuning and together they routinely remove most of the increase.

8 · Two years in, three hundred dashboards, and nobody trusts any of them

Situation. Adoption succeeded and then curdled. Hundreds of dashboards, wide duplication, and business users have gone back to asking analysts for numbers.

The obvious answer. A clean-up project, and stricter controls on who can publish.

The actual answer. Certify a small number, archive the unused, and make the trusted path the fastest one. Do not restrict creation.

Why the obvious answer is wrong. Restricting publication does not remove the demand; it moves the work into spreadsheets, where it is invisible and worse. You have converted a governance problem you can see into one you cannot.

What to do instead, in order. Pull usage telemetry — typically a third of dashboards have not been opened in ninety days, and archiving those with one-click restore is free and uncontroversial. Then take the ten dashboards that carry the most traffic and certify them properly: a named owner, definitions from a shared model, refresh monitoring, and a visible mark that says so. Then leave everything else alone.

The insight underneath. Trust is not restored by having fewer dashboards. It is restored by having a small number that are unambiguously reliable, clearly marked, and fast enough that using them is easier than asking an analyst. Governance that competes on convenience wins; governance that competes on authority does not.

What the eight have in common

Read them together and the same three moves recur, which is the actual lesson of the chapter:

  • Interrogate the requirement before satisfying it. ‘Real-time’, ‘consolidate’, ‘self-service’ and ‘clean up’ are all requests that dissolve into something smaller and cheaper when you ask what decision depends on them.
  • Separate the symptom from the cause. Three tools is a symptom of three definitions. A tripled bill is a symptom of a design change. Distrust is a symptom of ambiguity, not of volume.
  • Ask who owns it in two years. Most of these situations were created by a reasonable decision that nobody revisited when the context changed. The tool choice matters far less than whether there is a named person who will notice when it stops being right.

Check yourself

A company already on Microsoft 365 wants BI. What do you recommend?Power BI, without running an evaluation — at that size the evaluation costs more in time than years of licences, and skills and existing agreements beat features.
Three tools, three revenue numbers. Consolidate?Fix the definitions first. If each team rebuilds its own model on the new tool, you will have spent a year and still have three numbers.
Finance asks for real-time. What do you ask them?What decision changes if the number is an hour old. Live query makes every interaction a warehouse query, so you pay more for a slower dashboard to satisfy an unexamined requirement.
Three hundred dashboards and no trust. What do you do?Certify ten, archive the unused, leave the rest alone. Restricting publication moves the work into spreadsheets where it is invisible and worse.
← The preparation layer: Alteryx and what competes with itThe wider landscape: who sells what, and what changed →