Lakshya

Product & Delivery · Chapter 4 of 8

Measuring a product

Metrics, OKRs, and the specific ways each is gamed. Named in only 14% of postings, which is itself a finding.

3 min read0 diagramsAll 8 chapters

OKRs appear in 14% of the product postings behind this site and experimentation in 16%. The measurement culture the product literature assumes is a minority practice, which means bringing it credibly is a differentiator — provided you also know how it fails.

Choosing a metric that is hard to game

  • Prefer a metric the user has to do something real to move. Sign-ups are cheap; second-week usage is not.
  • Always pair it with a counter-metric. Speed paired with quality, engagement paired with complaints, conversion paired with refunds. A single metric with no counterweight will be bought at the expense of something you did not measure.
  • Beware ratios. They improve when the denominator falls, so a conversion rate can rise because you lost the hard traffic.
  • Name the lagging and the leading indicator separately. Revenue is lagging and unactionable within a quarter; the leading indicator is what you can actually work on.

Where OKRs go wrong

Two failure modes, and they are opposite. Sandbagging — targets set at what will certainly happen, so the exercise measures nothing. And OKRs as a task list — key results that are deliverables (‘ship the new onboarding’) rather than outcomes (‘raise activation from 34% to 45%’), which reintroduces output as the goal under a new name. The second is far more common and much harder to argue against, because the deliverables are real work.

The comparison question, again

Whatever number you report, the question that should follow is against what? Against last quarter, against the segment that did not get it, against what would have happened anyway. This is the same discipline as the control chapter in the interview spine, and it applies identically here.

A product manager who volunteers the counterfactual unprompted is immediately credible, because almost nobody does.

← Discovery when your users are not consumersDependencies, critical path and honest status →