Lakshya

Product Design & UX · Chapter 3 of 5

Research at the sample sizes you actually get

Expected in 83% of postings, and almost never with the sample size the textbooks assume.

3 min read0 diagramsAll 5 chapters

Research in most product contexts means five to eight conversations, not a statistically powered study. That is not a compromise — qualitative research at that size reliably surfaces usability problems, and treating it as underpowered quantitative research is the mistake.

  • Ask about the last time, not the general case. ‘Walk me through the last time you did this’ produces facts; ‘would you use a feature that…’ produces politeness.
  • Watch rather than ask where you can. What people do and what they report differ, consistently and predictably.
  • Separate usability from desirability. Five users will tell you whether they can operate it. They cannot tell you whether the market wants it.
  • Recruit for the segment that matters, not for who is available. Five of the wrong users is worse than none, because it produces confident wrong conclusions.
  • Know when you need numbers instead. Preference, magnitude and prevalence questions need instrumentation or a survey, not interviews.
← Designing for systems that are sometimes wrongDesign systems and accessibility →