The Interview Spine · Chapter 13 of 13
Does any of this predict performance?
The validity numbers on every hiring slide are from 1998, were corrected downward in 2022, and the correction is still being argued about.
Everyone in hiring quotes the same handful of correlation coefficients, almost always from a 1998 meta-analysis. Those numbers were substantially revised in 2022, and the revisions reordered the methods. Knowing the current figures — and the caveat that comes with them — is one of the cheapest ways to be the best-informed person in a hiring conversation.
What changed, and why
The revision's argument is technical and its consequence is simple. The older corrections for range restriction — the statistical adjustment for the fact that you only observe job performance among people you actually hired — used assumptions that produced substantial over-correction. Applying better ones lowered most estimates by roughly ten to twenty points.
- Work sample tests fell furthest, from about .54 to .33. They had been the top-ranked method for two decades and the basis of a great deal of ‘just have them do the job’ advice.
- Cognitive ability fell from about .51 to .31, and was later updated lower still. The authors state plainly that it is no longer the stand-out predictor it had been treated as.
- Unstructured interviews fell to about .19, with a credibility interval whose lower bound is below zero — which is to say, in some settings they may carry no useful signal at all.
- Years of experience fell to about .07, which is very close to nothing. That is the single most actionable number here, because ‘five to seven years’ is the most common filter in the market.
- And structured interviews became the top-ranked method at about .42, having fallen the least.
The credibility interval, which is the finding
The authors of the correction make a point that essentially nobody repeats, and it changes the practical conclusion completely.
Structured interviews should be read as “.42, plus or minus .24” — an 80% credibility interval from about .18 to .66. Their own words are that this “conveys a different flavor” than stating a single number.
They also suggest comparing methods on the low end of that interval rather than the mean. Do that, and structured interviews — which beat empirically-keyed biodata on the average — lose to it on the floor.
What drives the spread is not the method. It is the execution: whether the rubric is real, whether interviewers were trained, whether scores are recorded before discussion, whether the same questions are actually asked.
So the operational reading is this: a badly-run structured interview and a well-run one differ by more than the gap between structured interviews and most of the rest of the list. Arguing about which method to use is a much smaller lever than running the one you chose properly.
Which is also why Google's published internal findings — that rubrics save around forty minutes per interview, and that rejected candidates who had a structured interview were about 35% happier than those who did not — matter more in practice than the headline coefficient.
And be honest that the correction is itself contested.
A 2023 paper directly challenges the propositions underlying the recommendation against correcting for range restriction in concurrent studies, and cites counter-evidence that restriction in those designs is modest.
Roughly four-fifths of the input studies in the revised meta-analyses are concurrent, so this dispute governs the size of the correction rather than a detail of it.
The defensible position: quote the 2022 figures as the current best estimates, say plainly that they are lower than the 1998 numbers still in wide circulation, give the interval rather than the point, and note that the correction is under active methodological dispute. That is more accurate and more useful than presenting either set as settled.
One further honesty note: none of this is about technical interviews specifically. There is no meta-analysis of whiteboard coding rounds against on-the-job engineering performance. Anyone claiming a validity figure for the coding interview is extrapolating.
What the evidence supports doing
- Structure the interview. It is now the top-ranked method and the effect is robust across every version of the analysis.
- Stop weighting years of experience. At about .07 it is close to noise, and it is the filter most likely to be excluding people who would do the job well.
- Do not abandon work samples — the estimate fell, and there is separate published evidence that asynchronous work samples, where the candidate is not being watched, produce clearer reasoning and lower stress while preserving or improving problem-solving quality and code quality. The format matters, not just the method.
- Invest in execution over selection of method. The credibility interval says that is where the variance is.
- And close the loop. Check hires against the predictions made about them at debrief. Almost no organisation does this, and without it a hiring process has no way to improve — which is, in the end, why so many of them do not.
Check yourself