Employment Interview Reliability: New meta-analytic estimates by structure and format

This study sought to provide an update on evidence regarding the interrater reliability of employment interviews. Using a final dataset of 125 coefficients with a total sample size of 32,428, our results highlight the importance of taking all three sources of measurement error (random response, transient, and conspect) into account. For instance, the mean interrater reliability was considerably higher for panel interviews than for separate interviews conducted by different interviewers (.74 vs. .44). A strong implication of our findings is that interview professionals should not base perceptions of the psychometric properties of their interview process on interrater estimates that do not include all three sources. A number of directions for future research were identified, including the influence of cues in medium structure panel interviews (e.g., changes in tone or pitch) and the lower than expected reliability for highly structured interviews conducted separately by different interviewers.