Insightful Practice: a robust measure of medical students' professional response to feedback on their performance

Background: Healthcare professionals need to show accountability, responsibility and appropriate response to audit feedback. Assessment of Insightful Practice (engagement, insight and appropriate action for improvement) has been shown to offer a robust system, in general practice, to identify concerns in doctors' response to independent feedback. This study researched the system's utility in medical undergraduates.Methods: Setting and participants: 28 fourth year medical students reflected on their performance feedback. Reflection was supported by a staff coach. Students' portfolios were divided into two groups (n = 14). Group 1 students were assessed by three staff assessors (calibrated using group training) and Group 2 students' portfolios were assessed by three staff assessors (un-calibrated by one-to-one training). Assessments were by blinded web-based exercise and assessors were senior Medical School staff. Design: Case series with mixed qualitative and quantitative methods. A feedback dataset was specified as (1) student-specific End-of-Block Clinical Feedback, (2) other available Medical School assessment data and, (3) an assessment of students' identification of prescribing errors. Analysis and statistical tests: Generalisability G-theory and associated Decision D-studies were used to assess the reliability of the system and a subsequent recommendation on students' suitability to progress training. One-to-one interviews explored participants' experiences. Main outcome measures: The primary outcome measure was inter-rater reliability of assessment of students' Insightful Practice. Secondary outcome measures were the reaction of participants and their self-reported behavioural change.Results: The method offered a feasible and highly reliable global assessment for calibrated assessors, G (inter-rater reliability) > 0.8 (two assessors), but not un-calibrated assessors G