Dr Matt Homer publishes new paper on setting standards for medical trainees
Congratulations to Dr Matt Homer on his new published paper in Medical Teacher - Setting defensible minimum-stations-passed standards in OSCE-type assessments
This paper discusses how we set standards in exams for medical trainees, and it proposes a better method for part of how we do this.
It uses statistical modelling of a large exam data set (many exams, many examiners, many candidates, many activities) and argues that adopting this new approach would improve practice in this high-stakes setting.
Abstract
Introduction
Alongside the usual exam-level cut-score requirement, the use of a conjunctive minimum number of stations passed (MNSP) standard in OSCE-type assessments is common practice across some parts of the world. Typically, the MNSP is fixed in advance with little justification and does not vary from one administration to another in a particular setting—which is not congruent to best assessment practice for high stakes examinations. In this paper, we investigate empirically four methods of setting such a standard in an examinee-centered (i.e. post hoc) and criterion-based way that allows the standard to vary appropriately with station and test difficulty.