Exploration of a possible relationship between examiner stringency and personality factors in clinical assessments: a pilot study. A pilot study

The reliability of clinical examinations is known to vary considerably. Inter-examiner variability is a key source of this variability. Some examiners consistently give lower scores to some candidates compared to other examiners and vice versa - the hawk- dove effect. Stable examiner characteristics, such as personality factors, may influence examiner stringency. We investigated whether examiner stringency is related to personality factors. We recruited 12 examiners to view and score a video-recorded five station OSCE of six Year 1 undergraduate medical students at our institution. In addition examiners completed a validated personality questionnaire. Examiners markings were tested for statistically significant differences using non-parametric one way analysis of variance. The relationship between examiners markings and examiner personality factors was investigated using Spearman correlation coefficient. At each station there was a statistically significant difference between examiners markings, confirming the presence of inter-examiner variability. Correlation analysis showed no association between stringency and any of the five major personality factors. When we omitted an outlier examiner we found a statistically significant negative correlation between examiner stringency and openness to experience with a correlation coefficients (rho) of - 0.66 (p=0.03). Conversely there was a moderate positive correlation between examiner stringency and neuroticism with a correlation coefficient (rho) of 0.73 (p=0.01). In this study we did not find any relationship between examiner stringency and examiner personality factors. However, following the elimination of an outlier examiner from the analysis, we found a significant relationship between examiner stringency and two of the big five personality factors (neuroticism and openness to experience). The significance of this outlier is not known. As this was a small pilot study we recommend further studies in this field to investigate if there is a relationship between examiner stringency in clinical assessments and personality factors..