Measuring situation awareness in medical education objective structured clinical examination guides

Background: Medical errors are among the most prevalent and serious adverse events in health care. Lack of situation awareness (SA) is an important factor leading to such errors. SA can be understood using Endsley’s three-tier model: level 1 is perception, level 2 is comprehension, and level 3 is projection. While there is extensive literature on the theory of SA, it is difficult to measure and quantify. The purpose of this pilot study was to measure, identify, and characterize SA in some medical objective structured clinical examination (OSCE) guides, including a 1^st year National University of Ireland, Galway (NUIG) OSCE. Methods: Two independent observers analyzed two online OSCE guides and a 1^st year OSCE examination using a self-developed tool. This tool was an inferential measure of SA. The guides were first qualitatively analyzed using NVivo and then quantitatively analyzed using Excel. Results: The results indicated strong internal validity and moderate inter-rater reliability. There was limited statistically significant variance between the observers. The NUIG OSCE had relatively the fewest relative observations of SA and the Geeky Medics OSCE Guide had relatively the most observations of SA. In all guides, Level 1 SA was observed more frequently than Level 2 or 3 SA. Discussion: SA is an important factor in clinical decision-making and patient safety. The challenging aspect is how to best teach and assess SA in medical education. Simulations, such as informative and/or summative OSCEs, are considered a valuable and safe way to do so. Inter-rater reliability can be improved using tool training sessions.