Pediatric Hospitalists' Performance and Perceptions of Script Concordance Testing for Self-Assessment

Mary C. Ottolini, Department of Pediatrics, The Barbara Bush Children's Hospital at Maine Medical Center (MC Ottolini), Portland, Maine. Electronic address: Mottolini@mmc.org.
Ian Chua, Department of Pediatrics, Children's National Medical Center, George Washington University School of Medicine and Health Sciences (I Chua and J Campbell), Washington, DC.
Joyce Campbell, Department of Pediatrics, Children's National Medical Center, George Washington University School of Medicine and Health Sciences (I Chua and J Campbell), Washington, DC.
Martin Ottolini, Department of Pediatrics, Uniformed Services University of the Health Sciences (M Ottolini), Bethesda, Md.
Ellen Goldman, George Washington University Graduate School of Education and Human Development, George Washington University School of Medicine and Health Sciences (E Goldman), Washington, DC.

Abstract

OBJECTIVES: The cognitive expertise of Pediatric Hospitalists (PH) lies not in standard knowledge but in making decisions under conditions of uncertainty. To maintain expertise, PH should engage in deliberate practice via self-assessments that promote higher-level cognitive processes necessary to address problems with missing or ambiguous information. Higher levels of cognition are purported with Script Concordance Test (SCT) questions compared to Multiple Choice Questions (MCQ). To determine if PH use higher levels of cognition when answering SCT versus MCQ questions and to analyze participants' perceptions of the utility of using SCT self-assessment for deliberate practice in addressing clinical problems encountered in daily practice. METHODS: This is a mixed methods study comparing the cognitive level expressed according to Bloom's Taxonomy by PH answering MCQ versus SCT questions using a "think aloud" (TA) exercise, followed by qualitative analysis of interviews conducted afterward. RESULTS: A significantly greater percentage of comments were coded as higher cognitive processes (apply, analyze, evaluate, and create) for SCT versus MCQ (74% vs 19%) compared with lower order (remember, understand); chi-square P < .00001. Analysis of interviews revealed 6 themes. CONCLUSION: SCT questions elicited higher level cognition essential to clinical reasoning compared to MCQ questions. PH-indicated MCQ questions measure standard knowledge, while SCT questions better measure decision-making under conditions of uncertainty. PH-perceived SCT could be useful for deliberate practice in Pediatric Hospital Medicine decision-making if they could compare their rationale in answering questions with that of experts.