An Approach to Help Students Test Smarter

Those pursuing careers in a medical or health profession may be drawn to their field from an innate desire to help others and make a difference, a passion for lifelong learning, or the prospect of future career opportunities. Trainees working toward a career in one of these professions may not relish taking multiple-choice question (MCQ) exams, yet are faced with a litany of MCQ assessments from individual courses to standardized exams. High-stakes MCQ licensure assessments abound in medicine (USMLE) and in the health professions, including dentistry (INBDE), pharmacy (NAPLEX), and nursing (NCLEX), to name a few.

From the faculty perspective, the focus is often on the task of writing well-constructed MCQs for their course assessments to prepare learners for licensure exams. Resources are publicly available to assist medical and health professions educators with writing quality multiple-choice question items that effectively assess learning knowledge while avoiding common cues and flaws. Once MCQ items are developed, it is up to the learner to apply their knowledge and correctly answer the question. There are resources and books, such as Make It Stick, that outline effective strategies, such as retrieval practice, interleaving, elaboration, generation, and reflection, which students can adopt to better learn, retain, and remember content.

Live Guided Question Analysis Approach

What can be done to help learners when they are not performing well on MCQs to identify their knowledge gaps? One potentially useful process is “Live Guided Question Analysis,” which describes the use of a think aloud protocol when an educator or learning specialist works with learners to help them identify why they answer multiple choice questions incorrectly. Live guided question analysis can help to circumvent the Dunning-Kruger effect, where an individual may be unconsciously unaware of their own knowledge deficits and, as a result, unable to accurately self-diagnose gaps in their knowledge. In the live guided question analysis approach, a multiple-choice practice question is displayed that the learner has not viewed before. The learner speaks out loud while they read through the question and answer options, revealing their thoughts on why each answer option may be right or wrong. Throughout the process, the educator asks questions to understand the learner’s thought process and knowledge base, but does not incorporate content knowledge in their questioning. After the learner has fully talked through the question and all the answer options, they select and then check their answer. The educator then provides concrete feedback on why the question was right or wrong and learning strategies to consider.  

Goals of Live Guided Question Analysis

The overall goal of live guided question analysis is to identify how the learner recalls and reasons through content, which reveals the etiology of why points might be lost in an MCQ assessment. There are some concerns around the amount of time needed for the process and dependency on the educator’s analysis that some learners may potentially develop. However, the advantages of this approach allow the promotion of self-regulated learning, customization of learning strategies, and evolution of interventions to align with the goal outcome.  

Live guided question analysis goes beyond learner self-diagnosis and delves into the why, using the think aloud protocol, to help the learner develop more focused and effective interventions that are directly linked to their success.  

How might you start using live guided question analysis and the think aloud protocol to support learners at your own institution or as a teaching tool for clinical reasoning? Share your thoughts on the #MedEdPearls LinkedIn discussion.

MedEdPearls are developed monthly by the health professions educator developers in educational affairs. Previous MedEdPearls explored forward feeding, growth mindset, and psychological safety.

Acknowledgement: Thanks to Michael Terao, M.D., M.Ed., Assistant Dean for Student Learning with the Office of Student Learning at the Georgetown University School of Medicine, for his review of the content and for providing a link to the Live Guided Question Analysis resources.