How Evidence-Based Medicine Can Provide Solutions for AI's Most Common Errors

Picture a sleep-deprived intern at two in the morning, reviewing the chart of a patient with new heart failure. She sees a confusing set of lab values and types a question into an artificial intelligence (AI) chatbot. The AI platform replies with a confident, well-organized differential diagnosis with citations. The prose is fluent. The recommendations sound correct. Unbeknownst to our intern, several of the citations provided by the AI are fabricated, and the fluent prose disguises outdated recommendations and shaky clinical reasoning.

This scenario is no longer hypothetical for the trainees whom we teach. Learners already turn to large language models for answers to clinical questions. Patients arrive with convincing AI-generated printouts. And while we search for positive applications of generative artificial intelligence in medical education and clinical medicine, my work with medical students and faculty has encouraged me to ask: Are we doing enough to prepare our learners to be critical consumers of AI-based information?

We have barely begun to prepare our learners to manage the flood of generative AI tools that have entered common and clinical use, and we risk missing an opportunity to develop learners’ skills in critically evaluating evidence produced by these tools. Fortunately, evidence-based medicine (EBM) provides educators with a framework of rigorous inquiry and skepticism that makes it an excellent approach for teaching about AI-generated information.  

From the classroom to the bedside, it is up to medical and health professions educators to scaffold EBM skills so that our learners can directly connect them with the most common errors propagated by AI technologies in clinical contexts. My Harvard Macy Institute Program for Educators in Health Professions project, Prompt, but Verify, introduces medical trainees to these strategies as they use generative AI to develop rigorously researched patient education materials. Following the development of this five-module, asynchronous course, I share three common AI concerns in medical education and the EBM approaches that address each.

Hallucination and Source Verification

The most obvious error created by AI technologies is hallucination. Although hallucinations are decreasing as AI models advance, it is still critical to be able to discern fact from fiction in chatbot outputs. As an example, AI tools can produce author names, journal titles, and DOIs that do not exist, attached to claims that sound reasonable. Recent studies show just how convincingly current AI tools can manufacture supporting evidence on demand, with leading models producing inaccurate citations in a growing share of generated medical content.

EBM approaches in journal clubs and research courses provide students with source verification skills that can be leveraged to reduce the impact of AI hallucination. Learners must ensure that every claim has a citation, and every citation gets reviewed for accuracy. In my course, we ask learners to maintain a verification table, listing AI-generated claims alongside the source, the source type, and a judgment of whether it supports their claim. The exercise is unglamorous, but when learners have personally chased a hallucinated DOI to its non-existent source, they may be more cautious about trusting AI's seeming omniscience.

Cognitive Biases and AI Literacy

A second common error can be harder to spot because it can hide behind good writing. Cognitive psychologists call it fluency bias. Polished, grammatical, confident-sounding AI output can activate the same heuristics we rely on when we read a well-written review article, even when the underlying evidence is thin. Studies also warn of 'automation bias,' or the uncritical acceptance of machine-derived data, a phenomenon that can have substantial impacts on physicians' diagnostic reasoning.

Since AI tools can make an underpowered observational study sound as persuasive as a Cochrane review, it is especially important for learners to develop AI literacy skills that enable them to make productive use of these tools. EBM encourages a critical appraisal of evidence strength, the same skill our learners already practice when they rate studies for evidentiary quality. In EBM, we teach learners to ask important questions about every claim that survives source verification: What kind of study is behind it? How large is the sample or population? How well controlled? How generalizable? When learners pause to ask whether a fluent recommendation rests on a randomized trial or on a single case series, the fluency stops serving as an illusory marker of knowledge.

Algorithmic Biases and the Majority Report

A third error is algorithmic bias. Learners already appraise external validity when they ask whether a trial population resembles the patient in front of them. When asked of an AI system, the same question has three layers that AI users must consider. When applied to an AI system, the same question has three layers that users must consider. First, representation bias occurs when the training data do not adequately reflect the populations for whom the system will be used. Second, design bias refers to the variables and proxies emphasized because of choices made by the system’s creators. Third, deployment bias occurs when the system performs differently across patient groups in real-world use. Evidence suggests that these biases can cause AI systems to amplify historical and systemic biases in biomedical research when used uncritically.

In addition to addressing standard external validity questions when reviewing AI-generated materials, we ask students to engage in a practical EBM exercise we call the “majority report.” In essence, we ask learners to run the same prompt several times in different AI systems, adding or removing details that might cause the models to respond differently. Factors like patient race, age, and sex can cause different AI systems to reveal invisible biases in the training data, the historical research record, or in the question itself. If the recommendations shift in clinically meaningful ways that the evidence base does not justify, the model has revealed something about itself, and the savvy learner can see why we cannot accept AI output on faith. By seeking consensus across multiple systems, it may be possible for our learners to use AI in ways that harness its strengths while recognizing and mitigating its biases.

Creating a Future-Proofed Approach with AI-EBM

EBM is not without its flaws, and it is due for an update that recognizes some of the ways that its practices can justify decisions that are not in patients’ interests. However, the skills taught through EBM, such as source verification and critical appraisal, are exactly the skills that are most necessary as AI enters clinical practice. If our learners leave training fluent in prompting but unpracticed in skepticism, we will have lost something it took generations to build. To prevent that loss, we must encourage our learners to prompt, but verify.

Daniel A. Novak, PhD (Educators ‘26) is Director of Scholarly Activities and Associate Professor in the Department of Social Medicine, Population, and Public Health at the University of California, Riverside School of Medicine. HMI has shaped Daniel's career by connecting him with a community of educators who treat instructional design as an evidence-informed scholarly activity. Daniel's areas of professional interest include medical student research, AI, and learning science. Daniel can be contacted via LinkedIn or via email.