The Testing Effect While Preparing for Medical Licensure Exams

Flight simulation is an important component of nearly every pilot’s curriculum, whether a novice learning to fly for the first time, or a military pilot using a Full Mission Simulator (FMS) to train at the highest level.

And it makes complete sense, right? Before I actually fly the plane, let me gain the confidence to step into the cockpit through simulation of the experience…and let that experience be as real as possible.

Allow me to learn from my mistakes in the classroom so that I can refine my skills, and navigate tricky scenarios when I am in the air. The high-fidelity benefits of simulation training span many field and disciplines, where deliberate practice results in enhanced learning. And preparing for medical licensure exams no different.

What is the testing effect?

The ‘testing effect’ is a well-documented phenomenon describing the potent learning that takes place when individuals are tested on previously learned material. Testing has been shown to produce better retention than re-reading or re-studying material in multiple studies.

Many of us think of testing as a means to assess learning. However, there is a substantial body of research suggesting that testing is, in and of itself, an impactful way to learn material and commit information to long-term memory. Multiple theoretical explanations have been formulated to account for the strong effects testing has on learning, most of which are complementary hypotheses.

If you are reading this, it is likely that you will be taking a medical licensure examination soon. Countless hours will be spent learning new material, reviewing old material and applying this information to clinical scenarios.

On a regular basis, you will be making choices between different study methods. You may be using different types of resources such as video lectures, classroom lecture notes, review books, and question banks. Although each of these resources will likely serve a role in your studying, their impact on your learning and future examination performance may not be of equal value. And in the end, it will be the tests themselves that will have the greatest impact on your learning.

Learning is by all means an “active” process involving the retrieval and reconstruction of knowledge. Practice retrieval, the process of using cues to actively recall and reconstruct knowledge (ex: group discussion, reciprocal teaching, taking practice tests), has been shown in multiple studies to be the single most effective method of retaining knowledge (Roediger, and Karpicke, 2006). Deliberate practice through multiple-choice type questions has been shown to foster both improvement and expertise.

The Different Study Techniques

In one study performed at Purdue University, and published in Science, researchers looked at the effectiveness of three distinctly different study methods (concept mapping, retrieval practice, and repeated study) to assess their impact on knowledge retention. Below are clear descriptions of each of these learning methods:

• Concept mapping: recreating subject matter through diagrams to organize and encode meaningful relationships among concepts.
• Retrieval practice: process of using cues to actively recall and reconstruct knowledge (ex: group discussion, reciprocal teaching, taking practice tests)
• Repeated study: act of rereading notes, books, or other study materials

When comparing each of these methods head-to-head, it became evident that for many students, time spent rereading notes and review materials would be better utilized performing more active methods of learning, such as practicing retrieval of important concepts.

Of the three groups tested, students who took practice tests were found to have retained nearly 50 percent more information a week later than students who used the other two methods of studying. These results have been supported by further studies, contradicting the perception of many students preparing for medical licensure exams.

In repeated scenarios, when students were asked to make predictions of the long-term learning benefits of different methods of study, they indicated that both rereading of information and concept mapping (diagrammatic reconstruction of knowledge) would produce more learning than test taking.

While students believed they would do far better after elaborative studying as compared to taking practice exams, the latter group (retrieval practice) was found to have a greater impact on both learning and examination performance.

This phenomenon, the “testing effect” demonstrates that acts of retrieval have a potent effect on learning while enhancing long-term retention of the information being tested. These findings parallel research from Washington University in St. Louis suggesting that the common practice of rereading material is no more effective in improving learning than performing a single, initial reading of the text.

The practice of spending large amounts of time rereading materials despite limited benefit to memory retention is a phenomenon coined “labor-in-vain” learning. This method of preparation is pervasive among test-takers (medical students included), despite being ineffective. The process of practice retrieval has been shown to be of far greater value to learning than previously thought.

The Importance of Testing

Regardless of the level of formal guidance that programs offer for licensure exam preparation, self-study and self-guidance are the primary means by which students and residents prepare for standardized tests.

The incorporation of taking practice items is important for assessment purposes, but the true impact comes from the actual learning that takes place during the process of testing itself. In repeated studies, individuals have been shown to lack awareness of the testing effect and its potent impact on learning itself. In multiple cases, when asked to select only one learning modality by which to prepare, the majority of students resort to traditional study of material through rereading of books. Using active practice retrieval is typically thought of as an adjunct means of preparation rather than a primary study tool.

So remember, just like the fighter pilot, you are preparing for a climactic event. Your test may not come in the air, but it will come on the day you arrive to the examination center. And when that day arrives, you’ll want to be prepared, knowing that you have navigated the course thousands of time by taking thousands of practice questions.

Roediger, H.L., and Karpicke, J.D. (2006) Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. Psychological Science 17(3):249-255.