TrueLearn Data Lab: The Forgetting Curve & Memory Retention

“Tell me and I forget. Teach me and I remember. Involve me and I learn.” – Benjamin Franklin

Excelling on Board Exams or In-Training Exams requires critical thinking, test-taking skills and deliberate preparation. These exams also require you to recall and put to use medical knowledge you may have learned months, semesters or even years ago.

In TrueLearn Data Lab: The Importance of Repetition, we demonstrated that retaking questions is beneficial for reinforcing knowledge and improving performance. In today’s Data Lab, we explore the flip side of that coin – how soon after seeing a particular concept does your performance begin to deteriorate?

What is the forgetting curve?

According to the psychological model of human memory proposed by Atkinson and Shiffrin (Figure A), information enters short term memory initially, but certain triggers, such as repeated rehearsal of the information, improve the long-term retention.

Memory decay is presented by the “Forgetting Curve.” Many studies have modeled the Forgetting Curve decay over time using different mathematical functions, including logarithmic functions, exponential functions and power functions. For instance, power function y = b t-m is favored by Anderson & Schooler (1991). In this model,  y represents memory retention, b is the fit constant for scaling, t is the time from the nth rehearsal and m is the power function rate. Figure B illustrates various forgetting curves produced by this model using different decay rates. [1][2][3]Studies have demonstrated repetition reduces the slope of the Forgetting Curve, but the incremental reduction slowed with more repetitions, producing diminishing returns to additional repetitions. Keeping this research in mind, we turned to TrueLearn practice question data to see if this phenomenon occurs within our data sets, and to explore the implications of creating an effective and efficient study plan.

Testing Memory Retention

In this Data Lab, Truelearn examined practice question data from several different exams and analyzed our users’ performance when they attempted questions for a second or third time over a 30 day period.

In our version, the average score is used to represent memory retention, therefore the y-axis is the percentage correct. For every user in the specialty, their first, second and third attempt on every question is recorded. Then the average score is calculated according to which attempt the response belongs to. The time difference (in days) between each attempt is also computed. In tested products, the same model (logarithmic curve) fits our users’ data very well (R2 > 0.9).

This analysis was repeated using data from multiple TrueLearn SmartBanks, including ABSITE practice questions, COMLEX practice questions and several others. While the parameters of the models tended to vary slightly across exams, in all cases the classic “decay curve” model fit the data quite well.

Results

In an ideal world, you could maximize your performance by continually repeating every single question and concept multiple times until firmly committed to long term memory – and then you’d repeat the questions yet again right before your exam.  

However, in the real world, you’ll have to balance not only board prep, but also course work, rotations and other commitments. Keep these conclusions in mind as you strive to prepare for your exams as efficiently and effectively as possible:

1. The Forgetting Curve is a well-documented phenomenon that does apply to your medical knowledge and affects your performance on board exams.  Naturally, if you haven’t seen a concept or practiced something for months or years, it’s going to be more difficult for you to recall that information on your Board Exam or In-Training Exam.
2. We still recommend taking every question you can to prepare for your exam – leaving questions untaken is equivalent to leaving points on the table.
3. It can definitely be advantageous to revisit questions for a second time (or maybe a third time). If some time has passed and your memory has progressed down the decay curve, retaking these questions can boost your performance and ensure easier recall on your exam day. Every few weeks, mix in some repeated questions along with new questions to avoid slipping too far down the Forgetting Curve on certain topics.
4. Which questions should you invest your study time retaking? Concepts you have mastered are more likely to already be committed to your long term memory. So, if you do have time to revisit some questions for a second pass, focus on your weakest areas and on questions you missed before. TrueLearn offers the ability to create tests in both of these ways.

References:

[1] Atkinson, Richard C., and Richard M. Shiffrin. “Human memory: A proposed system and its control processes.” Psychology of learning and motivation 2 (1968): 89-195.[2] Ohno, Takeo, et al. “Short-term plasticity and long-term potentiation mimicked in single inorganic synapses.” Nature materials 10.8 (2011): 591-595.[3] Rubin, David C., and Amy E. Wenzel. “One hundred years of forgetting: A quantitative description of retention.” Psychological review 103.4 (1996): 734.