Being tested on material (rather than re-reading it) is one of the most powerful ways to improve your memory.
But classrooms today still rely heavily on reading chapters, sitting through lectures, and passively watching videos. The tests that students do encounter are designed to diagnose knowledge, not improve it.
This is a shame, because research shows that harnessing phenomena like the testing effect in the classroom can make a substantial difference in student performance.
At Amplifire, we make software that employs many findings from cognitive science—including the testing effect. As a result, the Amplifire platform significantly improves students’ performance in the classroom.
But what about the most extreme learners? Can cognitive science really make a difference to people who are also spending hundreds of hours diligently learning outside Amplifire?
We thought so, and we put this idea to the test with learners who want to become physicians and are preparing for the Medical College Admission Test (MCAT). Typically, these learners study several hours per day for two months before the exam.
But how would we tell if Amplifire has an effect? We can’t have someone take the MCAT twice, once with and once without Amplifire.
Of course, we could compare people who used Amplifire against people who didn’t (called a between-subjects design). But if we saw a difference, could we really attribute it to Amplifire? There are all sorts of reasons that people who chose to use Amplifire might do better. They could be more motivated; they could have better study habits; they could have more background knowledge…it would be impossible to say whether differences in test scores were because of Amplifire.
So we came up with a different approach. We compared people to themselves.
While preparing for the MCAT, learners at one of our test-prep clients encounter practice exams interspersed with Amplifire use. In a given week, they might use Amplifire on a few dozen topics. At the end of that week, their practice exam covers some of those topics and some topics that they haven’t gotten to yet. They might get to those topics the following week, or the week after.
A very simplified illustration of their experience might look like this, where a learner takes a practice exam that covers Topic A and Topic B. The learner happens to use Amplifire for Topic A before the practice exam, and uses Amplifire for Topic B after the practice exam.
Our hypothesis was that using Amplifire on a topic before a practice exam that covered that topic (relationship shown in blue) should improve exam performance on that topic. Using Amplifire on a topic after a practice exam that covered that topic (relationship shown in red) should not affect exam performance on that topic. (The benefits of Amplifire cannot travel back in time—yet!)
We analyzed data from 1,578 learners preparing for the MCAT and presented the findings at the 58thAnnual Meeting of the Psychonomic Society. The data supported our hypothesis:
Remember, practice-exam performance from each learner showed up in both bars. This within-subjects design means that the difference can’t be due to intelligence, motivation, education, study habits, or anything else that varies between learners. The difference is due to Amplifire.
An increase from 55% correct to 60% correct may not seem like a lot, but it translates to a reliable increase in MCAT exam score and rank. This improvement might mean getting into the medical school you want (say, Duke) versus going somewhere else (say, UNC).
This impact is especially noteworthy given all the other work that these learners were doing to prepare for the MCAT. They only spent an average of about 15 hours in Amplifire. What if they had spent 30? Indeed, what if they spent less time on unproductive activities like re-reading or watching videos, and spent that time in Amplifire instead? They might have been able to spend less total timeon test prep, but get a better score.
Stay tuned for our next post, when we describe another within-subjects analysis of hopeful lawyers preparing for the bar exam. Spoiler alert: Amplifire has a big impact there, too.
If you’d like to get your hands on Amplifire, reach out here.
