Most of the time, when people analyze the effect of an optional educational activity, they run into a correlationversus-causation problem. For example, using flash cards is associated with higher exam scores. But it’s harder to say whether the higher exam scores are because of the flash cards. It’s easy to imagine that the type of student who would make flash cards is also the type of student who would earn a high score on the exam; the flash cards might not have anything to do with it.
To avoid this problem when estimating the effect of Amplifire, we compared people to themselves. On simulated MCAT exams, learners were correct almost 15% more often when questions were related to what they encountered in Amplifire before the test (where they had no control over which concepts were tested). Completing all of Amplifire translates to several points—and a significant percentile boost—on the MCAT. For example, a learner in 2016 who would have gotten a 513 would instead earn a 516 by completing the offered Amplifire modules. That learner would move from the 90th to the 95th percentile, moving past half the students above her.