Question: Can software that harnesses cognitive psychology research improve real‐world learning?
Answer: Yes
Experiment 1:
Preparing for the MCAT (N = 1,578) In preparation for the Medical College Admission Test (MCAT), a popular test prep company’s learners studied topics in the Amplifire platform over the course of several weeks. In the same time span, they took practice tests that each covered some of those topics.
Analysis: Within each learner, we compared the proportion correct when Amplifire on a topic was used before a test on that topic versus when Amplifire on a topic was used after a test on that topic. We then averaged those proportions across learners to determine the overall impact of Amplifire.
Finding: Across topics, the proportion correct on tests was an average of 7.4% higher when Amplifire was used before the test (vs. after).
Caveat: The order of Amplifire modules and simulated exams suggested by the test prep company could have biased the results.
Additional Analysis: Each question in Amplifire has an underlying learning objective,which is shared with one or more items on the practice tests. We used that relationship to evaluate the effect of Amplifire before vs after the test.
Experiment 2:
Preparing for the MBE (N = 3,352) In preparation for the multi‐state bar exam (MBE), a popular test‐prep company’s learners studied topics in the Amplifire platform.
On each MBE topic, each learner mastered 0% to 100% of the material in Amplifire, as well as using several other study methods.
Analysis: Within each learner, we correlated the proportion of a topic mastered in Amplifire with the proportion correct on that topic on a simulated MBE. We then averaged those correlations acoss learners to determine the overall impact of Amplifire.
Learners (N = 1,253) who did not use the platform or who mastered 100% of every topic were excluded (no variance Æ no correlation).
Finding: When learners used Amplifire before they learned something, they were 4.9% more likely to answer a related test question correctly than if they used Amplifire after p < .001).
Implication: Mastering all of the material should increase a learner’s proportion correct by 3.6% (vs. mastering none).
