It is hard to overstate the impact of the COVID-19 pandemic on everyday life. Downtown real estate markets collapsed as remote office work became the norm. Supply-chain issues caused widespread food insecurity and prescription medication shortages. Lockdowns even changed how birds sing.  

Education was similarly disrupted. Within weeks, over 90% of classrooms abruptly shifted from in-person to online. At the same time, the upheaval caused by lockdowns and students’ concerns about the virus caused them to report increased anxiety, difficulty sleeping, concentration problems, and greater susceptibility to distractions. 

Amplifire’s Dr. Matthew Hays and Scott Kustes teamed up with Dr. Elizabeth Bjork of our Science Advisory Board to investigate how this disruption affected learners’ ability to stay engaged while learning online. Because learners have interacted with the Amplifire platform over 4.5 billion times, there were more than enough data points to let the researchers quantify the impact of the pandemic. 

Their first step was to define what constituted an interruption. Over the course of five studies, they evaluated increasingly rigorous ways to detect within the data when a student became distracted, from simple mean-and-standard-deviation approaches to more advanced statistical methods. They then compared the interruption rate during the peak of lockdowns – April and May of 2020 – to the same period in the years before and after. The researchers had hypothesized that there would be clear increases in the frequency of interruptions when lockdowns were most intense. 

Instead, they found no effect of the pandemic.

We asked Dr. Hays to explain. “You’ll see these clickbait headlines that say ‘scientists stunned’ and it turns out they just found a bug that they didn’t think lived in that part of the country anymore,” he said. “But I really was stunned. We pulled the data in different ways and re-ran the analyses to make sure that we weren’t somehow missing an impact.”

The researchers consulted with several other experts in learning and metacognition to interpret the findings. The consensus that emerged: Because Amplifire was designed from the start to be used online, it was immune to the disruption that so dramatically affected classroom learning. 

The researchers’ findings and conclusions were published in the Journal of Intelligence, thus informing the broader scientific community about what we might expect from the next massive disruption of student learning. Further, the interruption-detection methods introduced can help educators identify frequently interrupted students (who may therefore be struggling to learn due to external factors rather than a lack of effort or aptitude). Click here to read the paper in full.