Learning Versus Performance: An Integrative Review

Authors

Nicholas C. Soderstrom and Robert A. Bjork
Department of Psychology, University of California, Los Angeles

Abstract

Knowing how to manage one’s own learning has become increasingly important in recent years, as both the need and the opportunities for individuals to learn on The primary goal of instruction should be to facilitate long-term learning—that is, to create relatively permanent changes in comprehension, understanding, and skills of the types that will support long-term retention and transfer. During the instruction or training process, however, what we can observe and measure is performance, which is often an unreliable index of whether the relatively long-term changes that constitute learning have taken place. The time honored distinction between learning and performance dates back decades, spurred by early animal and motor-skills research that revealed that learning can occur even when no discernible changes in performance are observed. More recently, the converse has also been shown—specifically, that improvements in performance can fail to yield significant learning—and, in fact, that certain manipulations can have opposite effects on learning and performance. We review the extant literature in the motor- and verbal-learning domains that necessitates the distinction between learning and performance. In addition, we examine research in metacognition that suggests that people often mistakenly interpret their performance during acquisition as a reliable guide to long-term learning. These and other considerations suggest that the learning–performance distinction is critical and has vast practical and theoretical implications.

The Goal of Instruction

Whether in the classroom or on the field, the major goal of instruction is, or at least should be, to equip learners with knowledge or skills that are both durable and flexible. We want knowledge and skills to be durable in the sense of remaining accessible across periods of disuse and to be flexible in the sense of being accessible in the various contexts in which they are relevant, not simply in contexts that match those experienced during instruction.

In other words, instruction should endeavor to facilitate learning, which refers to the relatively permanent changes in behavior or knowledge that support longterm retention and transfer. Paradoxically, however, such learning needs to be distinguished from performance, which refers to the temporary fluctuations in behavior or knowledge that can be observed and measured during or immediately after the acquisition process.

The distinction between learning and performance is crucial because there now exists overwhelming empirical evidence showing that considerable learning can occur in the absence of any performance gains and, conversely, that substantial changes in performance often fail to translate into corresponding changes in learning.

The Beauty of Multiple Choice

It’s now well understood that self-testing is perhaps the world’s most powerful trigger for turning on the mental mechanisms that create long term memory and the ability to recall something in the future. Memory storage, like most brain processes, is a pattern of neurons connected at their synapses by electrochemical processes. Retrieving a memory through self-testing strengthens that pattern better than any other technique.

Urban myth would have us believe that multiple choice tests are mostly good for quick grading. But in 2009, researchers proved this myth about multiple choice was wrong. It turns out that…

• Multiple choice tests increase the retention and recall of information.
• They are better than recall tests or essay questions or extra study for building durable memory.
• They do not promote the mental phenomenon of “retrieval induced forgetting.”

Multiple choice is not merely the recognition of a correct answer. It can more accurately be described as recognition combined with recall because the alternative answers must each be contemplated in turn and then selected as right or rejected as wrong. Multiple choice retrieval unleashes powerful effects that increase both retrieval strength and storage strength of the correct answer plus the alternatives. It’s a virtuous circle of memory fortification.

Solving the Problem of Retrieval Induced Forgetting

Research has shown that multiple choice solves a serious problem with other kinds of testing—the problem of retrieval induced forgetting which occurs when the brain is asked to retrieve a memory. For example, I ask you to name the fourth planet from the sun. If you recall that it’s Mars, then the memory of Mars is strengthened as we would imagine, but related memories become weaker—the positions of Jupiter and Venus are suppressed. This is something we would not have imagined until it was shown experimentally.

Think of it this way. The memory of new learning is in a kind of competition with the memory of related, but older learning. To avoid confusion and to help make rapid decisions, animals and people need to instantly know which memory is most up to date. To accomplish this, a mental system evolved to suppresses the strength of older, related information that may compete with the more recent and likely more relevant information. New information causes old information to be forgotten.

“ The results imply that taking a multiple-choice test not only improves one’s ability to recall that information, but also improves one’s ability to recall related information.” — Little & Bjork, UCLA

UCLA

Enhancing the Memory of Related Information

Multiple-choice tests do not damage access to related information. Just the opposite, they enhance the retrieval strength of related information. In multiple-choice tests, a learner’s brain is asked to compare and contrast the truth of the competing alternatives. This strengthens the memory trace of those alternatives—quite the opposite of what occurs in the brain during a recall test, where forgetting of the alternatives is encouraged by the suppression of related memory.

Multiple Choice Plus Confidence in Amplifire

Confidence appraisals magnify the superior effects of multiple-choice testing. Thinking about one’s confidence focuses the brain’s resources and attention through activated dopamine circuits. The spotlight of consciousness is brought to bear on the question at hand. With confidence focusing attention on the plausible alternatives of a well constructed multiple-choice question, learning and memory benefit greatly. Students who use Amplifire report that their attention is focused, that learning occurs more rapidly, and that future recall is easier.

Amplifire Crosses 2 Billion Learner Interaction In Its Adaptive Learning Platform

BOULDER, Colo., May 13, 2019 — Amplifire, today announced it has reached more than 2 billion learner interactions in its patented adaptive learning platform. That’s a 9% increase in user interactions from the previous year. The company has also added more than 827,797 unique learners to its platform during this time period.

Amplifire engages learners and creates memorable learning experiences by adapting to each learner in real time. The platform detects and corrects knowledge gaps and misinformation that can lead to poor performance and cause errors.

The system includes a robust reporting suite that identifies misinformation, gaps, and struggle at the individual, team, location, and systemic level. Organizations can use this to improve performance. With 2 billion learner interactions, the system is able to deliver insights on learner behavior, predict who is at risk, and intervene where necessary to keep learners on the path towards mastery.

“Reaching 2 Billion learners is a a significant milestone, ” said Nick Hjort , SVP of Product and Development. “Over the past three years, Amplifire has seen learner interactions grow exponentially which reflects that Amplifire continues to be a leader In a crowded education tehnology market.”

“It has been an exciting year for us,” said Bob Burgin, Amplifire CEO and Founder of the Healthcare Alliance. “We continue to see growth in the Higher Education market while expanding our reach in the Healthcare and Call Center space. More importantly, we are making a difference by helping individuals focus on areas where they have knowledge gaps or are struggling so they can perform at their highest potential. We remain committed to improving learning efficiency.”

Learn how Amplifire improves performance, saving organizations money and time at https://amplifire.com/case-studies/

The Testing Effect: Putting Mom to The Test

Remember those flashcards mom made you cycle through every morning while she drove you to school? Times tables, vocab words. Then she’d start quizzing you on your spelling words. As much as you might hate to admit it, these methods worked. You remembered the information she was trying to pound into your head. But why did it work? The answer: the testing effect.

The testing effect has been given many names over the years—active recall, retrieval practice—but it all refers to the same mechanism: asking your brain to remember and retrieve information on cue. And, as Henry Roediger and Jeffrey Karpicke put it, “Testing has a powerful effect on long-term retention.”

Some might say, “Flashcards are a lot of work. What about just re-reading? That’s got to be as good as testing, right?” Wrong. Re-reading material might make you better at reading the test questions, but making yourself pull the information out of your brain makes you better at answering the questions.

Studies have compared students who prepared for an exam by testing themselves against students who prepared by reading. The result: the pre-testing students received higher scores on the exam than the reading students. Furthermore, the success of testing is linked to timing. The power of the testing effect increases as more time passes between the practice test and the actual exam.

Research has also shown that multiple practice tests can further improve retention and test performance. Repeatedly asking your brain to dig around and produce sought-after information creates stronger connections and retrieval pathways in the brain. In one study, students who repeatedly practiced retrieval (testing) doubled their proportion of correct responses on the final test compared to students who only practiced retrieval once.

So what can we learn from the testing effect? Not only are quizzes the way to go when we want to remember information, but also—at least when it comes to flashcards and spelling quizzes—mom was right.

Fact or Fiction: Learning Styles Lead to Better Learning

Have you ever been pushed into group work even though it wasn’t necessary for the task? Or build a 3-D model of your department’s workflow? Such activities may trigger a learner’s perceived learning preferences—auditory, visual, verbal, interpersonal, kinesthetic (tactile/physical)–but don’t result in better recall of the learning information. Educators and education institutions have long believed that learning styles not only matter but are the key to effective learning. Many educators have been encouraged to create learning experiences based on these “styles” to improve learning outcomes. But, do such experiences actually result in better learning? And, do these learning styles help instructors understand how learning actually happens?

An individual’s learning style can be determined by a learning inventory questionnaire, typically VARK or Index of Learning Styles. Instructors use the insights from these questionnaires to tailor their instruction to meet the perceived learning styles of their audience. While this knowledge is interesting, a learner’s perceived learning style should not be the sole determining factor for instructional methodology or keep educators from teaching in a manner best suited to their content. Learning styles should be viewed as learner preferences, not stand-alone requirements for learning.

Many learners believe they can only learn via a specific style. For example, after taking the questionnaire, a learner is told she is an auditory learner and should listen to lectures in order to learn the information well. Another learner believes he is a visual learner and says, “I can only learn if there are videos.” A test went well for another learner because he used hand-made flashcards, so he believes he is a verbal learner. Yet another learner reviewed for an exam by building models of molecules out of pipe cleaners and cotton balls, so he believes he is a kinesthetic learner. However, years of research calls these claims into question. In a study published in 2017 researchers found that there is actually no association with performance and a learner’s believed learning style, or rather preference. In fact, this persistent belief in learning styles is due to a type of cognitive bias known as confirmation bias: a learner believes something to be true, and when the belief is confirmed by observation, she neglects contrary evidence.

Harold Pashler, Mark McDaniel, Doug Rohrer, and Robert Bjork published work in 2009 that concluded, “there is no adequate evidence base to justify incorporating learning-styles assessments into general educational practice” (2009).

However, learning styles shouldn’t be disregarded entirely. Instead, they should be tied to the content rather than the learner. In other words, a play should not be just read, but performed. Art is to be seen and experienced. Anatomy and physics should be practiced in a laboratory. The incorporation of different learning styles when aligned with the content likely will increase engagement from the learner. Additionally, educators must understand what is going on in the brains of their learners in order to make their instruction more effective.

The brain does not work in isolation: visual cues aren’t interpreted separate from auditory cues. Signals from all senses are transmitted simultaneously into neural pathways where memories are formed. Learning occurs when these memories are retrieved. Memories are made after associations (emotional, physical, etc.) have been formed and the act of retrieval has been practiced. Forgetting (when the brain is unsuccessful in retrieving the information it is looking for) may seem like a bad thing, but is a powerful force that encourages deeper memory formation.

So what should educators do? Create learning experiences based on the content and utilize proven methodologies that foster long-term learning: metacognition, retrieval, feedback, direct instruction, and application. The delivery of content should consider how to maximize learner engagement, which leads to increased curiosity and in turn, memory formation, while remaining focused on the objective. Know your audience, but also know the science of cognition. Deliver instruction that cues the neural pathways that create long-term connections within the brain, sparking memory-formation and a thirst for more.

Forgetting Is Good for You

Matthew Jensen Hays, Ph.D.
Senior Director of Research and Analytics

We all complain about how lousy our memories are. But forgetting is one of the most important tasks your brain undertakes. In fact, your brain intentionally and immediately discards almost everything you experience. Sounds persist for a few seconds before they are irretrievably lost. Images are trashed after at most a half-second. Try it yourself: Close your eyes and count the letters in this sentence.

Even when something survives all of these filters, it usually vanishes soon after. How many times have you been introduced to someone only to have no idea what their name is after a minute or two?

What’s going on here? Why is your brain so intent on throwing away everything you work to learn?

A popular myth is that you’ve got limited space and your brain is aggressively trying to make room for new information. In reality, anything you’ve stored lasts nearly forever. You just lose access to it—which is why you have an “ohhh yeah” moment when you find your keys. Of course that’s where you put them. The information was in there. You just couldn’t access it.

It turns out that forgetting—in the form of this loss of access—is an adaptation. Consider the horror of perfect memory: You’d remember with perfect clarity every time you made a mistake, and every time someone was rude to you, and every time you hurt someone’s feelings. You’d call to mind every phone number—yours and others’—equally well. You’d vividly recall every time you bit your tongue, and every time you inhaled. You’d go insane.

So how do we remember anything?

Researchers in the psychological sciences have catalogued dozens of triggers that cause your brain to hold onto information. An example is when you encounter something repeatedly, especially if those encounters are spread out in time. Your brain expects it to come up again in the future, and so information about it is preserved. This “spacing effect” works on such a fundamental neurological level that it is even found in the not-quite-brains of sea slugs!

Another trigger is the retrieval of information from in your brain. People who prepare for a test by calling the material up from memory get good at bringing it up from memory. People who prepare for a test by re-reading the material get good at reading the test questions, but not knowing the answers.

These triggers operate on mechanisms beyond your conscious control. We’ve all thought “Jeez I really need to remember this” only to have it suffer the same fate as almost everything else. You may think something is important, but your brain doesn’t care what you think.

Instead, you need to use the triggers to get your brain to agree with you that something is important. For remembering someone’s name, you can test yourself under your breath. For more complex materials, you may want to rely on software that incorporates research on these cognitive triggers.

And the next time you blank on a movie star’s name—but still recall the entire theme song to a cartoon you haven’t seen since you were six-going-on-seven—remember that these are side effects of your brain generally taking pretty good care of you.

67% Improvement in call center performance

Call Center Challenges

Two of the biggest challenges organizations face are employee turnover and the subsequent poor performance of new employees. High turnover not only impacts the bottom line, it affects customer satisfaction and team morale.

Making matters worse, hiring and onboarding new employees is surprisingly costly. According to Bersin at Deloitte, turnover costs range from tens of thousands of dollars to 2x salary. Those astonishing figures include sourcing, interviewing, reviewing, reference checking, hiring, training, and unproductive time (4 weeks to 6 months).

Onboarding Training

The solutions are clear: First, reduce turnover by implementing practices that lead to employee retention. Second, institute training methods that onboard new hires as quickly as possible. And third, use principles from the brain sciences to make training stick in the minds of new hires so they rapidly become productive and satisfied in their work.

This study looks at Amplifire’s impact on training, turnover, and financial outcomes when it was deployed at scale in large organizations.

90.8% Less Training Time for New Hires

Amplifire’s adaptive algorithms personalize training so learners focus only on content they don’t know or are uncertain about. This saves time and money. In this case, Amplifire reduced training time from 240 minutes to 22 minutes. The total time to deliver Amplifire training to the 209 agents at this call center was 76.6 hours. Amplifire saved 683 training hours.

Employee Proficiency 5 Months Faster

Productivity is important to any organization’s success, and new hire productivity depends on effective training. For this technical services and installation firm, the number of jobs completed each day is one of the most critical profitability components. Margins are razor thin. Amplifire improved the jobs-per-day completion rate by 25%. Amplifire-trained technicians started out with 2.5 jobs per day, whereas non-Amped technicians took 5 months to reach this level of proficiency.

Learning That’s Gamified

All games use powerful triggers that cause learning, long term memory, and motivation. This is true of board games, video games (which now generate more revenue than movies), and sports. You can think of triggers as being out here in the real world in the form of things like people, nature, books, videos, and techniques in software. Certain triggers cause specific brain circuits to switch on, causing learning and long term memory so we can then remember what we’ve learned at a future point in time. Triggers are the cause, and learning is the effect. Some of the most effective triggers work through emotion and attention—two hallmarks of games.

These are the six gaming triggers built into the Amplifire algorithm:

Uncertainty propels game players forward because they don’t know exactly what is coming next. Uncertainty triggers curiosity, a mental state seen at right in a set of fascinating experimental results. Attention producing levels of dopamine skyrocket whenever a reward has a 50% likelihood of occurring (top curve). As Robert Sapolsky notes, “You have introduced the word maybe into the equation and that is reinforcing like nothing else on earth.” Dopamine and attention levels fall from this peak when the reward becomes predictable.

In Amplifire, the perceived reward is closing an information gap. First, you make a bet on your knowledge with the proposition that asks, “How sure are you?” That question stimulates uncertain expectations of reward—will you close the information gap? Maybe. Second, as you progress, Amplifire withdraws material that has been mastered. What’s left is increasingly harder material. This maintains high uncertainty and focused attention as you move through a module.

Feedback can boost learning by 500% when compared to non-feedback learning. In Amplifire, learners receive explanations about both correct and incorrect answers. This detailed elaboration strengthens both information storage and retrieval processes in the brain. Second, the review page shows learners precisely how they progressed through a module, from typically high levels of misinformation and doubt, to mastery of the material.

Confidence triggers a massive number of switches that affect learning. Making judgments of learning means storage and retrieval processes are activated. Asking “are you sure” results in metacognition (thinking about thinking) and causes both top-down attention and bottom-up salience. Confidence also spurs attention because it is correlated with social status—one of the most sought after personal qualities in the human experience.

Progress motivates future activity through the buoyant feeling that comes from reaching your goals. Amplifire adapts to each learner’s level of mastery so that the material is appropriately difficult, but not so hard that motivation suffers. This ensures a learning experience in a gratifying emotional state that the learner is likely to want to repeat, and repetition is a key cognitive trigger for durable memory.

Misinformation is a unique feature of Amplifire that makes clear the possibility that confidently held, but wrong information may lead to error, injury, or embarrassment sometime in the future. That emotionally alarming possibility, when revealed in Amplifire, focuses your attention on the learning so you can avoid that outcome.

A Note on Leaderboards: They are motivating in games, but create a dangerous, dispiriting risk in an educational setting because people can feel their core intelligence being judged and ranked. We believe that all people can learn enormous amounts of useful information. For some, it merely takes more time.

Study Less. Learn More.

If you’ve ever used flash cards to study for a test, you’ve probably run through a deck several times in a row. The second round felt easier than the first—a clear signal that you were learning and spending your study time wisely, right?

Actually, you were experiencing the fluency illusion. An immediate, second study session feels powerful (because the material is so familiar), but it provides almost zero benefit. Your subjective experience during learning is often unrelated to the quality of that learning.

In fact, many conditions of learning that feel difficult are far superior to the alternatives. Robert Bjork at UCLA calls these conditions “desirable difficulties.” Study protocols that feel more difficult are often much better at engaging the mental processes that support learning. Of course, the desirable part is not the extra hurdles during studying; it’s the higher test score at the end.

Unfortunately, most students haven’t read Dr. Bjork’s work. They use intuition to guide their study habits. But because what’s good for your brain is often counterintuitive, students usually wind up spending their study time poorly. For example, it’s better to let some time pass—hours or even days—before you run through a deck of flash cards again. The second round will be harder, but your score on a later test will be higher (a phenomenon called the spacing effect).

At Amplifire, we’ve assembled a Science Advisory Board that includes Dr. Bjork and other esteemed professors and researchers. We’ve based our software on thousands of pages of their research, plus work from other labs. The result: Amplifire makes a big impact on test scores and grades.

One of our clients helps law students become lawyers by preparing them for the Multistate Bar Examination (MBE). These learners spend hundreds of hours preparing, and must continually make decisions about how to spend that time. Then they take the test and get the score they have to live with.

We analyzed data from their practice exams to determine whether they would have scored better if they had used more Amplifire.

This type of analysis can be a bit tricky. Comparing two groups of learners (a between-subjects design) would either be unethical or invalid. Obviously, we couldn’t withhold Amplifire from some learners and require it of others. On the other hand, if we let learners choose whether to use Amplifire, any differences we found might have been due to study habits or motivation or anything else that varies from person to person.

We had to compare learners to themselves. But this can be tricky, too. We couldn’t have each student take the MBE twice—once with Amplifire and once without.

Instead, we looked closely at each individual’s behavior in our software.

Getting ready for the MBE requires studying dozens of topics. Learners can choose to do more or less work in Amplifire on each of those topics.

Our hypothesis: The more Amplifire you do, the better you’ll score. For example, if a learner completed 20% of Amplifire on Topic A, but 80% of Amplifire on Topic B, we expected them to score relatively higher on Topic B than on Topic A. This within-subjects design controls for the effects of aptitude, motivation, sleep quality on the day of the exam, and everything else that varies between learners. Any observed differences must therefore be due to Amplifire.

We analyzed data from 3,352 learners preparing for the MBE and presented the findings at the 58th Annual Meeting of the Psychonomic Society. The data supported our hypothesis. The more Amplifire the learners did—independent of everything else they could have done—the better they scored.

Doing all available Amplifire work increased the proportion correct on a simulated MBE by 3.6%. That may not sound like much, but keep in mind that these learners log hundreds of hours of other work on those same topics. The fact that Amplifire nevertheless made a difference is a testament to the ability of the software to adapt to what learners need according to the principles of learning and memory. Also, the MBE is a pass-fail exam. For many people, a single correct response is the difference between becoming a lawyer and becoming a lawyer’s assistant.

The cognitive phenomena that Amplifire harnesses are robust and have been demonstrated in many different contexts. The spacing effect, for example, is such a fundamental property of how brains work that it can be found in the rudimentary nervous systems of sea slugs.

Yet as we enter 2018, many educational settings still feature conditions of learning that merely feel better, but are actually far from optimal—and can even be counterproductive. Learning researchers have lamented for decades how infrequently their findings make their way into the classroom and other educational settings.

The benefits of Amplifire provide yet another validation of these scientists’ efforts, and give them reason to celebrate. Through our software, their findings are now reaching millions of learners.

If you’d like your students to start benefiting from hundreds of counterintuitive discoveries about how people learn, reach out here.

How to Get Real Insight From Your LMS

Chief Learning Officers spend copious amounts of time and money on learning management systems – as they should, but there’s one big problem: they can’t measure the precise impact the LMS has on not only training goals but overarching business objectives.

Most learning management systems lack the depth of data insights necessary to measure the ROI of your training programs. If your learning management system lacks the following analytic capabilities, you may need to integrate it with a knowledge engineering platform.

Three Essential Employee Insights to Pull From Your LMS

1. Confidently Held Misinformation (CHM) – This is the most important metric you’ve never heard of (unless you’re a huge nerd and read all of our content). What is it? It’s the precursor to error. The sooner you detect it, the less organizational risk you face. We’ve found that professionals across industries possess varying degrees of CHM. The variation (even among employees who report to the same boss) is astounding. CHM data goes beyond traditional reporting and prioritizes struggle areas based on learner-assessed confidence. Imagine the long-term benefits of focusing on high-risk topics and high-risk employees before lower-risk areas. Imagine the immediate reduction in error and injury.

2. Struggle Heatmaps – Your LMS (and/or its integrated, data-rich learning platform) should provide insight into four particular types of risk: Misinformation, Regression, Understanding, and Systemic. The risk posed by a lack of understanding can be visualized in the form of a struggle heatmap. Your reporting functionality should highlight employees who struggle to master a question or topic after several attempts, even after viewing the correct answer. Actionable analytics like this can be sent directly to managers who can help high-risk employees reach their potential.

3. Misinformation to Mastery – Reporting that compares starting knowledge to ending knowledge should not only provide a visual representation of success but should quantify the monetary ROI of your training program. For example, if you know the average annual cost of a specific problem you’re trying to solve as well as the number of incidents per year directly caused by misinformation, you’ll have an accurate estimate of the financial impact of relevant training content. Employees who can confidently deploy knowledge under pressure are vital to financial success.

These three data insights are just the beginning. Learn how you can integrate your LMS with a data-rich learning platform by contacting us.