A generation of college students spent countless hours at late-night pizza shops and burger joints shooting invaders from outer space without ever considering whether such a game could have benefits beyond avoiding homework.

Now a new study from the Stanford Graduate School of Education reports how some creative modifications made the classic videogame Space Invaders a powerful tool for improving the teaching of statistics. The research shows that exposing college students to "probability distributions" in the course of play  — without ever explaining or even mentioning these concepts in the game itself — was the first of two key steps to boosting students’ performance on a test of their understanding of statistics.

The second critical step was to have the students who participated in the study read a brief explanation about statistics after playing the refashioned game. The students who both played the game and read the passage had greater improvement on a statistics test than those who did just one or the other.

"The research demonstrates that even without having instructional content that maps directly onto curricular standards, games can prepare students to learn in more formal environments, such as school," according to the paper, which was published online in December in the Journal of Science Education and Technology. The study’s co-authors are Daniel Schwartz, professor of education at Stanford, and his former doctoral student, Dylan Arena, PhD ’12.

The study’s implications go beyond statistics instruction. Educators and video-game designers often think of video games as stand-alone learning exercises. "They take the teaching of declarative facts and procedures, and they gamify it," said Schwartz, who is director and founder of the AAA Lab at Stanford. "They don’t do what good games are really good at, which is to develop a foundation of compelling experience.  People forget that experience is an important teacher, even for schoolish content."

Schwartz and Arena have advanced an alternative vision for educational digital gaming that emphasizes preparation for future learning. For more than a decade, Schwartz’s research has examined whether learners benefit from discovery-based experiences: the kind of situations that require students to infer concepts, rather than receive them through a lecture or reading. "Often times these rich experiences get discounted because they do not look like the stuff that shows up on tests.  But if you see how well students learn the next thing after these discovery experiences, it turns out that they prepare students for future learning much better than if you just tell them the answer to start with," he said. 

For this study, Schwartz and Arena devised a game they call Stats Invaders, based on the classic video game released in 1978. In the original game, rows of aliens descend from the top of the screen, and the player shoots them with a laser cannon at the bottom. The player gets points for eliminating the aliens and levels up when all of the aliens in the round have been cleared. But if any of the invaders reach the bottom of the screen, it’s game over. The new version also features advancing aliens, but the physical arrangement of each attack wave is determined by a specific probability distribution: the pattern of likelihood that a given alien will descend from particular locations along the top of the screen. For example, in a normal distribution (think "bell curve"), most of the aliens would attack from the center of the screen with very few attacking on the far left or far right of the screen.

To encourage players to see the patterns, each level requires players to tune a bomb to the hidden shape of the alien ship. To do so, players have to choose which graph best represents the shape based on the patterns of attack. To be successful, players have to extrapolate the pattern from the individual alien attackers they subdue. The moment-to-moment excitement of shooting the aliens is what Arena calls "the twitch stuff". It’s the fast-paced action that keeps players on the edge of their seats.

Although Stats Invaders does not include any overt explanation of statistical concepts, the researchers wanted to see whether it would help players learn later on, when exposed to an academic text about statistical distributions.

In the study, 83 community college students began with a brief, ten-item pretest on statistical probabilities and distributions.

Next, all of the participants, except for a control group, had a 27-minute session playing one of two versions of the game. The main version, called "distribution mode", involved selecting the graph that represented the distribution of aliens across the screen, as described above.

(A comparison version, called "proportion mode", instead required players to select between two proportions representing the rate of special, bright colored aliens in the overall attack; this version was included to see whether the students would still gain intuitions about statistics without having the formal representation of graphs provided to them.)

After the game session, half of the participants who played the game (as well as half the control group) were given a passage about probability distributions to read. Finally, all of the participants took a ten-item posttest.

When Arena and Schwartz examined the results for the students who played the game, but did not read the passage, they saw little improvement from pre- to posttest compared with students who did not play at all. (While the students who played in "distribution mode" did better than their peers who played in "proportion mode", the results were not remarkable.) In this scenario, the findings suggest that students did not learn much from the game experience. But that does not tell the whole story.

Students who played either version of the game and read the explanatory passage significantly outperformed students who read the passage but did not play the game; they gained about fifteen percent more over their pretest scores. What’s more, students who both played the game and read the passage gained about thirty percent over those who only played the game and did not read the passage.

The game-plus-passage condition turned the test into what Schwartz calls a preparation for future learning assessment. The game provided the experiences that allowed the students to make more sense of the passage. In turn, the passage helped the students consolidate and explain what they had experienced.  In short, the game experience helped prepare these students to learn more effectively from the explanations in the text.

The idea for the study was hatched several years ago when Arena was reading psychologists Daniel Kahneman and Amos Tversky’s work on how people reason about probabilities and statistics. The psychologists found that people’s intuitive understandings about these topics are surprisingly poor; they have trouble seeing beyond individual cases to the overall patterns in probability. While the two scholars suggested that humans should simply rely on learning formulas, Arena had another idea: Use videogames to build up people’s missing intuitions about probabilities.

Everyday life typically doesn’t provide good opportunities for learning the statistical concepts of probabilities and distributions, Arena explained. People tend to focus on single instances or try to make causal predictions. They simply do not have a chance to engage experience in a format that helps them think about the overall pattern. Arena and Schwartz reasoned that videogames could help people develop better statistical intuitions by repeatedly engaging them in finding the overall patterns in chance events.

"Games are able to pull and focus all of our attention, and they do this by recruiting a whole bunch of monkey-brain machinery," said Arena. "You’re trying to do something that’s monumental, except that it’s on a compressed time scale so that you can achieve it in 100 hours instead of 10,000. And all along the way you get immediate feedback and well-structured input, which enables you to adjust your strategies very quickly."

Two years after receiving his doctorate, Arena has vastly expanded upon what Schwartz and he ascertained from the study. He’s co-founder and Chief Learning Officer of the Silicon Valley edtech company Kidaptive, which received a round of $10.1 million in venture capital funding in November. Its main product, Leo’s Pad, consists of a series of iPad "appisodes", which are interactive animated stories intended to promote early childhood learning. While these games cover a much wider range of lessons than Stats Invaders, they share the same approach: They are designed to enhance teaching that occurs outside the game.

Schwartz believes the study findings can be applied to other learning experiences in informal environments, such as museums. "These are designed to be compelling experiences. If you measure the learning from them directly, they may not look so good. But that’s missing their value, which is to prepare people to learn something more formal and explanatory," he said.

In one of Schwartz’s new studies, he’s partnering with the Exploratorium in San Francisco. The researchers are investigating whether the types of tinkering activities children engage in at the museum impacts what they learn from an educational video. "I think a lot of people go to the museum and say, ‘It looks like fun, but are they learning?’" he said. "This is the wrong question. Of course they’re learning something," he explained. "What people mean to ask is, ‘Are they learning anything useful for school?’ We’re going to find out."