How to tell whether a game teaches

Two separate gates

Both gates sit during development rather than after delivery. The first is accuracy: whether the content the game presents holds up against subject-matter expertise. The second asks whether the target group learns from it, and it takes learners to answer: expert review establishes that the content is correct, and learner testing establishes that it teaches.

Weston and colleagues compared revisions guided by subject experts with revisions guided by the learners the material was built for, and found that only those informed by learner feedback improved learning. Expert review remains the check on accuracy, which is a separate question.

Playtesting is formative evaluation

Tessmer’s work on formative evaluation supplies the term: playtesting, in an educational game, is formative evaluation, the same activity the instructional design literature already has a method for.

The Design-Based Research Collective set out build-test-revise as a recognised research method in its own right: a structured, repeated cycle of building, testing with learners, and revising, with its own standing in the literature.

Engagement is measured

Wouters and colleagues found no significant motivational advantage for games over conventional instruction on average. For testing, that makes engagement a design outcome to verify. Ask playtesters directly whether they wanted to continue, as well as whether they completed what was asked of them.

Fullerton’s playtesting practice adds two requirements: test with people from outside the design team, and revise between rounds.

Five users is a different test

Nielsen’s rule says five users surface most usability problems, and it is sometimes carried over into playtesting. Nielsen’s finding concerns interface usability, where problems repeat quickly across users. Whether a game teaches specific content is a different question, answered by repeated rounds with the target learners.

What the test group looked like, how many rounds happened, and what changed between them are the questions that separate evaluation from ceremony.