Normally when I’m talking with designers and developers about test questions being used in elearning or classroom-based training, the conversation takes one of two forms.
Form 1: Without testing, how do we know they learned?
For a number of individuals and organizations, testing is a standard part of the instructional process. This is built upon the belief that testing verifies the understanding of what has been learned. The test score provides a measurement that is used to report the learning that is perceived to have taken place.
Form 2: Is a test really necessary?
On the other side of the coin is a conversation arguing that testing is unnecessary and/or ineffective. Maybe testing, and the perceived measurement of verification of learning, isn’t needed – at least not in every single case. In this context, the test serves more of a compliance reporting purpose than one related to learning.
Testing as Verification
I would argue that there’s a common ground to both sides of the conversation: that testing is more about verification than actual learning. They may be coming at it from different angles – one side says testing verifies learning, the other says testing verifies participation in a learning program – but neither is saying that testing itself is an act of learning. And that’s a problem.
Asking Better Questions
I was recently asked to review the materials for a new learning program. The materials included a word document named “Test Questions”. There were 13 questions in the document:
- Five of the questions were True or False, with the correct answer being True on each one.
- The remaining eight questions were Multiple Choice, with the correct answer on each being D, All of the Above.
This test is not a powerful part of the learning process. It enables recall or recognition of a correct answer, and the pattern of correct answers dilutes even that minimal amount of value.
If our ultimate goal – especially in organizational learning – is to have learners being able to apply what they have learned, then wouldn’t it be better to have test questions apply what they have learned in context?
Earlier this year an article appeared in the New York Times called Steven Pinker’s Mind Games. The article focuses on the questions Dr. Pinker uses in his Psychologocal Science courses at Harvard.
The article is interactive and you can actually take a ten question exam using examples culled from Dr. Pinker’s work. The questions were designed to “test whether students understand the theories well enough to reason about them when they are presented away from a familiar textbook context and are applied to real life.”
Isn’t that what testing should do? These questions require that knowledge be interpreted in context, and what is more powerful learning than applying knowledge and skills in context?
I encourage you to check out the test, as it serves as a great example of how questions themselves can serve as learning opportunities if we take the time to build good questions.