Tips on creating test questions

1. Align questions with learning objectives

Before creating the questions, you should clarify which learning objectives should be assessed. Use the following questions as a guide:

To answer the test question, do participants…

only have to reproduce memorized knowledge or use that knowledge to identify something?
Yes? Then the learning objective is reproduction.
have to apply and understand a learned basic principle independently?
Then the learning objective is reorganization.
have to transfer a learned basic principle to a new but similar task?
Then the learning objective is transfer with a focus on application.
have to examine an issue according to specific criteria?
Then the learning objective is transfer, with a focus on analysis.
have to find a solution for new questions and aspects with several different possible approaches?
Then the learning objective is problem solving.

When the learning objectives have been defined, you can choose suitable questions and wording, as described below:

2. Create varied questions

A test should generally include different exercises. Different question types provide variety and ensure that tests don’t get too monotonous for participants.

Structure of an exercise:

A title for each exercise is a good idea, but it is not absolutely necessary. If you want to assign titles, it makes sense to use the headings of the learning steps.
The introductory text prepares the learners and defines the scope of the exercise.
The exercise text explains the issue.
The question text contains the actual exercise. It is important for the instructions to be clear. Learners should also fully understand what exactly you expect from them and, for essay questions in particular, the type and scope of the required solution.

Which questions should you use for which learning objectives?

Reproduction (knowledge): Have your participants quote, enumerate, name, define or reproduce something. Short, free text exercises are suitable for this.
Reorganization (understanding): This is about arranging, describing, interpreting, classifying, explaining, comparing and assigning. A suitable question type would be a true/false question.
Transfer (application): This is about applying, evaluating, creating, checking and converting. A large hotspot or a multiple choice question would be useful here.
Transfer (analysis): This involves analyzing, selecting, connecting and checking. Ask a single choice question (just one correct answer) or a hotspot image with a narrow scope.
Problem-solving (synthesis): Participants must justify, design, reason, plan or solve a problem. It would be a good idea to use an essay question here.
Problem-solving (evaluation): Estimate, assess, discuss, evaluate, comment. Have participants post comments that other participants can react to.

Tip: Write down your ideas for test questions while you are creating the course. This allows you to create questions quickly while ensuring that they match the course content. Also reading specialist literature or newspapers can also inspire new ideas. In this way, you can ensure that your test questions are relevant to contemporary topics and events.

3. Evaluation – can this be automated?

Most question types provided within the LMS allow for automatic evaluation: single and multiple choice, true or false questions, but also hotspot exercises can be automatically evaluated using pre-defined solutions. These are quantitative question types.

It is different for free text tasks, which require manual evaluation. A trainer must evaluate the submitted tasks individually and only then can the participant receive a constructive evaluation. This is a qualitative question type.

It is wrong to think that automatically evaluated exercises are not useful for promoting understanding or even the application of knowledge and understanding to tasks. The scope of the exercise is the crucial aspect. With a little practice and experience, you can also create a practical exercise as a hotspot question.