Abstract:

Most computerized adaptive testing (CAT) algorithms select items that each test-taker should be able to answer correctly at 50% to 60 % chance level because it can reduce the number of item and shorten the test length without deteriorating measurement precision. However, experience of taking such tests may cause backwash effects, such as loss of self-confidence, loss of self-efficacy, and loss of motivation (Kimura & Nagaoka,2011).

The previous questionnaire research immediately after CAT for Japanese university students revealed that test takers tend to prefer the longer test with easier items than the shorter test with more difficult items (Kimura, 2016; Kimura & Koyama, 2016).

After developing and implementing of a Moodle-based CAT system in order to manipulate target probabilities (TP) of answering correct, Kimura (2017) enhanced the CAT system to implement a multistage CAT system on Moodle.

The purpose of this paper is to investigate if test takers actually tend to select the longer test with easier items than the shorter test with more difficult items. A two-stage CAT was administered to 78 Japanese university students. A half of them started the longer test with easier items for the first stage, and the other half started the shorter test with more difficult items for the first stage.

After the first stage, they were asked to choose:
1) to continue the test with the same TP and number of item for their second stage, or
2) to continue the test with different TP and number of item for the second stage.

Theoretically, all test taker should end with the same measurement precision. The result will be presented at the conference and some implication of the multistage CAT system will be discussed.

Keywords:

Multistage testing, computer adaptive testing, target probability of answering correct, self-confidence, self-efficacy, motivation.