Abstract:

The multiple- and single-choice tests, despite their limitations, are a highly popular way of assessing students. In the usual way of administering such tests, students are discouraged from picking answers that they are not sure of, either by giving a penalty of negative points for wrong picks or by awarding no points for questions answers to which include wrong picks while giving partial points for unanswered questions. The intention of such practices is to make the students determine the answer rather than guess it, yet on the other hand, this obviously reduces the information feedback provided by the test to the teacher. For instance, if no answer was given, it does not necessarily mean that a student did not really know what to pick, it may mean he/she knew it, but was not confident and did not risk making a wrong pick.

In this paper, we report results of a three-year-long experiment, in which students doing final tests in computer science courses were encouraged to pick as many answers as they consider possibly right, to receive a fair share of points if the picks included the correct answer (e.g. in a single-choice test with four candidate answers, a student who chose two answers including the right one would receive 50% of the points given for picking only the correct answer). We find the additional information gained in this way to be valuable in two ways, to help identify:
(1) students who have considerable knowledge in the scope of the test yet also have low confidence in their knowledge;
(2) test questions or answers which are unintentionally stated in an indistinct way.

In some of the tests, we extended the list of questions with a self-assessment one, by asking students to pick the easiest questions (i.e. those they are confident they picked the correct answer), or, more explicitly, by asking students to declare the number of points they expect to receive. This serves as a secondary source of information on students’ confidence which is especially useful in the case of students who are reluctant to pick answers that they consider only probably correct.

Keywords:

Multiple-choice tests, single-choice tests, student confidence.