Abstract:

This paper deals with scoring schemes for Multiple-Choice tests (MC tests) and the statistical effects they lead to. The most important criteria for the evaluation (or selection and parametrization) of a certain scoring scheme is: "How many students will pass the test, when they select their answers only randomly?" An inappropriate choice of the scoring scheme (e.g. based on insufficient knowledge about the statistical effects) made by a single teacher or a board of education may even cause problems in terms of legality. Therefore the paper compares different schemes and presents a browser based tool which can be used to calculate the statistic effects for concrete MC tests.

In general MC tests can be used to test learning matters learned and/or understood by students quite easily. In some fields they are quasi the only tool for computer based tests. Caused by the continuously proceeding digitalization of teaching processes it is feasible that the importance of MC tests as well as the importance of their adequate usage will increase. A further aspect is (especially in Germany) a set of legal frame conditions, which are partly reflected in examination regulations. A special challenge is the fairness of scoring given to solutions, especially considering the possibility of pure guessing, half knowledge or problems in understanding the formulation of the questions.

Scoring schemes can be found/combined at different levels of abstraction, e.g. the scoring for a single MC solution (1 task, 1 person), or the set of MC solutions (n tasks, 1 person) or all results (n tasks, m persons).

The literature describes mostly:
- concrete MC tests for several fields (useful for students as well as teachers),
- design rules for MC tests as assistance for teachers (those rules are definitely very important , because the quality of the questions and response items is significant for a fair scoring).

In contrast to this the scoring schemes are described much less often and if they are described then usually only in a limited way, e.g. only a certain scheme or as comparison of two or a few selected schemes.

Here we want to give a wider overview and comparison of scoring schemes, which can be used by teachers or examination boards as substantiated base for discussions and decisions concerning the selection of scoring schemes.

Beside diagrams, tables and textual explanations an interactive tool is presented, which can be used to analyze statistical effects of the parameterization of certain scoring schemes. The consequences of certain scoring schemes shall be made more comprehensible by interactive graphical representations as well as concrete statistical conclusions (e.g. the part of students which will pass the test even when they only guess). The interactivity of the tool is important to understand the influence and sensitivity of the parameters and invites to explore the space of scoring schemes and their parameters in more detail. This allows to find schemes with (e.g. statistical) effects which fit best to the own understanding of fair scoring.

Keywords:

Comparison of scoring schemes for multiple choice tests, suitable scoring, practical implications of scoring schemes.