Abstract:

Introduction:
Most multiple-choice tests comprise questions with four options per item. However, a number of academic teachers believe that a larger number of options per question shall increase the scope of variability of test results. An increase in discrimination capability is particularly important with reference to selective examinations. In 2011 and 2013, nursing entrance test at Medical University of Warsaw (MUW) comprised 5-option item tests, which was an exception from 4-option item tests used in 2009-2010 and 2012.

Aim of study:
Assessment of the impact of change of the number of options in multiple-choice questions on the quality of nursing entrance exams for an MA programme at MUW between 2009-2013.

Materials and Methods:
A total of 250 multiple-choice exam questions, including 150 4-option items (2009-2010 and 2012) and 100 5-option items (2011 and 2013). In order to compare the quality of particular exams, the level of easiness, substitute differentiation power, and Pearson's linear correlation coefficient were established for each pool of questions. The comparison comprised the scope of variability of the results (coefficient of variation, scope of results, and quartile range) as well as the average easiness and capacity of differentiating particular questions in consecutive versions of the exam. One-way analysis of variance ANOVA and post-hoc RIR Tukey honestly significant difference test were used.

Results:
In 2011 and 2013, when the 5-option item tests were introduced, the difficulty of the exam expressed as the mean score amounted to 24.3 and 25.7 points, respectively. These values are comparable to the results achieved in 2010 (25.6), but they are clearly different from those obtained in 2009 (30.2) and 2012 (31.5). Similar differences were observed in comparison of coefficients of variation that were similar in 2010, 2011, and 2013 (17.1, 17.9 and 18.4%, respectively) and significantly different from those obtained in 2009 and 2012 (14.3 and 14.1%, respectively). Moreover, a greater symmetry (skewness ≈ 0) in frequency distribution of test scores was observed in the case of 5-option item tests compared to 4-option item tests. The reliability of the exam was variable, Cronbach's α coefficient ranged between 0.429 and 0.559. No statistically significant differences were found in discrimination capability of the exams performed in the form of 4- or 5-option item tests (ANOVA test, P > 0.05). It was also demonstrated that the 2011 exam (5 options) was significantly more difficult than that of 2012 (4 options) (ANOVA test (P = 0.0025) and post-hoc RIR Tukey honestly significant difference test (P < 0.01)).

Conclusions:
The introduction of an additional option item to the test questions did not significantly improve the qualitative parameters of the nursing entrance exams at MUW. Significant increase in selective capacity of the exam and reliability of assessment was not observed. It is recommended to use 4-option item tests and to develop a good test plan for future editions of the exam.

Keywords:

Educational measurement, nursing admission examination, nursing entrance test, number of choices per item, item discrimination.