Abstract:

Introduction:
Multiple choice exams have an increasing role in the university environment. One of its drawbacks is the greater ease for students to “cheat”, due to the simplicity of the answers. Cheating techniques are multiple. Because no writing is necessary in the answers, communication between students is greatly simplified, making it possible to create specific codes that are difficult to detect. However, when two students copy, the agreement between the results of both exams can help detect potential “cheaters” through a posteriori evaluation. The correct answers in both students are not evaluable, since we assume the presumption of innocence in both students. However, the wrong answers in both students should follow a random pattern, as long as the tests were taken independently. Therefore, the objective of this work is to evaluate the probability that two students have copied from each other, through the statistical analysis of the agreement between the erroneous answers of both exams. The group of students that were evaluated were taken Clinical Pharmacy, in the School of Pharmacy of the Miguel Hernández University of Elche.

Methods:
Cohen's Kappa coefficient is commonly used to determine the agreement between different observers on a categorical decision. In this case, the observers would be the students, and the decision would be the wrong answer on the exam. The higher the Kappa value (between 0 and 1), the greater the agreement between the observers. This procedure will be repeated for all possible combinations of students.

Results:
To carry out all the iterative processes, a code has been developed in R™. This code allows, in a simple way, to carry out all the operations and combinations of students. The evaluation has been carried out on a total of 90 students, and the calculated Cohen’s Kappa Coefficient was 0.48. As a main limitation, highlight the difficulty in determining a Kappa cut off value from which to decide that the students have cheated. There are various authors, who propose kappa limit values of 0.75, while others propose 0.8. Even so, the results of the evaluated group did not detect any high agreement between the requests.

References:
[1] Cohen, J. A. (1960). “Coefficient of Agreement for Nominal Scales”. Educational and Psychological Measurements. (1)
[2] Landis, J.R.; Koch, G.G. (1977). "The measurement of observer agreement for categorical data". Biometrics. 33 (1): 159–174. doi:10.2307/2529310. JSTOR 2529310. PMID 843571.
[3] Fleiss, J.L. (1981). Statistical methods for rates and proportions (2nd ed.). New York: John Wiley. ISBN 0-471-26370-2.

Keywords:

Kappa's index, cheating, pharmacy.