Abstract:

Like most universities, the Institute of Informatics at the University of Oslo experiences violations of the honor code for home exams requiring students to solve problems on their own. While some students simply hand in an exact or obfuscated copy of a solution made by a fellow student with or without her consent, others find solutions on the internet or even buy solutions through dedicated web shops. Estimates indicate that at some point up to 15-20% of all solutions handed in were either subject to or the result of an attempt of plagiarism.

With typically 400-800 students, manual checking of all solutions against all others requires somewhere between 80000 and 320000 comparisons, which is far beyond available resources. At any rate, solutions are distributed among 10-20 teachers, each evaluating only the solutions handed in by students belonging to her own class. Potential cheaters being fully aware of this, fraud cases frequently involve pairs of students assigned to different classes.

This presentation is a further development of work and results reported in Maus and Lingjærde (2008), the focus now being on the more lasting effects of automatic plagiarism detection. A distributed system called Joly is used by students to submit their home exams in Java programming. All submissions from this year and previous years are stored in a database. A new entry is compared with all other submissions in the database, and highly similar submissions are reported through e-mail to a teacher for manual comparison. If the suspicion is confirmed, a report is made to the university disciplinary committee.

Recent results indicate a positive effect of the system on the course. We seek answers to questions such as: Have the students adapted their behaviour to the plagiarism detection system? Do they find ways around the system? What are the strains on the system when the number of reported cases of plagiarism suddenly doubles? How to convince lawyers and laymen that a submitted answer involves plagiarism, even when various changes have been made to the program so it textually looks quite different?

The system is constructed to detect most or all of the commonly used methods for obfuscation of a program to make it look different but execute in exactly the same way as the original (examples include changing the names of variables, changing or adding comments, etc.) We briefly present three alternative comparison algorithms. In the comparison of these algorithms, one stands out because its results are more easily explained to, and understood by, the university administration.

Students were informed of the purpose of the Joly system. We observed the effects this had on the 15-20% of the students who previously would have cheated, by comparing statistics from 2004, the last year without Joly, with statistics from 2006, the first year with a fully functional system. The percentage of students who fail the exam dropped from 25% to 10%, while the number of students who take the exam dropped by 5%. Interestingly, as a result of the decreased failure rate at the exam, 5% more of the students who start on this course, pass the exam.

References:
Maus, A and Lingjærde, OC. The application of a novel plagiarism detection system to an introductory course in programming: Lessons learned. INTED 2008.

Keywords:

plagiarism, cheating, programming, java, exam, comparison.