Abstract:

The evaluation of academic teachers' work by students is an important dimension of the quality of education provided by universities. Teachers’ evaluations are used as key materials for academic decisions about hiring, promotion, tenure, or salary increases. Nevertheless, they have been criticized for being biased and reinforcing grading leniency practices on the part of teachers (giving high grades to get high evaluation scores). The aim of the present study was to investigate whether there is any indication of grading leniency in the context of Distance Education offered by the Hellenic Open University (HOU), after many faculty members’ requests on the issue. More specifically, the research questions were the following:
(1) Is there a positive correlation between student grades and tutor evaluation scores?
(2) If so, how can this relationship be explained?

Data were collected over three academic years (from 2020-2021 through 2022-2023) on students’ written assignments grades and their tutor evaluation scores submitted to an online questionnaire. The evaluation was anonymous; however, students had the option in the questionnaire to consent to the removal of their anonymity and to the usage of data such as their tutor evaluation score, grades, gender, etc., exclusively for HOU's research (N of consents = 20533). Details of HOU's privacy policy were also presented in the questionnaire. The investigation of the possible correlation between the written assignment grade and tutor evaluation score was carried out considering the fact that during the period where the tutor evaluation questionnaire is open, students know whether or not they are eligible to participate in the final examinations. More specifically, the following three cases were investigated:
a) Students evaluate tutors when they already know they are disqualified from the final module examination;
b) Students evaluate tutors when they know they have not qualified for the final module examination but can do so by completing one more written assignment; and
c) Students evaluate tutors when they know they have qualified for the final module examination based on their performance in three written assignments so far. It is also mentioned that tutor evaluation is completed before the start of the final examinations.

Spearman's rho tests were conducted to detect possible correlations between the student's written assignment grade and the tutor's evaluation score. Findings showed that in cases a and b, no statistically significant correlations were found during the three years. This fact rather opposes the bias hypothesis according to which students with low grades tend to punish their tutors with low evaluation ratings. In case c, weak statistical correlations were found during the three years. Even though, prima facie, this finding could be interpreted either as an indicator of bias (students with good grades tend to give higher tutor ratings) or grading leniency (tutors give high grades to get high evaluation scores), our interpretation tends to support the so-called validity hypothesis: students who learn more get higher grades and therefore give higher tutor ratings (Marsh,1987). Nevertheless, it is highly recommended that whenever a positive correlation between student grades and tutor evaluation score is detected, a cautious investigation of the context is needed, taking into account several factors that may influence the tutor’s score (e.g., module difficulty, module level, etc.).

Keywords:

grading leniency, distance education, tutor evaluation