Abstract:

Peer Assessment (PA) in higher education (HE), i.e., asking students to assess each other to inform teacher’s own assessment of student learning progress in and contribution to groupwork, is neither trivial nor is it the king’s road to accurate, unbiased educational assessment. However, granted a deep understanding by both teacher and students of its strengths and weaknesses, opportunities and threats, PA can be an adequate method in case classical assessment is impossible or may be grossly misleading, inefficient, or incomplete.
We will propose a new assessment framework for a combined Group-Peer Assessment (GPA). Our goal is to promote awareness and appreciation among (preservice) teachers of the unique features and ubiquitous advantages of GPA when conducted correctly and properly. The paper will mainly be based on our experiences using a paper-based approach, and on the development of interactive software tools to support teachers and students when conducting PA. Our assessment framework has a much-neglected measurement & computation orientation with a focus on validity, flexibility, and utility.
We will describe a unified family of peer rating, student scoring and group grading models in HE. This still growing family of models is called system Q and consists of:
(1) a collection of bounded scales (e.g., the percentage scale) with adjustable neutral elements and equipped with quasi-arithmetic operations of addition, subtraction, inverse, and scalar multiplication of ratings, scores, or grades.
(2) a collection of formulae for peer and student ratings and for student contributions and scores of three types: linear, rational, and exponential, corresponding to three types of constrained percentage scales.
(3) a collection of formulae for the quasi-arithmetic means of student ratings, contributions, and scores, that correctly represent the overall group work and its outcomes and guarantee that the average of student scores equals the group score (Split-Join-Invariance).
The backbone of Q is the concept of bounded scale. The traditional approaches to PA have uncritically adopted a statistical view of measurement and scaling. This has led to some paradoxical results when it comes to merging the peer ratings of students (focusing on group process quality) with the teacher-assigned scores of group work (focusing on group outcomes and products). It is well-known that adding and multiplying multiple ratings or scores without taking the boundedness of the scales into account may lead to results outside the accepted domain of ratings or scores (e.g., percentages). Suggested repair mechanisms (e.g., using ad hoc parameters, applying min and max operators, or simply capping) don’t do justice to the original collected data nor do they satisfy the teachers and students.
These problems can be solved with a quasi-arithmetic calculus that redefines addition and multiplication of ratings or scores so that they make sense on bounded scales: one can add, subtract, and invert ratings, scores, and grades without ever leaving the adopted bounded scale. Moreover, the quasi-arithmetic mean of ratings, scores and grades can be defined and applied in place of the arithmetic mean which is not well-suited for bounded scales.
Since ICERI2019, we have generalized, streamlined, and improved our assessment framework in several ways. However, the most significant innovation is a new scoring model with remarkable properties: the double-exponential scoring rule.

Keywords:

Group assessment, location, spread, peer assessment, student rating, contribution, score, split-join-invariance, symmetry, quasi-arithmetic mean.