Abstract:

The aim of this paper is to show the design and implementation of a web-based platform based on “paired comparison” that can be used both by professors, as tutor assessment, or by students, as peer assessment. The main reasons that motivates this development is to find a tool that enhances the validity and reliability of qualifications obtained, increases the involvement and engagement of students, improves student feedback, as well as improves assessor efficiency in terms of time and resources invested.

There are many different materials to be compared and assessed by using this tool, as for instance images, posters, models, short reports, long reports, oral presentations, etc…This tool has been tested at the moment for short reports in groups of around 20/25 students. However, it has been designed to be used in various settings.

It is important to distinguish between paired comparison and peer assessment.
On one hand, Peer assessment, is a process whereby students or their peers grade assignments or any type of materials developed by them based on a teacher’s crieria. This practice allows teachers to save time and moreover, improves students' understanding of course materials as well as improve their metacognitive skills.Peer assessment also allows teachers to help students understand the mistakes that they have made. This type of assessment will improve student’s subsequent work, thus, through peer-assessment students are able to see mistakes in their thinking and can correct any problems in future assignments. Rubrics are often used in conjunction with Peer-Assessment.

On the other hand, Paired comparison or pairwise comparison, refers to any process of comparing entities in pairs to judge which of each entity is preferred, or has a greater amount of some quantitative property, or whether or not the two entities are identical. This method can be used to order items along a dimension such as preference or importance using an interval-type scale. It is mainly used in the scientific study of preferences, attitudes, voting systems, social choice, public choice, and multiagent AI systems. Rubrics are not necessary with Paired-assessment, only a full description of the relevant criteria to be used in the assessment.

Even so, we present an algorithm to be used by a web-based platform that may allow students to compare pairs of assignments against a set of criteria to determine which one is better. This tool simplifies and speeds up the assessment process, while increases objectivity and reliability, above all when working (and hence, assessing) with fragmented time, that is short periods of time along different days to accomplish the assessment assignment.

Keywords:

Peer assessment, paired comparison, pairwise comparison, assessment software.