Abstract:

Peer review, the process of evaluating peers' work based on predefined criteria, is a common practice in massive online open courses, as well as in higher education today. Arguably, its main advantages are the possibility of achieving scalability in assessment, assessment of complex assignments, and learning that occurs due to critically evaluating peers’ work. Still, using peer review in higher education is characterized by both technical and organizational challenges, especially if it is used for summative assessments. In this paper, the design and implementation of a minimalist yet efficient peer review system are described. The system is based on Moodle and Microsoft Office tools and, unlike most current peer review solutions, it offers great flexibility in assigning and determining the quality of submitted peer reviews. The system was implemented to support summative peer review practices in a Communication skills course for engineering undergraduates. Assignments that would be difficult to grade automatically, for example, slideshows, resumes, and video projects, were graded using peer review. Due to its flexibility, the developed system can be used in other domains, especially when the validity and reliability of submitted peer reviews are important. The experiences and lessons learned from using the system on a scale of 700 students yearly for the last four academic years are discussed in the paper.

Keywords:

Peer review, summative assessment, scalability, engineering education, communication skills.