Abstract:

Performance-based assessment is one of the methods used for measuring the generic skills of students. For this research, the performances of the pre-university students were assessed by the students’ own lecturers using a set of generic skills instrument. The lecturers gave ratings to the generic skills that were exhibited by the students during their activities in the classrooms. This study aims at examining the behaviours of the raters in the process of assessing the students’ generic skills. The instrument used consists of nine constructs of generic skills, namely teamwork, communication, leadership, critical thinking and problem solving, life-long learning and information management, entrepreneurship, ethical and professional morality, social responsibilities and management. This study used Many-Facet Rasch Measurement (MFRM) model and the data analysis was done using FACETS 3.66.1. There were 23 lecturers involved in assessing 271 pre-university students. The findings of the research show that the percentage of agreement amongst the raters is between 37.3% - 43.1% and the value of Rasch-Cohen Kappa is between 0.92 – 1.00. The raters act as trained assessors and work independently during the assessment process. The results of data analysis show that there exist differences in severity amongst the raters when assessing the generic skills of the students, and also inconsistencies among few raters in their ratings. The bias analysis shows that there is no specific pattern with regard to bias interaction between the raters and the items. However, the male raters show more bias interaction towards the items as compared to the female raters. The study on the behaviours of the raters is important because the findings can help the raters to carry out fair and appropriate assessment of the students’ generic skills.

Keywords:

Performance-based assessment, rater behaviours, generic skills, Many-Facet Rasch Measurement, pre-university students.