Abstract:

University students sometimes feel that they obtain marks not reflecting their efforts. Classical assessments methods based on exams seem to be unfair both due to possible biases on the professor criteria or because of the particular circumstances which influence students when they are doing the exam (nervousness, environmental conditions in the examination room, stress, etc.). Farshid (2008) has recently informed about the benefits of using 360-degree assessment in order to make the learning evaluation process more valid and reliable. We have extended the methodology introduced by Farshid to evaluate research projects works at the university.

We asked a group of psychology students to carry a brief research out as the practical part of the psychometric subject. They had to upload their reports, which they needed to write according the APA editorial style (American Psychological Association, 2001), in a WebCT course using the virtual group tool. After a first private phase (a week long) reports were made public so that the rest of classmates could read every report. Students had a week to read all the reports. Then, every group made a public presentation of their researches. After the presentation the work was self-evaluated by authors and evaluated by classmates and the professor.

An evaluation form was developed in order to collect the judgements from the authors, classmates and professor. The questionnaire contained 14 items designed to evaluate four dimensions: APA style, social utility of the results from the research, presentation quality, and subjective quality. Every dimension had its own importance factor as a percentage of influence on the final mark. All the items were Likert-type questions with 11 alternatives ranging from “Excellent” to “Very poor”. The final mark given to a report was the arithmetic average of the averaged classmates’ judgements; authors’ averaged judgements and the professor’s judgement. Additionally, an opinion poll was used to know the opinion of the students about the assessment methodology.

We found several expected correlations between the classmates’, authors’ and professor judgements which support the validity of the assessment procedure. But the main finding is in the data from the satisfaction questionnaire. Instead of being highly satisfied with the procedure because it is fairer, many students reported to be dissatisfied with the system. On top of that, some students abused their possibility to rate their own work and they overestimate their marks.

Although self-assessment has shown to be a good predictor of actual marks (López, Ruiz-Ruano and García, 2008) we have observed an incomprehensible overestimation in the authors’ and classmates’ judgements. This might have happened because of the informal groups in the class and must be deeply studied in order to modulate this distortion of the assessment process.

References:

- American Psychological Association. (2001). Publication manual of the American Psychological Association (5 ed.).

- Farshid, M. (2008, November). 360-degree feedback in writing classroom applying a holistic model for evaluating students’ writing assignments, from a qualitative description to a quantitative rate. ICERI 2008. Madrid.

- López, J., Ruiz-Ruano, A. M., y García, J. (2008, November). Relationship between self-assessment and marks in higher education: Linear, logistic and bayesian analysis. ICERI 2008. Madrid.

Keywords:

360-degree assessment, self-assessment, practical research projects, higher.