Abstract:

The use of rubrics has become widespread in a large part of the teaching community due to their effectiveness as a tool to support the assessment of learning based on various criteria, set by the teacher. Due to their simplicity of application, the rater chooses the level of fulfillment of each criterion according to a numerical scale, often accompanied by detailed descriptions of the different levels of performance. In the field of industrial design it is also possible to find the use of rubrics to assess different competences, but the published works on the use of rubrics for the assessment and self-assessment of scale models and drawings, and the study of the relevance of the use of detailed performance-level descriptions in this field are not as numerous as those existing in other fields.

In order to explore this topic, this paper presents a study comparing two different ways of applying rubrics in two subjects of the Degree in Industrial Design and Product Development Engineering at the Universitat Jaume I, both taught by the same teacher: 'Modelmaking workshop' and 'Design for the environment and habitat'. In the first case, students evaluate their own work individually, specifically scale models. A rubric has been designed to provide detailed performance-level descriptions. In the second case, students assess as a group the work done by another group, consisting of the drafting of a brief for the design of a new domestic product and the graphic representation of this product. In this case, unlike the previous case, the rubrics designed only describe the two extremes of each assessment range, but do not indicate which criteria must be fulfilled for each of the intermediate assessments. The assessment given by the students in each subject is compared with the assessment given by the teacher using the same rubrics, in order to check the rater reliability in each case. For our study, the threshold for significant differences between student and teacher ratings was set at 10%. In other words, we consider that both ratings differ too much when they exceed 10% of the rating scale.

The comparison between student and teacher ratings using the rubric in the first subject showed significant differences in only 11.9% of the ratings, thus demonstrating high rater reliability. On the other hand, the comparison between student and teacher ratings using the rubric in the second subject showed significant differences in 44.4% of the ratings.

As a result, this paper presents a rubric for evaluating scale models capable of achieving high rater reliability, demonstrating the importance of the use of detailed performance-level descriptions. The results show that most of the time the use of these rubrics can help students to give a more objective assessment of a practical work, being possible to assign a rating very similar to the one that would be assigned by the teacher using the same rubric. However, failure to provide adequate performance-level descriptions in a rubric may contribute to poorer rater reliability. Therefore, when designing a rubric to assess physical models or graphical representations of the product, it would be advisable to take these indications into account in order to achieve more objective assessments that are closer to those of the teacher.

Keywords:

Industrial design, rubrics, self-assessment, peer assessment.