Abstract:

This study reports on an action research project of assessment reform within a national academy of music. The overall aim was to create a more rigorous grading strategy by the introduction of explicit criteria. The teachers volunteered to participate in a project designed according to principles associated with “action research”. This took the form of a reflective process of individuals working with others to propose a new course of action in assessment and grading. The primary interest was in particular in the process of agreeing on a set of explicit criteria that could be used for grading purposes.

Key research questions were:
• What motivated the introduction of analytical assessment?
• If any, which were the issues in agreeing on a set of criteria?
• What is the potential use of the criteria?

As part of the project, the concept of a ‘rubric’ was introduced. The purpose of a rubric is to provide a matrix with the intention of making judgments of students work more transparent and systematic. It consists of two dimensions known as ‘criteria’ and ‘standards’. Criteria refer to properties or characteristics to be assessed, while standards refer to performance levels. Issues of implementation varied in scope from details to concerns contesting the idea of analytical assessment in the discipline. Some argued that artistic performances are of a different kind compared to science, and that this should be reflected in the selection of assessment instruments. In the former case, examiners may have to compare a bass with a soprano, or a lyrical voice with a dramatic voice, which is not a straightforward thing to do.

This study reports on the process of moving from a culture of tacit and holistic assessment of vocal performances to an analytical approach. An expressed belief underpinning the project was that this would improve the quality of judgments both in individual cases and across the range of performances. This turned out to be harder than expected due to the complexity of judging artistic vocal performances, and especially by agreeing on a rubric that could be consistently managed in authentic settings. Of the two dimensions - criteria and standards – the former occupied most of the time and yielded the most visible outcome. The issue of standards was barely touched upon and no descriptors were made explicit. This demonstrated a limitation of the rubric and prompted the need for additional resources to be able to discern what really makes one performance better than the other. One lesson learned was that the application of a rubric can never substitute experience and tacit knowledge, but represents a valuable tool for learning and in making judgments more consistent and transparent.

In this session the rubric will be presented in full, and its rationale, benefits and limitations will be discussed along with issues of change.

Keywords:

Analytical assessment, rubric, criteria, standards.