Abstract:

After one decade using rubrics to assess Problem-Based Learning (PBL), the management team of the Faculty of Medicine from the University of Girona noticed that the evaluation system showed some dysfunctions. On the one hand, different competences were assessed in the same rubric dimension. On the other hand, the collection of the marks was manual and some teachers, due to lack of time in the classroom, filled in the rubric at the end of the session. This practice led to a loss of objective evaluation criteria and, therefore, to a less detailed assessment of each student.

Based on the above, the objectives of this study were:
(a) to design a rubric for assessing students during PBL sessions, which should be more practical than the one used during the last academic years in the Medical degree of the University of Girona;
(b) to develop an application to enable the online administration of the rubric, thus facilitating the collection of grades, and the assessment and information of students.

The methodology used to develop the rubric was as follows. An action protocol was drafted, and the working group as well as the guidelines to be followed were defined. A committee was appointed. It was composed of 6-8 professors of different subjects, belonging to basic modules, medical, surgical, primary and transversal areas, as well as the director of the EMU, the degree coordinator, and 3 representatives of the University of Girona’s Network of Educational Innovation in Evaluation. This committee met once a month for six months to draw up the new rubric. Moreover, it was organized into 4 subcommittees, with 2-3 members each one, to modify the items in each of the dimension of the rubric.

The committee analyzed the pre-existing rubric, in particular, the suitability of the number and type of questions in each dimension, as well as the rating scale. The assessment included 5 questions for each competence:
(i) learning skills;
(ii) communication;
(iii) responsibility; and
(iv) interpersonal relationships, with scores from 0 (never) to 5 (very often).

Then, it was proposed to reduce the number of items in the rubric, to assess only one aspect in each item and stablish 5 scoring options for each item. In addition, the commission selected those items that would be graded daily, as well as those that would be assessed in each PBL, at the middle and end of the module, or only at the end of the module. Furthermore, the weight of each question was established according to characteristics of each PBL.

The first version of the rubric was sent to a sample of teachers and students to get their feedback, and it was then modified according to their comments. The final version of the rubric incorporated specific cases that had not been considered in the original rubric, such as:
(i) the anomalous situations that occur in cases of delays, absences or impossibility of carrying out the PBL session, or
(ii) when to provide students with the partial mark of the PBL so that they could have an indicative report on their progression.

Once the modification process of the rubric was completed, the rubric was computerized by developing an application that allows the online administration of the rubric, or even its offline treatment, and the periodic generation of reports (monitoring and evaluation) for students.

Keywords:

Rubric, medicine, competence, assessment, university.