Abstract:

This study aims to understand if the use of a new assessment method in medical instruction fulfils and meets the initial expectations of the students of an Otorhinolaryngology course of the Integrated Masters Program in Medicine of the Faculty of Medicine of the University of Lisbon.

A multimedia-format exam was implemented as an assessment method since 2014/15, and the need for analysing its benefits of its characteristics.

This study aims not only to analyse the degree of student satisfaction, but also to identify the advantages and disadvantages of this assessment method, when compared to others, such as paper-and pencil Multiple-Choice Tests, Oral Examination with the presence of patients, and Objective Structured Clinical Examination (OSCE).

The multimedia exam lasts 75 minutes and is structured upon 30 multiple-choice format questions, containing endoscopic images, clinical histories, complementary diagnostic exams, videos of surgical interventions and examples of medical errors.

Responses are registered in a form and graded afterwards by an optical scan reader.
The present study took a quantitative approach for collecting and analysing data to infer the degree of student satisfaction about this assessment method considering that, since its implementation, a satisfaction survey has been applied to the students after the implementation of the exam.
The survey consists of 15 questions distributed in 3 parts:
i) Sociodemographic characterization;
ii) 5-point Likert scale questions about design, adequacy and the technological infrastructure supporting the exam;
iii) open and closed questions about the overall assessment of the exam as well as students’ initial expectations.

The results of the 593 responses of students who attended this course, in the last 3 years, indicated that 50% are satisfied with the new assessment model, and from those, 11.6% are very satisfied. Regarding initial expectations, about 40% of the students indicated that this type of assessment exceeded their expectations.

With this study, we also learned that the degree of satisfaction is more strongly correlated with the design of the exam items, namely the presentation of videos and images, and the opportunity of strengthening the knowledge about the course subjects.

The content analysis performed to the responses to the open questions also revealed that the most positive aspect identified in this type of exam is the promotion of the connection between the theoretical and practical contents of the course. However, students pointed out, as an improvement to be made, to have immediate access to corrective feedback after the exam is finished.

This study aims to provide a deeper knowledge about students’ acceptance of different assessment methods and its usefulness for the pedagogical practice in medical instruction.

Keywords:

Assessment, medical education, multimedia.