EVALUATION OF PROFESSIONAL COMPETENCES: CAN WE CORRELATE THE FINAL RATING OF A MULTIPLE CHOICE TRADITIONAL TEST WIT THE SPECIFIC COMPETENCES EVALUATION?

A. Pino Vázquez1, H. González García1, M.B. Coco Martín2, R. Cuadrado Asensio2, A. Mayo Iscar3, E. Urbaneja Rodríguez1, A. López Miguel2, C. Villa Francisco1, M.J. Maldonado López2, C. Medina Pérez4

1University of Valladolid (SPAIN)
2IOBA/University of Valladolid (SPAIN)
3Statistics and operational research/University of Valladolid (SPAIN)
4IBIOMED/University of Leon (SPAIN)
Introduction:
The accumulation of knowledge, likewise the acquisition of skills, are continuous processes and indefinite in time. Health education requires a corporation not only of knowledge, but also skills, attitudes and values needed to become a future prepared professional. The development of skills in the area of health sciences are composed of a complex educational process where suitable strategies for a comprehensive assessment of the skills to be acquired. However, traditionally, this assessment is based on the measurement of knowledge through the application of various types of tests: multiple choice questions and short questions, that not always are idoneous having into account the criteria of objectivity, validity and reliability . This situation has led to the urgent need to develop new educational and evaluation methods, to assess clinical competencies.

Objective Structured Clinical Examination (OSCE) is a test designed to assess clinical skills. The value to change this assessment is to reflect the importance of learning that takes place and that is necessary and not just what students know for the final evaluation. The power of this test lies in the mixture of assessment methods: it is able to explore three of the four levels of the Miller pyramid: knowledge, know how and show how. The OSCE consists of a circuit of stations that allow the teacher to observe the student interaction with patients, to certify their clinical skills, thinking skills, problem solving skills, integrating diagnosis and communication and interpersonal skills, a comprehensive assessment which requires any professional.
Purpose: To compare the results obtained by students in 6th of Medicine with different assessment methods: Traditional: test and short questions, and OSCE.

Material and Methods:
We analyzed the results obtained by 139 students in the 6th year of the Faculty of Medicine of Valladolid in the subject of Pediatrics, after a traditional exam in the course of Pediatrics (with classical methodology of teaching and assessment) and after evaluation with OSCE in the subject "Revolving Practical Pediatrics". The latter was used for the first time as a program of educational innovation (rotary structured clinical, practical content in a virtual classroom, seminars, simulation and evaluation of skills by OSCE). We calculated the coefficient of intraclass correlation coefficient (ICC) to measure the agreement in the results of the two tests.

Results:
The ICC for agreement between the two tests was 0.24. Even the most generous of the analysis in these assessments, Pearson correlation coefficient was at a very low level, 0.34. This level of agreement could be considered irrelevant when we are comparing the values in two tests that should (theoretically) give the same results.

Conclusions:
We did not find a strong relationship between the final marks obtained on the traditional teaching and the final marks of the competence assessment following a teaching innovation program based on the acquisition of skills.

Therefore the traditional examination skills assessment after a teaching innovation program based on the acquisition of skills are different tests whose results appear to be different.