Abstract:

Since 2011, in the subject matter of Psychometrics of the Faculty of Psychology of Universitat Autònoma de Barcelona (UAB) students have developed a project based on the Spanish review system for evaluating test quality (http://www.cop.es, evaluación de tests). Formulated as a Problem-based learning (PBL) project, the students have to evaluate in groups of 3 to 6 persons the psychometric properties of a logopedic, educative or psychological test previously selected by the professors among those available at the test library of our Faculty (http://psicovirtual.uab.cat/piap). Every year about 60 groups evaluate 10 tests in average. Developing this project helps students to work on the content of the subject matter from a real case that they could face as psychologists and to achieve the intended competences. To increase motivation among students and to provide of test reviews to the university, the best review of each test with sufficient quality is published in the test library of our university (faculty beneficiary). Recently (2019-20), this project has changed the beneficiaries, https://www.eoslhe.eu/test-reviewing-as-service-learning). Following the same review system our students participated as reviewers in the eighth edition of test reviews in Spain (Viladrich et al., 2020). In this case, the tests were selected for the Consejo General de la Psicología de España (COP) and editorials (i.e., Ciencias de la Educación Preescolar & Especial S.L, Giunti EOS, Pearson Educación, and TEA Ediciones) provided part of the material needed for doing the assessment. Although roughly speaking the students’ task was the same, to review psychometric properties of test scores, the impact of the product on the community increased: the final report, based on the best review of each test and an external reviewer, has been published in the COP (https://www.cop.es/index.php?page=evaluacion-tests-editados-en-espana) making it publicly available (professional beneficiary). The aim of the present work is to present the perceived main strengths and shortcomings considering the type of beneficiary. Retention and percentage of students who passed and students’ satisfaction is similar no matter the final beneficiary. The main advantage is that having a professional beneficiary has more institutional support that just a faculty beneficiary. Having institutional support is crucial in order to apply innovative teaching methods, as there is a need for additional resources (e.g, materials, reduced groups, etc). In this respect, after nine years of experience our university increased the human resources destined to this project, facilitating a better scenario for implementing a PBL methodology. Furthermore, the editorials donated tests for doing the revision partially helping to cover the costs of the materials and to increase the tests or versions available in the test library. Also important was that the professional beneficiary was perceived as a challenge by the professors and invigorated their determination to produce high quality reports.

The main perceived shortcoming with both beneficiaries is that students and professors perceived this methodology as time-consuming. Moreover, when the beneficiary was professional, the professors spend more time monitoring the project as extra meetings were scheduled in order to agree on reports of different versions of the same test. To sum up, the advantages of PBL with professional beneficiaries overcame the shortcomings.

Keywords:

Test review, Test quality, Psychometrics, Service-learning.