Abstract:

How we teach in university contexts must be adapted to the findings of scientific research. In the last few years, it has been emphasized that regular evaluation by means of self-assessment quizzes and formative feedback (which points out the strengths and weaknesses of the person being evaluated) are two teaching tools that promote learning among students. This contrasts with the traditional summative evaluation, usually operationalized in a single evaluation at the end of the course whose sole purpose is to sort students throughout a continuum and determine whether they meet a certain cut-off point, without the aim of allowing intervention on the specific learning needs. In this work, we present the analysis of the responses to self-assessments conducted in the context of the subject Psychometrics from the Psychology degree during the 2021/2022 and 2022/2023 academic years. The responses were analyzed under the G-DINA model which, by using a matrix that specifies which competencies are assessed by each item (e.g., remembering, understanding, applying), classifies the examinees in a latent competency profile indicating whether they mastered each of these competencies or not. The analyses were carried out using the FoCo application (https://psychometricmodelling.shinyapps.io/FoCo/), which incorporates analyses from classical test theory and also allows the estimation of the G-DINA model. From the information provided by the application, individual reports were generated for students, as well as group reports summarizing class performance. Both reports could be used by students and teachers to act on learning needs during the course. These results are encouraging based on the positive feedback from the participants. The main difficulty throughout the evaluation process was to motivate students to endorse the self-assessments at the end of each subject topic, rather than right before the mid-term or final exam. Work remains to be done on the implementation of a system that allows the integration of the evaluation environment (in our case Moodle) with the analysis environment (FoCo), so that feedback can be more immediate. The use of this application, programmed in the R environment, can also serve to encourage the learning of this programming language by Psychometrics students, a line that we are currently working on. In general, this line of work favors the integration of advances in Psychometrics and increasing access to information technologies.

Keywords:

Self-Assessment, Competencies, Psychometrics, Cognitive Diagnosis Models.