Abstract:

The job market places increasing value on graduates’ transversal competencies, such as working in multidisciplinary and multicultural teams. The development of transversal competencies is nowadays a learning goal across Higher Education Institutions (HEIs). An assessment is needed to determine students’ competencies throughout their learning journey and at the conclusion of their graduate studies, which is rapidly becoming the required level of education for an increasing number of graduate-level jobs. However, it remains challenging to develop an assessment method fit for the purpose of measuring transversal competencies.

The Multiple Mini-Interviews (MMI) method is internationally used for the assessment of cognitive (e.g., creativity) and noncognitive (e.g. ethics) transversal competencies in procedures for admitting candidates in the Health field. The MMI imply multiple observations of each applicant through simulated standardized stations that represent frequent scenarios in medical practice (e.g., giving bad news). Stations require the participation of trained interviewers and may stage role-play scenarios which call for the involvement of trained standardized patients (SPs) who are individually trained to perform reliably with all the candidates (e.g. a colleague in trouble).

In this paper, we aim to describe the adaptation of the MMI to assess graduate students’ transversal competencies in different work contexts. This adaptation is part of an ongoing project that started with the mapping of the relevant competencies for the future of work, followed by a group brainstorming for the scenarios’ construction, the selection of scenarios to be tested, the definition of the assessment criteria and scale, the interviewers and SPs’ (hereafter called standardized characters - SCs) training, and the preparation of the pilot study.

The adaptation resulted in a MMI circuit with 10 stations, each one aimed at assessing three or four of the following transversal competencies: problem solving, generation of novelty, open mind-set, learn to learn, positive attitude at work, teamwork, effective communication, leadership, and market orientation. Criteria to rate the performance level in each competency were defined and listed in the instructions. The training of the interviewers included the individual analysis of the scenarios and the criteria for performance assessment, the analysis and classification of videotaped simulated performances in each scenario considering the instructions and the assessment criteria, the discussion, in a team meeting, of doubts, suggestions for improvement, and the harmonization of assessment practices. The SCs’ training involved the analysis of the scenarios, competencies under assessment, criteria for assessment and role specifications, and a discussion with the researchers to clarify instructions and timings. The following step was the preparation of the pilot study to test the adequacy of the MMI.

With the development of the MMI method, we ultimately expect to give HEIs the support to assess their students’ transversal competencies profile, evaluate their readiness for the transition for the labor market and potentially identify gaps for improvement.

The MMI method is currently being adapted to enable full-remote assessment of the transversal competencies, fostered by COVID-19 constraints and recommendations for HEIs.

Keywords:

Assessment, graduate work, higher education, multiple mini-interviews, transversal competencies.