Abstract:

Currently, vocational education and training (VET) face significant challenges, primarily stemming from the digitalization of the world of work. Simple jobs, like cashier positions, are becoming increasingly automated (Frey & Osborne, 2017). In the case of more qualified jobs, such as those of an industrial clerk, various trends are noticeable. Even here, a broader range of routine activities is being computerized and automated, while tasks that cannot be easily replaced by technology have become more demanding. According to Tynjälä (2013), an increasing number of qualified employees with VET degrees must engage in problem-solving. In this context, the promotion of higher-level skills, particularly vocational problem-solving skills within VET, is gaining importance.
VET curricula are evolving to address these demands.

However, to achieve such ambitious goals, not only curricula but also instruction in VET schools and companies, as well as assessment in intermediate and final examinations, must focus on problem-solving competences. This forms the curriculum-instruction-assessment triad (Pellegrino, 2010). Nonetheless, examination tasks, in particular, present challenges. Analyses of previous examinations reveal significant limitations in measuring problem-solving competences (Wuttke et al., 2022). One reason for this is that teachers and trainers, who submit task proposals to the chambers, may not be adequately qualified to design tasks that measure problem-solving competence. This gap led to the development of a training program aimed at enhancing the competence of VET teachers and trainers in designing and implementing problem-oriented, technology-based examination tasks on digital learning platforms in commercial vocations.

The training program consists of different modules and was conducted in two formats: blended learning and blended learning with flipped classroom elements. The success of the training was evaluated using a pre-posttest design, following Kirkpatrick and Kirkpatrick's (2011) evaluation levels, which include "reaction," "learning," "behavior," and "results." The program took place over several runs from November 2020 to October 2022, involving a total of 152 participants (59 male, 93 female).

Test results show a high satisfaction with training among participants (reaction level, with an average rating of 3.0 on a scale from 1 to 4), and demonstrated significant knowledge gain (learning level, t=3.473, p=0.001). The knowledge gain about characteristics of problem-based tasks and problem-based learning was measured with a multiple choice and open answer test. At the end of the training, the participants were able to evaluate technology-based examination tasks with regard to their problem-solving potential and had the necessary skills to work out problem-oriented tasks themselves. In order to evaluate the applicability and transfer on the behavioral level of the training, the digital problem-oriented tasks developed by the participants were assessed by trained experts (behavior level).

Keywords:

Technology-based Problem-solving Tasks, Problem-oriented Examination Tasks, Digital Test Environments.