Abstract:

Due to an increasing digitalization, many routine activities in companies are automated. Remaining tasks are complex; employees therefore more often need problem-solving competencies. Those are anchored in German vocational education and training and hence, problem solving is increasingly included in instructional processes. However, professional examination practices in Germany are criticized because primarily knowledge reproduction is measured (Wuttke et al., 2022). This is partly due to the fact, that the development of complex problem-oriented tasks and their technology-based implementation have hardly been addressed in teacher education and training to date. Consequently, there is a need in providing teachers with corresponding knowledge and skills. This formed the starting point for the development of a training programme for the development of problem-oriented and technology-based examination tasks for teachers and trainers in commercial professions.
The training was designed in such a way that (1) the characteristics of problem-oriented tasks were addressed. Furthermore, (2) diagnostic aspects were addressed. We (3) trained to use a digital platform as a tool for the technology-based development of the tasks. Because this kind of examinations offer a variety of design options for an authentic implementation of tasks based on professional reality and existing requirements. In order to provide information on the design and use of digital media, the training concept was further expanded to include (4) media didactic training.
Our central research question is, if the training is successful, i.e. if participants are satisfied with the training, consider it as useful, and report learning. As a measure for the success of the training we also analyze the tasks developed by participants in the course of the training.
We assessed the success of the training in a pre- and post-survey and test. The evaluation was done using three of the four stages of Kirkpatrick and Kirkpatrick (2006), namely reaction (satisfaction with the training, fulfilled expectations), learning (knowledge tests to task development and quality criteria) and behavior (developed tasks). Here, we focus on satisfaction and expectations as well as learning.
The training sample includes N=146 teachers and trainers in vocational education. Results so far indicate the training was successful. On a scale from 1 (not true at all) to 4 (completely true) the participants report a mostly positive reaction to the training. This applies to satisfaction with the training (median = 3), the self-assessed training benefit (median = 3) and the self-assessed learning gain (median = 3,25). Changes in the knowledge tests (α = 0.667 for features of problem-orientied tasks, α = 0.742 for diagnostics) partly confirm these findings. Our analyses show a significant (t=-5.554, p=0.000) knowledge gain in the test on features of problem-oriented tasks. However, there is no knowledge gain concerning diagnostics. The results are discussed with regard to the professionalization of teachers and trainers in commercial professions.

References:
[1[ Wuttke, E., Seeber, S., Geiser, C., & Turhan, L. (2022). Zur Problemhaltigkeit von Aufgaben in kaufmännischen Abschluss-und Zwischenprüfungen–Ergebnisse aus Aufgabenanalysen. Zeitschrift Für Berufs-Und Wirtschaftspädagogik, 118(1), 25-52.
[2] Kirkpatrick, D. L. & Kirkpatrick, J. D. (2006). Evaluating training programs. The four levels (3. Aufl.). San Francisco: Berrett-Koehler.

Keywords:

Teacher education, problem-oriented and technology-based examination tasks, vocational teaching and examination professionals, vocational education and training.