Abstract:

In recent years, vocational education and training (VET) has changed its perspective from transfer of knowledge towards competency development. This occurs, because the transfer of declarative knowledge is not sufficient to develop skills which are necessary to solve complex tasks in a working environment (Shavelson 2012; Erpenbeck & Sauter 2013). Therefore, it is crucial for competence measurement to create authentic action situations to improve the validity of tests, which should result in a higher adaptability of test results on reality and everyday situations (cp. Hartig & Jude 2012). Video elements have the potential to increase the validity of tests for competency measurement significantly, and therefore ensure that test results reflect the test persons skills for actual problem solving (Klerk 2012; Bennett et al. 1999).

The quality of technical equipment used in institutions of VET is constantly improving. Computers and electronical test centers enable it to use video elements on a large scale for training and education. While there is a variety of research on the use of videos in teaching (cp. Mayer 2014), the determinants of design and use of video elements for assessments is hardly examined (cp. Seidel & Jödicke 2014; Cheng & Bischof 2007). This is the motivation of this contribution.

For a study on the vocational competencies of qualified medical employees, a computer-based competency test was designed and applied based on ILIAS, a digital learning management system. More than 1,000 test persons have been assessed. The purpose of this contribution is to empirically examine the effects of different aspects of video design on the behavior of the test persons, using the results of this competency test.

First, criteria for classification for the test items and videos are elaborated. Second, the effects of these criteria are tested statistically using correlation and regression analyses. The analyses are based on the data of 21 videos and the related items. Statistical criteria include the parameters of the items and determinants of difficulty levels as well as motivational and emotional aspects. In the result, the contribution shows the effects of different aspects of video design – like the charisma of the actors or the presence of authorities – on the difficulty of the item and the motivation of the test persons. Furthermore, technical aspects – like the length of the videos or the format of the items – are considered.

The results are a contribution to the discourse of test design and the use of video elements in competence measurement. Practically, the findings may be used to improve the construction of video-based items. The results are not limited to the context of VET, they may be relevant for all areas of education.

Keywords:

Multimedia based testing, video based testing, assessment, competence measurement, learning management system, ILIAS, computer-assisted-testing.