Abstract:

Students evaluations of teaching quality in higher education are difficult to validate. Different criterias of validity (e.g. instructors judgments) have often low measurement accuracy or low validity. In order to examine multidemensional student evaluations, student data can be compared with assessments by instructors and third persons. The research project “Externe, pädagogisch-psychologische Beratung von Dozenten/innen zur Verbesserung der Lehrqualität mit Fokus auf effektiver Nutzung sozio-kognitiver Konflikte” (“SoKonBe”) is introduced which aims at improving teaching quality of university instructors and compares the evaluation of teaching quality in higher education. In the project, the instructors themselves, their students as well as external instructors evaluate the lectures – the external instructors by means of video sequences of the lectures. Teaching quality was assessed by means of the “Heidelberger Inventar zur Lehrveranstaltungsevaluation II” (HILVE II).

So far it was necessary for external instructors to attend the lecture or to know the instructor in his teaching. Otherwise there was the risk that no teaching quality was judged, but only indirectly, the quality of research, observed behavior or the quality of the “hearing quality”. In this contribution, we present a special developed evaluation web portal for a better, easier, faster, more cost-effective and more valid external evaluation of teaching quality. So it is possible that external instructors from the same or different study subjects evaluate lectures without direct presence.

In order to use the web portal, the external instructors register themselves from their PC via a socio-demographic questionnaire. After that they find the assigned video sequences (15 minutes of the whole lecture) in their user account. These can be viewed directly on the website of the Department of Educational and Development Psychology of the Chemnitz University of Technology (password-protected). The external lecturers then go directly to the online HILVE questionnaire. In addition to this front-end user area, there is also a back-end administrator area for academic staff and student assistants. In this, data can be viewed as well as videos can be assigned to the participating instructors. The program takes almost completely the email communication with the external instructors. These will receive automated notifications after the release, the registration, the final assessment as well as a reminder in the case of non-participation or in the case of delay. The portal offers the instructors the opportunity to receive a feedback on the self-perception and the perception of others (two perspectives) of the teaching unit he developed.

In total the data of 1714 students from 45 instructors were collected. With the help of the web portal also 335 evaluations from 88 external instructors (exclusively professors from 30 german universities) were collected. The judgments between students and external instructors correlated higher (r = .59) than between students and teachers (r = .25) or external lecturers and teachers (r = .26). It appears that the evaluation with the help of the web portal is a valid and effective measurement. This project is funded by the German Federal Ministry of Education and Research (BMBF).

Keywords:

Web evaluation, teaching quality, university instructors, students’ evaluations.