Abstract:

During the past decades, the use of student evaluations of teaching (SET) performance has been an important but controversial tool in the improvement of teaching quality. Although SET are implemented in many universities, not everyone is convinced of the reliability and validity of these ratings. Recent research demonstrates that evaluations can be influenced by factors other than teaching ability such as those concerning the lecturer, the students and the course—influencing or biasing the results of student evaluations. This research examines the validity of SET. The main aim of this study was to determine whether a halo effect occurs in the completion of SET ratings and to estimate the magnitude of this effect. In this paper, we present the results of a study with regard to the existence of a higher-order factor that might influence students’ perceptions of teaching and, thus, explain the variance in rating scales. The sample consisted of 1125 full time undergraduate students at a franchised Australian university programme in Malaysia. An 11-item teaching effectiveness with 5-point Likert scale was administered to these students. The scale was designed to measure two dimensions of teaching effectiveness - lecturer attributes and module attributes. Reliability analysis and exploratory factor analysis are performed. After an exploration using Cronbach’s alpha testing and factor analysis, these items are introduced in a confirmatory factor analysis using structural equation modelling. The results of this study raise issues regarding the interpretation and utility of SET ratings since the attribute of charisma is having a central trait effect on student evaluations.

Keywords:

Student evaluations of teaching, SET, lecturer evaluation, validity study, student ratings.