Abstract:

This presentation reports on the results of an experiment that compares the evaluation of teacher candidates’ lessons using three different observation rubrics: The Texas Teacher and Evaluation Support System (T-TESS), The System for Teacher and Student Advancement (TAP), and the Rapid Assessment of Teacher Effectiveness (RATE). Candidates were enrolled in a university teacher education program at a prominent Texas university. Teacher observation is an important component of teacher mentoring, coaching, and training, and is a major factor in a teaching candidate’s overall assessment. Reliability, in particular inter-rater reliability, is critical to the validity of these assessments. However, most observation measures do not report data on their reliability. Poor reliability can lead to misdirection in coaching decisions and mis-classification of a candidate’s teaching ability. Our experiment randomly assigns six trained observers to each of three conditions, defined by three different observation measures. Subjects, all of whom are experienced teacher educators, review 20 videos, rate the teachers, and results are analysed for inter-rater reliability (using three different measures), inter-instrument agreement (using correlations and rankings), and how they correlate with an independent overall measure of the candidates’ teaching quality. Differences in results among the instruments are described and their hypothesized causes are discussed. Implications are drawn for teacher education programs in particular, and teacher evaluation in general.

Keywords:

Teacher evaluation, teacher observation, reliability.