A FRAMEWORK FOR BACKGROUND QUESTIONNAIRES OF LARGE-SCALE ASSESSMENTS WITH A FOCUS ON TEACHING PRACTICES THAT INFLUENCE STUDENT ACHIEVEMENT IN MATHEMATICS

In the context of large-scale assessments of student achievement, the results of empirical analyses of background questionnaire data suggest that among a variety of school-related factors teaching practices affect student achievement. Although many large-scale assessments already question teachers on such practices, underlying frameworks remain generally based on common-sense anecdotal propositions rather than on empirical data. This presentation reports the results of an empirical study that examined the extent to which teaching practices surveyed in a teacher background questionnaire aligned with practices found in the literature shown to improve student achievement. Childs and Broomes (in press) provide a list of contextual variables that are typically considered when trying to explain student outcomes on large-scale assessments. They showed that students, teachers and parents have the most direct impact on student achievement. At the classroom level, teaching practices have the most influence on student achievement. We therefore compared frameworks found in nine published works on the relationship between learner-centered teaching strategies and student achievement. The comparison yielded five dimensions: a) cognitively challenging tasks; b) meaningful and engaging hands-on activities; c) independent learning opportunities; d) group work; and e) assessment and feedback to support learning. These dimensions appear to align with those targeted by current Mathematics reforms. We then statistically examined the results of one specific teacher background questionnaire because it featured a large proportion of items focusing on teaching and assessment practices. The data were collected from 790 schools and the inclusion of 4,501 teachers. The teacher background questionnaire contained a total of 181 questions, 32 of which were grouped into following three main questions. Principal Component Analyses (PCA) with a varimax rotation were conducted. The PCA clustered 22 items around four components with eigenvalues greater than 1 and representing 66.4% of the total variance. These components were found to correspond to four of the five types of teaching practices resulting from the literature review. The Cronbach reliability coefficients for each of the four summated scales are .83, (group work), .83 (meaningful and engaging activities), .87 (cognitively challenging tasks) and .76 (assessment and feedback). The results of this study offer some support for at least a four factor framework as the basis for a teacher practice section of background questionnaires. However, the match is not perfect. For example, the item ‘write solutions using mathematical language and symbols’, typically associated with communication skills, loaded on the ‘meaningful and engaging tasks’ component rather than on the ‘cognitively challenging activities’ component which included communication. Similarly, the item ‘student presentations to other students’ ’, from Question 3, should have loaded either on the ‘assessment/feedback’ component, on the ‘group work’ component, but it did not load on any component. Furthermore, absent in the results was a component related to 'independent work'. The observed discrepancies noted above may be explained by the specific large-scale assessment context of this study. During the presentation, we will engage discussion around such issues.