Abstract:

Attainment gaps are now a serious topic of concern for university administrations, and the old attitudes that students are solely responsible for their own failure, have been superseded by an institutional concern for equity in summative assessment. While many systemic factors remain to be resolved, we suggest early-warning signs which may be useful in detecting student under-attainment, and the means by which these might be addressed.

To investigate this, we monitor the responses to classroom polls in a series of 6 workshops of a large first year undergraduate finance course with over 1000 students, asking students at the start of the workshop whether, and to what extent, they have prepared for the workshop, and at the end of the workshop, about their own perceived increase in learning that day. We also use in-class quizzes as a formative assessment tool, asking all students one quantitative and one qualitative question each workshop, recording their responses. We link this to a pre-course demographic questionnaire, and a pre-course survey on how they feel about the course before its commencement. Finally, we track their final exam results. Students are traceable through the study, though anonymised from the investigators.

We find:
1. Male students are significantly more likely to be represented amongst the lowest-performing.
2. Failure to prepare before the workshops, to any degree, strongly predicted low attainment. Interestingly, the most significant indicator was a lack of such students reporting having looked at the questions and having attempted a few; this moderate level of preparation seems to lift students out of the lowest attainment zone.
3. Reporting significantly less learning at the very highest level, is strongly predictive of the lowest achievement, but the other levels of perceived learning are not. Students who go on to perform poorly do not necessarily report that they feel they have not learnt anything after lessons.
4. Not participating in classroom polls in the second half of the course is highly predictive of poor performance on the exam, as is ceasing to participate in the second half, having frequently participated in the first half of the course. This ‘drop-out ‘ effect outweighs any influence of getting the answers right or wrong.
5. Considered on their own, getting quantitative questions wrong is more predictive of poor performance than getting qualitative questions wrong.
6. Students failing to contribute their opinions of any kind in the second half of the course regarding their own preparation and learning was significantly predictive of the poorest attainment. Over the whole course, simply not responding to the questions on their own perception of learning was most influential of students’ under-performance, perhaps indicating that they were attempting to avoid reflecting on their own learning.
7. These conclusions are very similar, whether we examine the poorest 30%, 20% or 10% of student attainment on the final examination.

We add to the literature by examining multiple measures of student engagement and participation simultaneously, and at repeated intervals through a course, rather than taking one snapshot of general perception. The greatest warning signs are a failure to remain engaged in classroom response through the whole course, and quantitative rather than qualitative poll questions seem to be most effective at identifying students in need of extra help.

Keywords:

Formative assessment, student engagement, perceptions of learning, enhancing attainment.