Abstract:

This presentation is based on the final dissertation from an MA in ICT & Learning from Aalborg University in Denmark that will be completed in June 2018. The project is based on empirical data from the Technical Faculty at Lund University. Since 2003 they have systematically used the evaluation system Course Experience Questionnaires (CEQ) that measures the quality of student learning. In-house research by Dr. Torgny Roxå and Dr. Katarina Mårtensson has revealed that teachers in practice only use the qualitative response and more or less ignores the otherwise considerable amount of quantitative evaluation data. In this project, I investigate if alternative visualizations of the quantitative data and supplementary educational processes can make teachers also use those data with the object of improving their teaching and as an outcome better learning for the students.

I use Engeström’s Change Laboratory model as a meta-structure for the project going through the first four stages (Charting the situation, Analyzing the situation, Creating a New Model Concretising and Testing the New Model) until the 5th stage where the model is ready to be implemented:
Through a participatory design process, I investigate what prevents teachers from using the quantitative feedback and what they need to make active use of them. The study consists of three steps:
- Exploratory interview with teachers about their (absence of) use of quantitative data.
- Exploratory interview with the same teachers with discussion of various proposals for presentation of quantitative data and possible supplementary material for teachers to make more use of them.
- Focus group interview with other teachers from the same education about how they will apply the proposals for presentation of quantitative data based on the results from the first two interviews.

In the data analysis, I use Engeström’s Activity Theory (2011) from an organizational perspective and I use the four central questions that Engeström claims any theory about learning should relate to; who is learning? Why are they learning? What are they learning? How are they learning?

So far, the research has revealed that it is not just a question of improving the data visualization. In order to make teachers increase their use of their quantitative evaluation data you should also consider the information material that teachers and students are receiving as well as upgrading pedagogical courses that teachers are offered.

References:
[1] Edström, K (2008): “Doing course evaluation as if learning matters most” in Higher Education Research & Development, 27:2, p 95-106
[2] Haapasaari, A., Engeström, Y. and Kerosuo, H. (2016): “The emergence of learners’ transformative agency in a Change Laboratory intervention” in Journal of Education and Work, vol. 29, no. 2, p. 232-262
[3] Roxå, T. and Mårtensson, K. (2011): “Improving university teaching through student feedback – A critical investigation” in Nair & Mertova, Student feedback: The Cornerstone to an Effective Quality Assurance System in Higher Education. Chandos Publishing.
[4]Trowler, P. and Cooper, A. (2002): “Teaching and Learning Regimes: Implicit theories and recurrent practices in the enhancement of teaching and learning through educational development programmes” in Higher Education Research & Development, vol 21, no. 3, p. 221-240
[5] Wilson, K. L (1997): “The Development, Validation and Application of the Course Experience Questionnaire” in Studies in Higher Education, v22 n 1 p 33-53

Keywords:

CEQ evaluations, Activity Theory, Quantitative data analysis, participatory design.