Abstract:

Why is it that research and evaluation studies of learning technologies have had so little impact on implementation decisions and teaching practices in higher education (HE)? Has a body of evidence been built that can inform and provide a firm foundation for subsequent developments in academic practice? Is evidence being generated and reported that can inform the future practices of teachers and students in higher education? If the HE sector aspires to greater use of an evidence-informed approach to innovation and change, we need to ensure that the research and evaluation of learning technology projects produces findings and ‘lessons learned’ that can inform other practitioners and policy-makers.

While there are issues about what types of evidence are considered during any implementation decisions (Price & Kirkwood, 2013), concerns have also been expressed about the lack of a well-established body of evidence and about the quality and validity of many research and evaluation studies. In our review of research literature, reports and case studies of technologies for teaching and learning in higher education (HE) we identified many problems with the ways in which studies had been conceived and conducted, such that it was difficult to generalise any findings about effectiveness. There were issues relating to assumptions and beliefs underpinning research studies and the approaches used to investigate the impact of technologies (Kirkwood & Price, 2013a). Frequently, there was a lack of clarity about the nature of the enhancement that technology was intended to bring about and what impact it would have upon the student learning experience (Kirkwood & Price, 2014). Furthermore, few published accounts of applying learning technologies in HE showed evidence of a scholarly approach to university teaching. Frequently, interventions appear to be technology-led rather than being a response to identified teaching and learning concerns (Kirkwood & Price, 2013b).

In the presentation we will outline a number of the shortcomings identified in published studies and consider the implications. We then suggest ways in which these limitations could be avoided through a more rigorous approach to conceptualising, designing, conducting and reporting research and evaluation studies relating to technologies for teaching and learning. The manner in which HE institutions support and develop academic teachers also needs to be reconsidered.

References:
[1] Kirkwood, A. and Price, L. (2013a). Examining some assumptions and limitations of research on the effects of emerging technologies for teaching and learning in higher education. British Journal of Educational Technology, 44 (4), 536-543.
[2] Kirkwood, A. and Price, L. (2013b). Missing: Evidence of a scholarly approach to teaching and learning with technology in higher education. Teaching in Higher Education, 18 (3), 327-337.
[3] Kirkwood, A. and Price, L. (2014). Technology-enhanced learning and teaching in higher education: What is ‘enhanced’ and how do we know? A critical literature review. Learning, Media and Technology, 39 (1), 6-36.
[4] Price, L. and Kirkwood, A. (2013). Using technology for teaching and learning in higher education: A critical review of the role of evidence in informing practice. Higher Education Research & Development. ‘Online first’

Keywords:

Learning technology, research, evaluation, enhancing learning.