Abstract:

In the UK, and elsewhere, recent changes in the higher education policy environment have placed an increasing emphasis on evaluating and reporting university business through the medium of data and analytics. Most higher education institutions now also view the increased use of learning analytics (LA) as being central to realising improvements to student academic achievement, university retention outcomes, and learning & teaching practice.

As a consequence, universities worldwide have launched, supported, and invested heavily in a range of LA projects. These have commonly included initiatives such as establishing Cloud-based ‘data warehouses’ or ‘data lakes’; the visualising of complex data into ‘LA dashboards’; the aligning of LA with learning design approaches; and the evolving of algorithm-based predictive learning analytics models.

These well reported implementations are often accompanied by a prevailing ‘technological determinist’ ideology in which it is accepted that the implementation of ever more technologically complex LA initiatives naturally leads to the realisation of the improvements outlined earlier. However, to date, widespread evidence on the impact of such LA implementations remains inconclusive. Sometimes forgotten is that such LA initiatives also require human engagement and intervention to work. To date, little work has been done to explore how and why educators incorporate LA into their teaching and learning practice, as well as the influence of the eco-systems and contexts in which such implementations take place, on that practice.

This study addresses this imbalance by utilising the Theory of Practice Architectures (TPA) to investigate the evolution of LA practice in the Faculty of STEM at Europe’s largest distance learning organisation, the Open University, UK (UKOU). The findings are based on two sets of semi-structured interviews conducted in 2017 and 2023 with over 30 faculty staff engaged in applying a range of LA implementations to their learning and teaching practice over six years. Using this longitudinal approach, the study has been able to track the evolution of self-identified LA themes associated not only with individuals but also with particular pieces of learning, as well as identifying the arrangements that shape educational practice. The findings from this study will have implications for LA practice across many higher education settings. It will be argued that LA implementations should be designed and integrated into sites of practice that recognise the material-economic, cultural-discursive, and social-political arrangements with which they are entangled and that constrain or enable the practice of using LA. The presentation will provide transferable recommendations for strategies and structures to support educators in the use of LA implementations in other higher education contexts, and move the field beyond merely demonstrating potential.

Keywords:

Learning analytics, educational practice, theory of practice architectures, distance education.