Abstract:

Learner engagement is one of the most important constructs in education research and practice. It is “a critical component of long-term achievement and academic success”, a determinant of “students’ … experiences in school, both psychologically and socially”, and a factor affecting “students’ resilience and the development of resources for coping adaptively with stressors” (Gobert et al., 2015). Engagement has consistently been found to be positively associated with and a good predictor of learning and learning-related outcomes.

Last year, COVID-19 changed the higher education landscape drastically, with increasingly more educational and training institutions supplementing, complementing or even partially or fully replacing face-to-face learning with online or blended learning. This has put online engagement into a sharper focus than ever before, resulting in it taking on increased and increasing significance.

Tan and Koh (2018) have proposed a framework to operationalise online engagement with data from the learning management system (LMS) – a critical component of online learning –by providing measures of different aspects of online engagement. This paper presents a case study of the implementation of Tan and Koh’s (2018) framework and discusses the extraction, pre-processing and preparation of huge raw data from Canvas.

In particular, the paper covers how Canvas data files are transferred automatically to cloud-based servers (AWS S3) using massive server-less function calls (AWS Lambda), how this transfer is tracked, how a local data catalogue that describes the content of Canvas data files is created and then queried using a virtual database creation tool (AWS Glue) and a data analytics tool (AWS Athena), respectively.

Based on the extracted datasets from Canvas, the paper also discusses how SAS programming is used to generate the online engagement metrics by extracting click-through transactions and filtering irrelevant ones; sorting and tabulating user identification, time stamp and session code; generating session data; and finally computing the online engagement metric.

From the final dataset comprising the online engagement metrics, the nomological validity of the online engagement metrics was assessed by examining their relationships with academic performance using multiple regression. The results indicated nomological validity, providing support to use the online engagement metrics to facilitate the explanation or prediction of learning outcomes in an online learning context.

In line with and in addition to the above, it will be useful to investigate the antecedents and consequents of online engagement to aid teaching and learning and optimise their outcomes, as well as enhance student satisfaction. It is hoped that this case study, which illustrates the measurement of online engagement metrics from LMS data, can make a contribution towards enhancing teaching and learning.

References:
[1] Gobert, J D, Baker, R S, & Wixon, M B, 2015. “Operationalizing and Detecting Disengagement Within Online Science Microworlds”, Educational Psychologist, Vol 50 No 1, pp 43-57.
[2] Tan, J W C, & and Koh, H C, 2018. “Operationalising Online Engagement”, International Academic Conference on Global Education Teaching and Learning (Vienna: Austria), November 2018.

Keywords:

Learner engagement, online learning behaviour, framework for online engagement, implementation of online engagement.