Abstract:

Analyzing students’ detailed learning processes—such as video viewing histories in MOOC courses or e-book usage logs—can provide valuable insights into behavioral characteristics associated with learning performance. However, such tools are not always available due to financial or instructional constraints. This study therefore examines whether meaningful signals for predicting learning performance can be extracted solely from logs on Learning Management System (LMS), which have become widely used in higher educational settings. In particular, we investigate whether the operation sequences of LMS menu items can be leveraged to predict final report scores.

Two types of prediction models were built and compared. The first model uses graph2vec to obtain distributed representations of graph structures, treating each student’s LMS operation sequence as a graph and extracting structural learning patterns as features. The second model uses simple behavioral features such as access counts and timing. Unlike end-to-end approaches such as GNNs, graph2vec offers a degree of interpretability, capturing local LMS operation patterns common across students and enabling the identification of learning behaviors associated with high performance.

The dataset consists of LMS logs from all 15 sessions of the course “Introduction to Data Science and AI (Python class; typically taken by first-year students)” offered across departments at Tohoku University during the second semester of the 2022 academic year. The data include logs for 82 students and their final report scores. Predictor variables were constructed in two ways:
(1) 128-dimensional graph embeddings obtained via graph2vec, and
(2) 82 selected LMS operation types chosen from 226 recorded menu actions, including course-home access, downloading lecture materials and exercise data, assignment submission, and viewing the Q&A or class information boards. The dataset comprises 13,650 log entries.

Prediction performance was evaluated using a random forest model with holdout validation repeated 100 times using different random seeds. The model using only simple features achieved an average R² of approximately 0.50, nearly double the roughly 0.25 obtained using only graph features. However, when both types of features were combined and their contributions examined using SHAP, two graph-based features appeared within the top 10 features. Examining 4-gram sequences for students with high scores on these graph features showed frequent access to lecture materials and exercise datasets relevant to the final report, as well as repeated viewing of materials from more challenging class sessions, even when not directly required for the report. This indicates that graph features may capture learning behavior patterns related to academic performance that simple features fail to represent.

These results likely reflect that in selective higher education contexts—where students pass a rigorous entrance examination—the variance in fundamental academic ability is relatively limited, especially among lower-year students. Consequently, differences in overall learning volume may contribute more to grade disparities than the nuanced learning approaches captured by graph features. Nonetheless, the findings suggest that graph-based modeling of LMS learning patterns, combined with structural feature extraction, holds promise for identifying behavioral patterns that contribute to learning performance.

Keywords:

Graph embeddings, lms, academic performance.