Abstract:

Due to the COVID-19 pandemic, all teaching moved from a face-to-face environment to a fully remote environment in Japan in 2020. Assessing students’ performance remotely could be executed not only by the traditional forms of summative assessments such as using essays, assignments, and a final exam, etc. but also by more formative assessment approaches such as interaction activities, forum posts, etc. Since all the learning activities in remote learning are recorded in LMS (Learning Management System). Teachers could monitor students’ learning processes to evaluate target courses, improve students’ learning outcomes, and predict learning performance. However, it is difficult for the teachers to monitor and assess students’ learning processes using the log data.

This study focuses on how to acquire learning logs automatically from a Japanese commercial LMS and how to visualize students’ learning activities by the retentioneering method.

Manaba is a Japanese commercial cloud-based LMS. It has 250 educational institute users by September 2020. However, Manaba could only support displaying learning activity log data, but not downloading it. In this study, firstly, we developed a program in Python to scrape students’ learning activity log information from the Manaba web pages. We collected 56446 lines of clickstreams log data from 122 students in two computer literacy hybrid classes in the fall semester of 2022 (2022/9~2023/1). Each data included the information of student’s name, ID, access time, URL, function category of the page, type of the page, and the title of the page. In this study, the ID, access time, and the type of the page information were used in the next retentioneering analysis.

Secondly, to visualize and assess students’ remote learning activities, we used the Retentioneering Python library to analyze students’ clickstreams and page transit trajectories. By graphing the network of students’ clickstreams log data, we could clearly see which learning pages were connected by students’ transitions. And by the weight of the edges between each page, we could understand how many students transit from one learning page to the other learning pages.

Thirdly, we use a step matrix to show the sequential learning pages that the students accessed step by step. By the step matrix, we could know how many steps had been passed before the remote learning ended, and whether all the necessary learning pages were accessed before the learning ended.

Lastly, we clustered students’ learning behavior into 6 clusters by the kmeans clusterization algorithm. We named the clusters into report-centered without reading Q&A thread group, report-centered with reading Q&A thread and resubmit reports group, learning contents-centered and comment-based report group, learning contents-centered only read comment group, little reports just reading Q&A thread and comment group, little reports but posting comment group. From the clusters, we found that students’ remote learning behaviors were complicated. We could divide their main learning pattern into content-centered and report-centered students. And we found some students never read the Q&A thread or the teacher’s comment.

This study contributed to visualize students’ remote learning activities into a state-transition graph by the retentioneering method and teacher could monitor students learning process and assess their learning performance by their learning activity patterns.

Keywords:

Online learning, learning analytics, learning activities, retentioneering.