Abstract:

In the following study, we deal with data-driven support for novice programmers -- students working in large groups. To support students, we introduce a virtual assistant. Based on achieved data (learning analytics), the assistant aims to simplify communication and feedback, sending reports and advice to students in two programming courses.

Both courses include multiple programming assignments tested by an automated assessment system. In the middle of the assignment period, the virtual assistant sends reminders to stagnating students and sends them some hints to "kick up" their activity. After every deadline, students receive motivating statistics, e.g., whether they appeared in the top 20%. On the other hand, if the assistant detects students who stopped in their progress, it sends supportive messages to them. These students can also get personal support from their lecturer, who is notified too.

We experiment with massive groups of students during two semester periods within CS0 and CS1 courses, with approx. 1,000 and 850 students. Each experiment includes a treatment and a control group. Evaluation is focused on the performance, behavior, and motivation of students.

We show that using learning analytics, the virtual assistant is a handy way to support large student groups, positively influencing their performance, behavior, and motivation.

References:
[1] VIBERG, O. et al.: The current landscape of learning analytics in higher education. Computers in Human Behavior 89(1), 2018, pp. 98-110, doi: 10.1016/j.chb.2018.07.027
[2] RANJEETH, S. - Latchoumi T. - VICTER, P.: A Survey on Predictive Models of Learning Analytics. Procedia Computer Science 167(1), 2020, pp. 37-46, doi: 10.1016/j.procs.2020.03.180
[3] SCHUMACHER, C. - IFENTHALER, D.: Features students really expect from learning analytics. Computers in Human Behavior 78(1), 2018, pp. 397-407, doi: 10.1016/j.chb.2017.06.030
[4] AHSAN, Z. - OBAIDELLAH, U.: Predicting expertise among novice programmers with prior knowledge on programming tasks. IEEE Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2020, pp. 1008-1016
[5] SULIR, M. et al.: Large-Scale Dataset of Local Java Software Build Results. Data 5(3), 2020, pp. 1-11, doi: 10.3390/data5030086

Keywords:

Virtual assistant, novice programmers, automated assessment, learning analytics, motivation, educational data mining, program comprehension.