TUTORING PROCEDURAL PROBLEMS WITH AN AUTOMATIC SMART EVALUATOR

The learning process has much to do with the trial-and-error paradigm. The aim of this paper is to present a novel computer-based application to carry out this task of tutoring the trial-and-error process, relieving the teacher of repetitive work and providing at the same time a stimulating experience for students, allowing them to obtain a better performance from their effort and greater control over their own learning process. The number of attempts before a problem is successfully solved depends on the individual student, as it is necessary to let the students fight the problems until the right solution is found. Nevertheless, the intelligent evaluator provides clues to pave the way for the right solution.

Multiple choice tests are easy to automate. However, there is a fundamental flaw in this type of assessment. In a trial and error learning process they are not of great help because students can simply try to find the right solution without going deeper into the subject and sooner or later they will forget which option is the right one and why (if at any point they ever got it right). It is basically an appropriate method of assigning a grade to the student. It is clear that one of our missions as teachers is to evaluate students, but even more important is to stimulate and train them before evaluating them.

This work presents a new software package, that we have developed at Universidad de Valladolid, which focuses on procedural problem solving, (automatic) correction and feedback distribution on an online Internet platform. The students' experience would be as follows. At the beginning of the course, the students get and account in GitLab Community Edition, self-hosted and administered by the teacher. These accounts are created automatically extracting student data and pictures from a Portable Document Format (PDF) listing. For each student, the startup algorithm sets up one git project. Git is a software package that records and manages changes to files and documents. It makes it easier to collaborate, because it shows who has changed what and helps to merge contributions. Moreover it can trigger actions due to specific events in the workflow, which is particularly helpful to provide automatic feedback. In our case, the individual project consists of a python notebook in which the teacher challenges the student with a number of problems. Python notebooks are becoming increasingly popular nowadays because of their flexibility to create engaging learning materials with a high degree of interactivity. Among other powerful resources, it allows the teacher to include mathematical formulas, video, dynamic simulations, three-dimensional graphics and executable source code snippets. Students can commit their work to the centralized server indicating that they want to get feedback about their work. Our application executes the answers of the students in a Docker container, allowing the teacher to test typical errors in the procedural problem automatically and therefore provide enriched feedback in form of hints to the students. The evolving nature of this process is handled efficiently as a git project. In this paper, we describe the application and we discuss its implementation in an actual course. Finally the results of a survey among the students which participated in the experience are analyzed.