AUTOMATED GRADING AND FEEDBACK PROVIDING ASSIGNMENTS MANAGEMENT MODULE

Practical programming is one of the basic skills pursued in the first course of engineering degrees. However, it becomes a great challenge for both students and lecturers. Firstly, a lot of assignments are required to help students to improve their skills in every aspect of the subject. Secondly, each assignment submitted by a student must be carefully assessed by a lecturer from different points of view, so that the student can get early and personalized feedback that allows for a steady improvement. If students get feedback only after the final submission of the assignment, the learning process will not be optimized, especially given their lack of experience on testing. As a result, manually assessing the huge amount of students’ submissions is tedious and time-consuming.

Since the submitted assignments are executable programs, some features can be assessed automatically. Yet depending on the complexity of the assignment, the whole process might require a high degree of customization. Further, integration with the Learning Management Systems (LMS) of the home institution is a must, since both students and lecturers are used to this LMS environment, and learning how to use yet another tool is time-consuming and degrades the learning experience.
Moodle, one of the most prominent LMS, is the LMS in place in the Telecommunications (ICT) Engineering School at Universidad Politécnica de Madrid. Moodle lacks tools for the compilation and execution of programming language assignments. Some modules are available in the state of the art that might help in these tasks, such as Virtual Programming Lab (http://vpl.dis.ulpgc.es/) or JUnit Question Type (http://docs.moodle.org/20/en/Junit_question_type). These modules allow students to code their programs online, which are them compiled and run in the Moodle server. This was not possible in our case, since the server was a sensitive infrastructure that must be protected from erroneous or malicious code uploaded by the student through compilation and execution on another server with restricted permissions. Programming assessment plugins for Moodle can be also found in the state of the art such as Online Judge (https://github.com/hit-moodle/onlinejudge) or EPAILE (http://moodle.org/mod/forum/discuss.php?d=70077). However, these tools do not fulfill our requirements since the former was not configurable to our specific environment and the latter was discontinued.

This paper describes our experience with the design, implementation and validation of an assignments management module based on Moodle LMS. The module provides support for ad-hoc programming assignments management, and advanced programming features such as compiling the source code, running the resulting programs and testing their functionality. The features are provided in a sandboxed computer to avoid damaging the Moodle server. The module also integrates with Moodle grading system, so that students get their marks and an early feedback. The system requirements and functionality, as well as the solution architecture are described.

Furthermore, the new module has been validated with a set of 460 students enrolled in the subject called Programming, which requires the students to submit six assignments during the four-month course. The submission and assessment process have been instrumented and recorded to enable the subsequent analysis. This paper presents the results as well as the conclusions obtained and the lessons learned.