Abstract:

The learning path in programming-related courses involves that teachers must regularly and timely assess the knowledge and practice acquired by students. For that, teachers must prepare the assignments that students must develop on their own and then submit for assessment. Two factors are closely related with the student success, namely, the number of activities they deal with and the (timely) feedback they receive for their submissions. That is, the more assessments the students work in and the faster they get feedback from teachers, the better they will develop their programming skills. Although the benefits for students are clear, teachers require some tools to help them in the assessment process, otherwise unapproachable.

Some features can be assessed automatically, thus releasing teachers from this tedious task. The programming assignments usually involve text files with code and executable programs, and there are many software libraries that help in dynamically and statically assessing them. Indeed, some authors have reported their successful experience on developing and applying tools for the automatic assessment of programming assignments such as the Virtual Programming Lab, JUnit Question Type, or Online Judge. These tools apply and combine different grading criteria, so as to provide students with proper and timely feedback on their submissions. However, what about the effort required for teachers to design and develop the assessment programs?

Creating, configuring and releasing an assessment task is not an easy job. First, the teachers must look for the tools and libraries that help them with the assessment steps. Then, these tools must be properly integrated, so as to produce an assessment process tailored to each assignment. Finally, the resulting artifacts must be integrated with the learning environment in place, so that students can easily use it. As a result, automating the assessment of programming assignments emerges as a time-consuming task for most teachers, who spend many hours preparing the assessment, and then configuring and integrating it with the existing learning infrastructure.

To tackle the aforementioned issues, this paper describes our experience with the design, implementation, and validation of a tool to support teachers in developing automatic graders of programming assignments. Specifically, the tool provides a Graphical User Interface (GUI) that allows teachers to combine a set of predefined grading tasks that will process the student submissions, and an easy configuration of each task. The tool provides support for ad-hoc programming assignments management, and advanced programming features such as compiling the source code, running the resulting programs, testing their functionality, or assessing the code style and documentation. The tool integrates with Moodle, so that students do not need to learn yet another learning environment, and get their marks and instant feedback on site. The system requirements and functionality, as well as the solution architecture are described.

Keywords:

Programming assignment, automatic grading, evaluation metrics, assignment assessment, online assessment, moodle, programming.