Abstract:

Since the demands of learning coding skills are increasing, the universal education on programming is spreading. Instructors commonly have to handle both lecturing and grading assignments. Students commonly submit their assignments as source codes in programming education. There are several automated assessment systems for programming assignments such as DOMjude, Web-CAT, and BOSS which are supposed to reduce the works of instructors and make the assignment submission convenient for students. However, it is rare to find a system which can not only determine the correctness of program output but also evaluate the code quality.

SonarQube is an appropriate tool to manage the code quality during the program development lifecycle. The abilities of SonarQube such as checking duplication, unit testing, and finding potential bugs are useful for improving the performance of the program. As an open source platform, SonarQube supports several predefined rules to check the commonly occurred vulnerabilities and pitfalls such as misusing loops, inappropriate return values, and unnecessary branches. Besides, it is also available to add custom rules to check the particular types of shortcomings.

Our previous work named neoESPA is yet another evaluation system for programming assignment which has been served in undergraduate courses during the past two years. NeoESPA supports several programming languages such as C/C++, Java, Python, and Haskell. It provides a web front-end for students to submit their source code, then compiling the code in the back-end, and executing the compiled program with predefined inputs, at last, comparing the outputs of the program with criteria outputs to judge the correctness.

In this paper, we apply particular features of SonarQube on the neoESPA to provide a quality evaluation function for evaluating programming assignments. We utilize the vulnerability detection of SonarQube on neoESPA to help the students to correct their coding problems by cutting/adding scores and giving appropriate suggestions for their assignments based on the detected result. After analyzing the code submitted by students during the past two years, we synthesize the mistakes students make most frequently and build a rule set for quality detection. The rules we use in our system cover particularly all parts such as correctness of conditional expressions use, runtime error pre-checking, an appropriate loop ending, and required method use. We also design three scoring models to evaluate the assignments from different ways: one for calculating the score of correctness which based on the output of the compiled program; another one for assessing quality of the program by checking the rules; last one for merging the two scores from previous models to grade a final score each assignment. For measuring the score of quality more precisely, we give each rule with a weight to present its importance.

Students will not be able to get the full score if their codes have vulnerabilities. Otherwise, they will get more score even the result of their code is not fully correct if it satisfies all the rules. We believe that both situations can encourage the students to catch up a higher score by working hard on correcting their code.

Using our proposed work, it is more intuitive for instructors to grasp which part of programming training should be reinforced; it also can encourage students to have a good development style even from small programs such as homework.

Keywords:

Programming assignment evaluation, code quality detection, web based assessment system.