Abstract:

Programming courses are an important part of computer engineering, science, and IT curricula. It is widely accepted that computer programming is difficult to learn for novice students. Moreover, programming requires students to have analytic skills to generate solutions for a given problem. There has been much research done on measuring the quality of student code and student programming ability to overcome difficulties in learning programming. However the mixture of such results varies from one academic environment to another due to different cohorts of students, cultural settings, and the education systems. Moreover, relatively few studies have yet examined how a student arrives at a solution on complex assignment or how students’ programming process differed from experts on a given programming assignment. Thus, this study is set to discover differences in the process of composing a solution to a simple problem between professional and novice programmers.

To identify the differences, we utilized the ViLLE learning environment on an introductory programming course to collect data that includes complete programming solutions and a number of submissions to a “Robot exercise”. To mitigate issues with student plagiarism, students received their coding exercise from a predefined pool, where each exercise had a similar complexity and a common goal. This study analyzed submissions from 197 students and 3 experts. Submissions which could not be compiled or whose execution never finished (e.g. infinite loops) were not included in the study. We used both quantitative and qualitative analysis techniques to examine and compare the static quality of the code solution between students and experts.

The analysis protocols of this study are partly derived from the ISO/IEC 9216 quality framework to analyze the static quality of source code. In quantitative analysis we found that both the experts and novice programming students used similar programming features. However, less than 7% of students only used a brute force algorithm, whereas the rest behaved similarly to the experts to solve the problem. Where the students and experts differed was found to be in the process of composing the solution. Notably, it was found that experts were very efficient in abstracting the sub problems in order to generate a solution to a given task. The students struggled to identify the core of the problem from its description. Regardless, there were not many differences between the static quality of programming solutions produced by students and experts. The differences were found in how experts and students apply reusability and portability techniques such as defining methods to develop coding solutions.