Abstract:

Teaching programming is an activity receiving significant attention nowadays. Assessment of students’ knowledge in introductory programming courses can (partly) be done by presenting simple source code snippets (fragments). Students should be able to correctly answer the question: “What is the output of the given code?” When preparing the code snippets, teachers must be aware of the code’s complexity. Particularly, when preparing many different versions of the same test (to assess a huge number of students), they should try to provide same or similar complexity tasks for all students. A possible solution to this problem is to turn to automatic generation of questions containing source code snippets.

In order to attain complexity consistency in the process of automatic production of questions for programming courses, there needs to be a way to automatically measure the complexity of source codes. In this paper we describe a source code metric that we have defined and that considers the source code complexity from a perspective of the student’s effort required for manual calculation of the program output, if the input is known. The metric measures the complexity using weight values assigned to each of the operators and branch statements in the code. We also describe our findings from the research that we have conducted in order to determine appropriate weight values for the arithmetic operators. The main goal of this research was to improve the accuracy of the code complexity calculation for automatically generated tasks that contain source codes, by determining weight values for basic arithmetic operations with simplest operand types. We also provide remarks for the future experiments that we plan to conduct on the subject.

Keywords:

Source code complexity metric, automatic generation of tasks containing source codes, assessment in introductory programming courses.