Abstract:

During the last decades, software is gaining more and more in importance and it already is an essential part of our daily life. New inventions (e.g. autonomous vehicle) rely heavily on sophisticated programs and people are willing to entrust their lives in this kind of technology. So, it is crucial to get a deeper understanding for the cognitive processes involved in programming software.

The current study is a supplement to the findings of Nivala et al. [1]. This work used eye tracking technology to get insight to the development of visual expertise in software engineering.
Based on the definition of “expert”, it must be mentioned that there are only novices and advanced programmers as participants in this study but no real experts. An expert normally is a person, who has a long experience in a specific domain. Their skills are a result of deliberate practice, a training that is focused on a constant improvement of certain skills [2]. The sample contained 9 advanced programmers and 15 novices. All of them were familiar with C programming language.
Due to the technical limitations of the analysis software, its first results are based on a very preliminary analysis. These clearly showed that advanced participants performed better in error detection and in understanding the code. In contrast, the novices had to struggle with the error detection, although they spent more time in scanning the code.

The aim of this study is to do a more in-depth analysis to get further information about the individual code reading behavior. The tasks 2 and 5 are selected due to their high statistical standard deviation and divided in a reading and response phase. Task 2 is correct and produces a line of numbers as an output. Task 5 is an erroneous code, which does not produce any output.
The more advanced programmers perform way better when the source code is incorrect and does not produce an output (BP = .417, p = .048). They are faster in the reading phase (BP = -.566, p = .004) and need a fewer number of fixations (BP = -.413, p = .045). In the response phase, there are no significant differences between the two groups.
When the source code is completely correct, as shown in task 2, there is no statistical significant difference between the novices and the advanced programmers in both phases.

The collected data and the results are in line with the findings of the expertise research. Experts are due to their training very good in problem solving in their domain. Furthermore, they act in a qualified and efficient way [2], [3]. This is shown in task 5, where the error was unspecific and not obvious to the less experienced participants while it was quite clear that task 2 will produce an output.
The findings of this study will be used to identify difficult educational content in C programming courses and to do respective modifications in teaching.

References:
[1] M. Nivala, F. Hauser, J. Mottok, and H. Gruber, “Developing visual expertise in software engineering: An eye tracking study,” IEEE Global Engineering Education Conference, EDUCON, pp. 613-620, 2016.
[2] K. A. Ericsson, R. T. Krampe, and C.T. Tesch-Römer, “The role of deliberate practice in the acquisition of expert performance,” Psychological Review, vol. 100, no. 3, pp. 363- 406, 1993.
[3] K.A. Ericsson, “Expertise and individual differences: the search for the structure and acquisition of expert’s superior performance,” Wiley Interdisciplinary Reviews: Cognitive Science, vol 8, no. 1, pp. 1-6.

Keywords:

Eye Tracking, Expertise Research, Software Engineering, Programming, Instructional Design, Research Methodologies, Advanced Classroom Technology.