Abstract:

The code review of a program is probably the most important part in quality assurance. Surprisingly, there is only little research on the improvement of this process and it is still unclear, how these skills can be taught to students in a more appropriate way. To get a more detailed insight into this process, eye tracking technology might provide a suitable research method. According to this idea, an eye tracking study is designed.

In its design, this study is a replica of Uwano, Nakamura, Monden and Matsumoto from the year 2006 [1]. The earlier results of the Japanese research group strongly imply that programmers are reviewing source codes in the programming language C in a certain way, resulting in particular eye movement patterns. These patterns become observable by using modern eye tracking technology.

The main intention of this study is to verify the described eye movement patterns which were originally discovered by Uwano et al. [1] and harness them for the teaching and learning of software engineering. Furthermore, it should be surveyed, if there are any experience-related changes in the eye movements of the participants while they are reviewing a piece of source code [2]. For this purpose, an eye tracking study is designed using six erroneous C-codes. A sample of 25 programmers (18 novices and 7 advanced) is recruited for this experiment. The recorded eye tracking data is supplemented by interviews and questionnaires, which give insight to the personal backgrounds of the participants and the cognitive strategies involved in code reviewing.

Due to the rather small sample size and the ongoing data collection, the results do not show statistical significant data yet, but they proof experience-related differences between both groups. The advanced participants outperform the novices in the case of error detection and can solve the task in a more elegant and efficient way. They need lesser and shorter fixations, which implies that they are able to grasp more information in a shorter time period. A possible explanation for this finding is the fact that all advanced programmers mentioned during their interviews that they are building mental models of the code and therefore they know exactly, what each line of code contains. In summary, it appears to be most likely, that programmers are using the concept of chunking, while they are reviewing source codes [3].

References:
[1] H. Uwano, M. Nakamura, A. Monden, and K. Matsumoto, “Analyzing individual performance of source code review using reviewers’ eye movements,” Proceedings of the 4th ACM Eye Tracking & Application Conference (ETRA), San Diego, CA, USA, 2006, pp. 133-140.
[2] K.A. Ericsson, R.T. Krampe, and C. Tesch-Römer. „The role of deliberate practice in the acquisisiton of expert performance”. Psychological Review, vol. 10, 3, pp. 363-406, 1993.
[3] S. Sonnentag, C. Niessen and J. Volmer, „Expertise in software design“, in The Cambridge handbook of expertise and expert performance, K. A. Ericsson, Ed., Cambridge, MA, USA, Cambridge University Press, 2006 pp. 373-388.

Keywords:

Eye Tracking, Software Engineering, Code Review, Expertise, teaching programming skills.