Abstract:

This eye tracking study investigates the use of heuristics in one of the most challenging parts in modern software engineering: the generation of an object model for a software product based on a requirements specification. During their training, software engineers are taught different techniques to solve this task. One common technique is the noun/verb analysis. Despite teaching this approach, it is still unclear to what extent novice and expert programmers are making use of it. In the best possible scenario, the noun/verb analysis works as a heuristic and helps programmers to make fast and accurate decisions. To survey this topic, an eye tracking study is conducted.

Participants in the study are 40 software programmers at four levels of expertise (novices, intermediates, experienced programmers, experts). They have to solve ten decision tasks. In each task, participants read a requirement specification and then have to choose one out of three presented class diagrams that they considered the best solution. During the whole task, their eye movements are recorded.

The results show that participants of all expertise levels use the noun/verb analysis. To confirm this assumption, every single noun and verb is marked as an area of interest and the individual dwell time for each one is calculated. The dwell time differs for all groups, whereby the more experienced programmers perform faster. But if the dwell time is converted to percentages, it can be seen that all participants are investing the same time proportion in gazing at nouns (around 40%) and verbs (8.5%). Furthermore, the performance differs between groups: the more experienced programmers tend to perform better in case of choosing the UML diagrams (F(df3)= 2.71; p=.060). Both experienced and expert programmers are not following the noun/verb analysis blindly. Rather they use their previous gained knowledge additionally. In this case, they cease making use of this heuristic and start to use their previous knowledge and experience. This results in a more logical way of modelling.

It can be concluded that all participants are making use of the noun/verb analysis, as confirmed by reliable eye tracking data. The amount of time, which is invested in gazing at nouns and verbs, is almost the same for all groups. The main difference between the groups is that experience has strong impact on the individual modelling approach. Novices do not have available well-structured and broad previous knowledge on modelling tasks. Therefore, they hold on using the technique, which they have learnt in their training. Experts use this technique, too, but they realize that they have to build up the model in a more logical way.

Taken together, it can be stated that the noun/verb analysis is a valuable and important heuristic in software engineering and programming. Its usage depends on the level of expertise of the programmers, however. Novices use this technique frequently, while experts use it implicitly in a cognitive way.

For the teaching of software engineering, it can be concluded that it might be important to teach students that this technique helps to find class candidates but still to consider previous knowledge respectively domain knowledge. They should be encouraged to model in a more flexible and logical way.

Keywords:

Eye tracking, heuristic, software engineering engineering, software modelling, expertise.