Abstract:

This research aimed at investigating the usability of AgentSheets software by means of the Cognitive Dimensions questionnaire (Blackwell and Green, 2000). The subjects were 20 school age children attending the 5th year at a primary school in Cosenza (Italy); all the participants had sufficient skills in the understanding of English Language and in the use of multimedia tools. The research took place in two phases: in the first one, some theoretical information about AgentSheets software had been presented to the learning group (this authoring tool allows non-programmers to create simulations with agents, interactive games, virtual worlds, training tools, information gathering and personalizing agents’ behavior through a user-friendly interface). Then, children used the software to realize simulations with virtual agents, working individually.
In the second phase of the research, after a general explanation of the Cognitive Dimensions questionnaire (CDs) to the classroom, each subject was asked to individually compile it. The CDs questionnaire is divided into five main sections: Background Information, Definitions, Parts of Your Systems, Questions about the main notation, and Question about sub-devices. We analyzed the results concerning Viscosity (resistance to local change), Visibility and Juxtaposability (ability to view components easily and ability to place any two components side by side), Premature commitment (constraints on the order of doing things), Hard mental operations (high demand on cognitive resources), Role-expressiveness (the purpose of a component is readily inferred), and Error-proneness (notation invites mistakes).
The results show that, according the children, AgentSheets software does not exhibit a pronounced viscosity; it is “quite easy” (in a scale from “for nothing easy” to “easy”, “quite easy” and “very easy”) to make changes to the previous work, or rather only some goal-related operations required other no-goal-related individual actions. Moreover, all the subjects agreed that no particular changes were more difficult or especially difficult to make. Regarding visibility and juxtaposability, the majority of the subjects affirmed that it is “quite easy” to find the different components of the authoring tool as well as to see these parts at the same time to compare/combine them. Regarding the investigation of “premature commitment”, all the subjects agreed that the system does not force the users to defined paths, but allows to go about the job in any order subjects like.
The results also show that some hard mental operations for some tasks had been required, i.e. the programming of the virtual agents’ behaviour, whereas no mistakes seem particularly easy to make or irritating (error-proneness). According the majority of the subjects it is “easy” to tell what each part is for in the overall scheme and that there are some parts which are particularly difficult to interpret; furthermore, there are parts that children did not know what they mean, but that put them in (role- expressiveness). In conclusion, the AgentSheets software shows a good level of usability; in fact, the subjects confirmed that only in some cases their interaction with the tool interface was difficult. Hence, the visual management of the programming environment resulted quite easy, proving that is possible to completely exploit the potentialities provided by the software.

Keywords:

usability, cognitive dimensions questionnaire, educational software.