Abstract:

This paper describes a comparison, carried out within a Swiss funded research project, among three different technological approaches to support procedural learning: wearable Mixed Reality (MR), handheld MR, and PC interactive video. An adequate case study for the comparison of the three approaches was the Tangram game, being simple and complex at the same time. A specific application to solve the tangram puzzle was developed in 3 versions: for HoloLens 2, for Android smartphones and for PC as an interactive video. Each implementation of this Tangram MR application supports the learner to solve the tangram through the same 4 steps with incremental levels of scaffolding. To help users get familiar with the different devices and interaction modalities, a short tutorial was also created to be shown before using the app.

A usability expert review was then conducted involving 7 experts, with the objective to identify usability issues, collect suggestions for improvements, and compare the different devices-apps. The procedure followed during the evaluation consisted of these phases: an initial presentation of the project and testing objectives by the project team; a practical test session individually conducted by each expert, using the 3 devices-apps; a final discussion with the experts where the 3 devices-apps were compared on the basis of different recognized user experience criteria, including effectiveness, efficiency, satisfaction, attractiveness, motivation of use, sense of immersion, cognitive load, physical effort, etc.

The expert evaluation highlighted several issues and provided some useful recommendations. The most interesting remark was the need to take into greater consideration the affordances of each device, in particular the in-time and in-space distinctive feature of Mixed Reality. Some notable results also emerged in the final discussion include: both effectiveness and efficiency were judged medium-high or high for the three conditions, while the greatest satisfaction was got by the HoloLens, probably due to the "novelty" effect. HoloLens presented higher attractiveness, motivation of use, sense of immersion, but also implies a greater cognitive load compared to the other devices. The physical effort was very high for Handheld devices and very low in the interactive video. Learnability was higher for handheld compared to the other devices.

In conclusion, none of the approaches was considered the best or the worst. It depends on the experience intended to be created: wearable MR on HoloLens 2 is considered a comfortable and high-quality device for immersive experience, but it requires a training phase for novel users; Handheld MR on well-known easy to use devices is appreciated, but heavy to hold for a long time, while computer with interactive video is easy to access, with an intuitive interaction, but not very attractive.

Keywords:

Mixed Reality, Wearable, Handheld, Interactive Video, HoloLens 2, Usability Evaluation.