Abstract:

Mixed Reality (MR) is a space where real and virtual environments are merged through the user’s perceived coexistence of real and virtual objects, where the user can interact in real time. It is currently an emerging technological approach that can be adopted for training purposes, since it reinforces sense of immersion and increases engagement.

There are various MR devices with hardware limitations and strengths, which may impact on the feasibility of MR training applications in specific domains. HoloLens2 is currently one of the best wearable devices on the market for its peculiar features, that provide optimal immersive hands-free experience of interaction. For this reason, it has been selected as the main device for MR provision in the context of the MARHVEL research project. This project is about the assessment of use of hyper video and MR for procedural learning.

In a first phase, a technical review and a feasibility study were conducted with the objective to develop a real case training. A prototype has been implemented with the goal to understand potentials and issues of HoloLens 2 for the development of mixed reality application in the vocational education field.

As a case study, the chosen educational activity consists of a computer assembly procedure; the application guides the user in building a computer through visual and audio feedback, explaining where and how components should be installed. For the experiment, a real motherboard and four virtual components (ram, processor, fan and graphic card) were used.

The application presents two simulations with different degrees of freedom. The learner is guided step by step to place the four virtual components in the corresponding sockets, following a predefined order, as a beginner, or in any order, as an advanced user.

For this work the Unity development environment, MRTK and Vuforia Engine have been used to produce the application running in the HoloLens2 device.
The results emerged during the development and preliminary testing of this application with a sample of users allow a number of considerations to be drawn about potentials and issues of HoloLens 2 for learning a procedure.

Among the main arisen strengths, the learners appreciated the learning by doing approach realized through the hands-on training, the self-learning immersive and engaging experience for learning a procedure. In addition, the use of virtual objects reduced both the training costs and the risks of their potentially dangerous manipulation.

Some issues were also revealed during the implementation and testing of the application. Light has emerged to be a crucial factor to guarantee proper object recognition and tracking. In addition, since the computer assembly procedure involves direct manipulation, other critical factors include the size of the objects that need to be manipulated as well their distance from the viewer’s eyes.

Another issue emerged during the testing phase with real users concerns the HoloLens2 direct manipulation input model: while in general gestures are intuitive, the pinch gesture for grasping big virtual objects, has been judged non-intuitive, because the fingers appear to pass through the objects themselves.

The results of this case study represent a first step to confirm the feasibility of future marker-less applications for the Marhvel project and are useful to verify in practice the specifications and recommendations declared by the HoloLens2 guidelines.

Keywords:

MR, Mixed Reality, HoloLens2, training.