Abstract:

This paper explores the use of virtual environments for the purpose of training visually impaired individuals in the use of the Sound of Vision sensory substitution solution. Sound of Vision is a wearable device which scans the environment using various cameras and converts the extracted information into audio and haptic stimuli. Proficiency with this advanced sensory substitution solution comes through training.

Part of the Sound of Vision training strategy is based on a set of gamified virtual environments, as a safe, cost-effective, motivating and efficient tool. Most serious games are specifically designed for training or educating users, rather than entertainment, and in most cases, users play them out of necessity to gain a practical skill set - which is also the case for our visually impaired users. It includes a set of realistic environments that expose the user to daily tasks, a fantasy environment which exposes to similar tasks but with a fantasy storyline and a large number of mini-games, each focused on developing a specific skill. This paper focuses on these mini-games contained in the Sound of Vision Virtual Training Environments, the skills they teach the user and the ways they contribute to the serious game by providing engagement, entertainment and challenge. Each mini-game provides an environment which favors various essential skills such as navigating a 3D environment, detecting an object's location or differentiating between two or more sources of information.

Among the mini-games, we mention:
Tunnel – places the user in a long tunnel that can have multiple 90 degree turns. The task given is to reach the end of the tunnel.
Pickups - tasks the player with going to different locations and collecting pickups. When he reaches the designated location, the current pickup is collected and the next one appears.
Slalom - the player has to traverse an area where multiple obstacles are placed randomly, without touching them. This challenge is designed to train 3D navigation on a more advanced level than Tunnel.
Asteroids - places the player in a fixed position inside an empty space where asteroids appear around them and home in on the user’s location. They have to shoot all the incoming asteroids before they are hit by one.
Catcher - the mini-game limits the player’s movement to one axis (left-right) and creates three lanes in front of them. Pickups are spawned on each lane randomly and head towards the player’s side. The user must catch as many of them as possible by moving left and right between the lanes.
Maze - the player is placed inside a maze and must find a path that leads to a checkpoint to advance to the next level. At lower levels of difficulty all the paths lead to a checkpoint, while at higher levels only one path is correct and the user must find it by trial and error.
Decoys - places the player in a square room. In the room, initially, there is only one pickup. When the player reaches it, another one is spawned, together with a decoy. The player must only collect the pickup.
These mini-games can provide a valuable learning environment for visually impaired users and contribute to the overall effectiveness of the training program designed for a Sensory Substitution Device such as Sound of Vision.