Abstract:

As a lifelong neurodevelopmental disorder, Autism Spectrum Disorder (ASD) affects the brain and psychic-emotional system of children. Children with ASD encounter a lot of challenges in all kinds of learning. Some of them may suffer from developmental delays in several areas, such as cognition, social skills, motor skills, executive functioning skills, etc. (Shiri et al. 2018, Cai & Cao 2021). Depending on the seriousness level, children with ASD tend to have social difficulties. Many of them have difficulty in handling their daily life situations. Some of them may encounter difficulties in understanding or predicting the emotions of people around themselves, in communicating with others using non-verbal cues, and in expressing to people emotionally and socially (Marchi et al. 2018). Children with ASD suffer difficulties in communication and social interaction, some with attention abnormalities to process information in unusual ways (Cai et al. 2017, Lu et al. 2017).

Besides the difficulties of social skills and communicating skills, some children with ASD may encounter difficulty on learning or processing new knowledge, difficulty on retaining information in memory, difficulty on multi-tasking, etc. Children with ASD need a lot of attentions and cares from their guardians or parents. It brings heavy financial burdens and time to their families taking care of those children.

But those therapies or treatments may not be cheap or always available, with a lot of time and involvement of professional trainers or guardians. With the technology evolvement, the virtual reality (VR) and augmented reality (AR) have been introduced into the serious games for mimic treatments and training to children with ASD. The VR/AR serious games integrate the education and learning components provides a safe virtual training environment for children with ASD.

In our research projects, we work closely with special needs schools and special needs educators. A series of VR/AR serious games have been developed to train children with ASD on the social skills and living skills. The social skills include the communication skills, concentration/focus improvement skills, and emotion recognition skills. The living skills include shopping skills, school bag packing skills, personal hygiene skills, cross road junction skills, house cleaning skills, etc. These tasks sound easy to most children to perform independently, but may be tough for those children with ASD. In this paper, we will share our research experiences on the VR/AR serious games that we developed to train children with ASD in both social skills and living skills.

The research on VR/AR serious game for communication skills and emotion skills training include: virtual emotion learning serious game and virtual pink dolphin interaction serious game. The research work for living skills training include: virtual supermarket, virtual home, shower taking serious game, road crossing serious game, and school bag packing game. We will share more details in this paper about these research activities and experiments with special needs schools in Singapore.

References:
[1] Cai Y, Q Cao (2021). When VR Serious Games Meet Special Needs Education. Publisher: Springer Nature. eBook ISBN: 978-981-336-942-9.

Keywords:

Virtual Reality, Augmented Reality, Serious Games, Special Needs Education.