Abstract:

Robotics is one of the main driving and integrative technologies that will ultimately lead to advanced cognitive artefacts and new autonomous devices. Robots co-existing with children in domestic and public environments are expected to behave as companions and tutors, also engaging in playful interactions. In this paper, we describe an on-going project that lies at the intersection of service robotics, entertainment robotics, human-robot interaction, and deep learning, and contributes to the goal of introducing robots into everyday use to the benefit of human users by providing companionship, education, dependable and acceptable support, and entertainment for children at preschool and primary school levels.

The system is based on a “modularity in design” approach and has three main modules (M): M1 – emotion recognition for interactive storytelling; M2 – interactive writing and drawing, M3 – training a humanoid robot to play multimodal games. Three use case scenarios along with expected educational benefits (see below a brief description) are described with full detail.
Use-case scenarios are the following:
1. Children in a kindergarten learn English through interactive storytelling by a humanoid robot; they listen stories from a robot and establish dialogues with the robot;
2. A humanoid robot teaches a child how to write and draw text/figures;
3. Humanoid robots play multimodal games (Tic-Tac-Toe) with children.
Expected benefits:
1. Efficient constructive educational activity. The children will be actively involved in the process of knowledge construction. Enhanced creativity.
2. Engaging educational activity. Feedback on the task performance. Monitoring children’s progress.
3. High levels of children engagement. Enhanced cognitive and social abilities.

Our project goes beyond the international state-of-the-art on four directions:
1. We extend the typical focus on one-directional, one-on-one interaction between people and robots, as we take into consideration interactions between multiple children and multiple robots.
2. We extend the focus on sensorimotor interaction by considering emotions as key factors in multimodal (integrated) children-human robot interaction, simultaneously through text, audio, and images;
3. We extend the deep learning applications in endowing social robots with the possibility of acquiring skills automatically, such as learning to play multimodal games. More, we use emotion detection for extending the robot behaviour from simply winning all the games to make the players (children) as happy as possible;
4. We design an intelligent planner for coordinating the multiple roles of the robots interacting with multiple users, feedback and progress monitoring, for the set of the three proposed use-case scenarios, using a specification-based approach.

The paper concludes with avenues for further developments.

Keywords:

Humanoid robotics, child-robot interaction, emotion detection, interactive storytelling, multimodal games, deep learning.