Abstract:

In our bachelor degree (Digital Arts and Entertainment) we have a lot of motivated and passionate students. However the workload is high and it can become difficult for students to meet all the requirements of the study program and maintain a healthy work/life balance. The study program is open to all students and as a publicly funded institute we often lack the means to adequately track students and help them to reach their goals.

As a part of the solution, we are developing a chatbot solution that uses a 3D chatbot to help with the coaching of students. The virtual avatar integrates with the learning management system and uses elements of cognitive based therapy to help students overcome typical problems that plague our students such as procrastination, lack of study planning and communicative problems (with peers and staff). The chatbot should be intelligent enough to forward students to the correct persons if the system cannot determine the correct course of action.

The development of a 3D chatbot poses many problems and contains many moving parts. The technical implementation has support for 3D animation, face and emotion recognition, language understanding, speech recognition and text to speech.

The chatbot uses a simple pattern recognition system as a first step to recognize the intent of the user and extract entities. If no pattern is recognized, the user input is handed over to a third party language understanding service (e.g. Microsoft LUIS). The intent is then processed by our system which delivers the correct response. The subjects of the discussion are maintained in a DAG (directed acyclic graph) to make it possible to maintain a working state of the conversation. It is therefore possible for the student to reference an earlier response of the chatbot.

For the speech recognition and text to speech we also rely on third party services (i.e. Google Cloud speech). For the response model a programming language was developed that makes it possible to gives responses to the user (student) via multiple channels. The primary channel is off course text, which can be converted to speech, but it also possible to for example start a video in response to a request of the user. The coach bot can also receive cues about which animation to play as part of a response, or for example the facial expression that should be assumed.

Finally, the 3D chatbot can be hosted into any number of platforms including mobile which makes it possible for students to access the chatbot whenever needed.

Keywords:

Coaching, chatbot, 3D.