Abstract:

Although the teaching of handwriting is not compulsory in all countries, it is widely accepted that this activity highly improves young children’s personality, basic coordination abilities and communication skills. In this context, we present an intelligent tutor that evaluates not only the quality of the written symbol, but also the child’s personality and emotional state in order to adapt its teaching strategy.

In the first part of the paper, we propose a tool designed for automatic quality evaluation of handwritten symbols. We acquire the letters using a digital pen that transmits the space and time coordinates. We transform these coordinates in a binary image representation of the letter which is compared with the prototype letter. The evaluation module computes several parameters related to the legibility, size and space. An overall quality evaluation of the handwritten symbol is made.

Human communication is a combination of verbal and non-verbal interactions. Our intelligent tutor tries to follow the behavior of a teacher by assessing child’s facial expression and voice pattern. The pedagogical strategy is modified depending on the child’s interest in the handwritten application. Therefore, an important aspect of our intelligent tutoring system is the recognition of the child personality traits (based on a simple and intuitive personality test) and affective state (based on recorded speech and face information). The microphone and the camera of the system are used to collect speech and image signals.

The second part of the paper describes the approaches used to recognize child’s emotions using two modalities: recorded speech signals and face detected images. The first modality presents the features computed for the speech signal. The second modality describes the face landmarks considered for expression classification, including space coordinates for eyes, eyebrows and lips. Both proposed modalities use similar training methods, namely the multilayer perceptron neural network and the radial basis function network. Each modality identifies three types of emotions - positive, negative and neutral – and the final result is computed through decision fusion. Depending on the identified emotion and long term child’s attitude, a recommendation for a specific strategy is made. It is expected that the correct interpretation of the child’s affective state and the empathy with the animated tutor will encourage the child's interest in the calligraphy application.

Keywords:

Intelligent tutor, handwriting, emotion recognition.