Abstract:

In the deaf-mute education, the teaching method is single and the teaching quality is backward because of the deaf-mute's own factors, the sign language differences in different regions and the small number of teachers who are proficient in sign language. Therefore, it can not only strengthen classroom interaction, but also improve the teaching quality of the deaf-mute to convert sign language to speech for teachers to understand and speech to corresponding sign language for the deaf-mute to understand. In recent years, great changes have taken place in teaching methods and teaching environment because of the rapid development of artificial intelligence technology. The artificial intelligence technology provides a technical basis for the personalized and intelligent development of education. In this work, we proposed a method of sign language/speech conversion to realize the mutual conversion between sign language and speech in the classroom teaching of deaf-mutes, through the analysis of the deaf-mute teaching problems and the characteristics of classroom teaching. Firstly, a gesture dictionary, a gesture corpus with 6000 gestures and a text corpus with 5000 sentences are established. And a speech corpus is recorded. Then, we use pulse-coupled neural network to segment the gesture image to obtain the part of the gesture, and use the text label program to get the context-dependent labels information of the vowel and consonant of the text. Next, we get the text corresponding to the gesture by using the generative adversarial network, and speech synthesis using hybrid LSTM method. Meanwhile, we use the method based on long short-term memory network to realize the conversion of speech to text, and using the sign language dictionary to match the gesture corresponding to the keyword, and using the OpenGL graphics library function to play the gesture corresponding to the keyword after programming and smoothing. Finally, we realize the mutual conversion of sign language and speech. The experimental results show that the accuracy of sentences in speech to sign language conversion is 88.6%, and the degradation mean opinion score of sign language converted speech is 4.3, which has a good intelligibility. Furthermore, the experimental results are applied to the teaching practice of the deaf-mute to break the limitation that teachers who do not know sign language cannot teach the deaf-mute. The experimental results provide help for the interaction between teachers and students, teachers' flexible adjustment of teaching programs and the improvement of teaching quality of the deaf-mute. At the same time, the experimental results provide some support for the application of artificial intelligence technology in teaching, and promote the development of education in China.

Keywords:

Artificial intelligence, deaf-mute education, intelligent analysis, sign language, speech conversion.