Abstract:

Motivation:
The proposal of the national "One Belt and One Road" strategy has gradually strengthened the connection between the central Asian countries and China, and many Chinese universities have Donggan students who come to study and exchange. Due to language differences and the lack of teachers proficient in Donggan, there are many communication barriers in classroom teaching. To solve the problem that teachers fail to understand students' ideas, we propose a Donggan speech recognition method for Donggan students' classroom teaching.

Method:
This paper proposed a speech recognition method based on a transfer learning strategy to guide the training of the low-resource Donggan language model and acoustic model by learning resource-rich source domain knowledge. Due to the high similarity in language composition between Donggan and Mandarin, when the target task is Donggan speech recognition, we choose Mandarin speech recognition with abundant corpus as the basic task to construct the corresponding Donggan speech recognition system through transfer learning. In the model training stage, we first keep the original network structure the same, and only the output layer of the network is replaced based on the fine-tuning method. Then we train the whole model with a small learning rate for the target domain task. Finally, we adopt the transfer learning strategy on the pre-training model to expand the training data in the target field and improve the model's generalization ability.

Result:
In the experiment, we constructed a corpus that includes 4,616 sentences, lasting 6 hours, recorded by five male native speakers of Donggan, each of whom recorded 923 sentences. The training set, verification set, and test set of each experiment are carried out in a ratio of 8:1:1. Experimental results show that the proposed transfer learning strategy for Donggan speech recognition reduces the word error rate to 25.8%.

Conclusion:
In the paper, we proposed a low-resource Donggan speech recognition method by transfer learning for the Donggan students learning in China. The established Donggan speech recognition system can timely feedback on students' needs and their questions about classroom contents to teachers through text, thus improving the communication between teachers and students in the class.

Keywords:

Second-language learning, teacher-student interaction, speech recognition, transfer learning, deep learning.