Abstract:

We present an innovative multilingual platform, CALL-SLT, for language learning based on a spoken translation game (Wang & Seneff, 2007), intended to help a second language (L2) learner to improve fluency. The system shows the learner sets of meanings presented in a L1 gloss form (for example, ASK-FOR POLITELY 1 BOTTLE WATER for English speakers) that have to be verbalized in the target L2 language. The possible meanings are extracted from a list of example sentences, defined by the teacher. They are organized by situation (e.g., in a restaurant) and can be structured into fine-grained lessons that pick out subsets of sentences based on predefined lexical, syntactic or semantic properties (Rayner et al., 2010a).

The system is based on two components: a grammar-based speech recognizer and an interlingua-based machine translation (MT) system, both developed using the Regulus platform (Rayner et al., 2006). In order to check whether the sentence pronounced by the learner is correct or not, the system first performs speech recognition. The MT system then determines if the recognized sentence corresponds to the meaning of the initial gloss. To do this, it transforms the sentence into the interlingua representation and matches it against the representation of the gloss. Depending on whether matching was successful or not, the level of difficulty for the next example is adjusted up or down. A help button allows the student, at any time, to access a correct sentence in both written and spoken form. A recent evaluation during an open day event for school students at the University of Geneva showed that more than half of the students were positive about the use of the tool for foreign language learning (Rayner et al., 2010b).

The present article describes a more elaborate evaluation exercise carried out with Arabic-speaking students learning French at the University of Al Ain, UAE. The French version of the system contains 21 lessons intended to teach fluency in a restaurant domain. Each of them focuses on a specific speech act (order a dish, book a table, ask for something, pay, ask where something is, etc.), a way of speaking (e.g. using conditional, future, yes-no question, infinitive, etc.) and a grammatical question (e.g. yes-no question, infinitive, numbers, official time). The experiment has been designed to evaluate the system’s efficacy for improving command of the domain language, measured using a knowledge test carried out before and after exposure to CALL-SLT.

References:

• A Multilingual Platform for Building Speech-Enabled Language Courses, Rayner M., Bouillon P., Tsourakis N., Gerlach J., Baur C., Georgescul M. and Nakao Y,. in: Proceedings of the L2WS Workshop, Tokyo, Japan, 2010a.
• A Multilingual CALL Game Based on Speech Translation, Rayner M., Bouillon P., Tsourakis N., Gerlach J., Georgescul M., Nakao Y., Baur C., in: Proceedings of LREC, Valetta, Malta, 2010b.
• Automatic assessment of student translations for foreign languages tutoring, Wang C. et Seneff S.., in: Proceedings of NAACL/HLT 2007, Rochester, 2007.
• Putting Linguistics into Speech Recognition, Rayner M., Hockey B.A. and Bouillon P., in: CSLI, Stanford, 2006.

Keywords:

Translation game, speech recognition, machine translation, evaluation, L2 learning, specific domain language learning.