Abstract:

Background:
At a time when markets are globalizing, second language acquisition is increasingly important in current society. More than a way to stand out, multilingualism has become a necessity, especially in multilingual countries such as Switzerland.
Like many aspects of the day to day life, language learning is being revolutionized by technological progress. Today, CALL (Computer-assisted language learning) software allows users to acquire a new language at their own pace, without any constraints of time and place. Thanks to the great evolution of speech recognition technology, it is now possible to interact orally with computers. On this basis, the Faculty of Translation and Interpreting of Geneva University developed its own software called CALL-SLT, a conversation partner exclusively based on speech recognition. Several experiments on the platform showed that speech recognition allowed users not only to learn grammar and vocabulary in second languages, but also to practice oral production and to improve pronunciation skills.

Objective:
The usefulness of CALL-SLT as a learning coach has already been proven. The aim of the present study is to see if it can be turned into an evaluator. Since the particularity of CALL-SLT is that users can only interact orally with the system, we decided to focus on pronunciation.

Our experiment has three major aims:
1. to discover whether speech recognition technology can assess pronunciation;
2. to determine whether automatic evaluation can be as reliable as human evaluation;
3. to see whether it is possible to automatically attribute a language level to each candidate.

Methods:
In collaboration with Intercountry, a French company specializing in teaching engineering, we developed a test designed to measure the pronunciation in English of French speakers. The experiment was conducted in two steps: first, candidates took the test and were assessed by CALL-SLT; next, their data were collected and assessed by an Intercountry English teacher. We then compared the results of both types of evaluation to see if automatic evaluation can match human evaluation. A total of 442 data from 17 participants were analyzed.

Conclusion:
The results of this experiment are very encouraging. Although the evaluation made by CALL-SLT does not perfectly match the evaluation made by the native speaker, we found out that automatic evaluation of the pronunciation by CALL-SLT is actually possible. Moreover, the speech recognizer and the severity of the automatic evaluation can still be adjusted in order to agree more closely with the human evaluation. Finally, we noticed a correlation between the score obtained on CALL-SLT by the candidates and their language level, which suggests that an automatic level attribution is also practicable.

References:
[1] http://callslt.unige.ch/
[2] BAUR, Claudia (2015): The Potential of Interactive Speech-Enabled CALL in the Swiss Education System: A Large-Scale Experiment on the Basis of English CALL-SLT. University of Geneva. Thesis.
[3] DEJOS, M, PETROVIC, C., BOUILLON, P., GERLACH, J., DUVAL, H., RAYNER, E., TSOURAKIS, N. (2016) : CALL-SLT: a first experiment in a real FFL training for employees of French companies. In : ICT for Language Learning. Florence. 
[4] PETROVIC, C. (2016): La reconnaissance vocale pour apprendre une langue étrangère : Expériences avec CALL SLT en collaboration avec Intercountry. University of Geneva. Master’s thesis.

Keywords:

Pronunciation, L2, CALL, CAPT, CALL-SLT, speech recognition, automatic evaluation.