Abstract:

The use of virtual humans [1] as interlocutors in spoken dialogue systems (SDSs) enable opportunities for simulated spoken interaction in a low-anxiety environment [2]. We learn languages through interaction and, consequently, the lack of practice with native speakers and speaking anxiety are recognised as problematic in education. Research has shown positive effects when using SDSs in language learning, primarily in higher education, based on studies measuring learning outcomes in terms of motivation and L2 development (proficiency) [2]. However, few studies have explored these results by focusing on students’ self-reported experiences. This paper reports on a longitudinal study conducted with 14-year-old Swedish students (N=22), exploring their experiences of speaking English with embodied virtual humans in the AI-driven SDS, Enskill [3]. The study was designed to include ten speaking sessions over four months. Ratings (emojis and tags) and reflections were collected in a systematic quantitative way using a digital logbook (LoopMe). Additionally, three questionnaires (five-point Likert scale) were administered, one pre-trial (Q1) about demography, and two identical (Q2 and Q3) about speaking experiences (early and late). The findings from the ten speaking sessions showed a positive trend, consistent and sustained over time in these students’ self-reported experiences, even ending with higher ratings. The novelty effect wore off. The students reported being cognitively satisfied, emotionally engaged, safe, and able to relate socially to the virtual human. A positive association was identified between students with high ratings on the social dimension of the speaking activity and their overall experience of the system (emoji rating), although there was no established statistically significant difference. The students’ experiences of speaking with virtual humans were mostly tagged as easy (64%), fun (49%), feeling satisfied (22%), and frustrated (23%). Above all, explanatory reflections signalled that SDS constraints and communication breakdowns caused frustration. These results will contribute to deeper insights that could be useful for researchers in the field as well as stakeholders such as language teachers and developers of SDSs. Future work is suggested around the impact that students’ social relating with virtual humans plays on their experience of practising speaking with them in an SDS, and its eventual effect on learning.

Keywords:

Experience, foreign language (L2), longitudinal, speaking, virtual humans.