Abstract:

Spoken dialogue systems (SDS) (Bibauw et al., 2019) enable the possibility of practising foreign language speaking skills in interaction with embodied virtual humans as native speakers in simulated everyday life situations. SDS within intelligent computer-assisted language learning (ICALL) is an emerging research field with studies mostly measuring effect (ibid). Instead, by taking a student perspective and using a scientific social media platform-based digital logbook (Lackéus, 2020; LoopMe, 2021) as a research method, we can learn more about students’ experiences in the SDS. This paper aims to show how this digital logbook was used in a study that longitudinally investigated 14-year-old Swedish students’ (N=21) experiences of practising speaking English in a selected SDS in an educational context. The digital logbook produces real-time data about the students’ self-reported experiences in the SDS over time to better understand the learning situation. Some snapshots of data produced in the students’ digital logbooks of the conducted study will illustrate what kind of data this instrument can provide.

The logbook contributed with data collected instantly after each speaking session (N=10) in the SDS during the study. The students systematically reflected in four repeated open-ended items and rated their experiences of the speaking session in the SDS by selecting a suitable emoji and some tags. The emojis are smiley symbols representing a five-graded Likert scale (rating -2 to 2), measuring the overall experience. The battery of eligible tags is word labels predefined by the researcher to collect data about the cognitive, emotional and social dimensions of the students’ experiences. The data produced in the digital logbook are analysed through descriptive statistics presented with graphs and tables generated in its platform. Data are analysed both on an individual and a group level. Snapshots of data collected reveal, for instance, that the most frequently used tags of all students’ total usage were easy (64%), fun (49%), having asked questions (56%) and answered questions (54%) in spoken interaction with virtual humans. The students’ mean value of their self-reported overall experience in the SDS rated through emojis was higher towards the end of the study. However, some temporary declines were identified along the way. The students’ longitudinally rated overall experience were then further analysed through equivalent open-item reflections reporting on, for example, experienced constraints in the SDS, causing frustration. Applying this kind of digital logbook as an instrument for collecting data demonstrates how we can tap into the students’ perspectives by quantifying self-reported qualitative longitudinal data and combining them to deepen our understanding of the studied learning situation.

References:
[1] Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: research synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning. DOI: 10.1080/09588221.2018.153550
[2] Lackéus, M. (2020). Collecting digital research data through social media platforms: can ‘scientific social media’ disrupt entrepreneurship research methods? In W. B. Gartner & B. Teague (Eds.), Research Handbook of Entrepreneurial Behavior, Practice, and Process Cheltenham, UK: Edward Elgar Publishing
[3] LoopMe. (2021). Lead and follow learning and development. Retrieved from www.loopme.io/

Keywords:

Social media platform-based digital logbook, Spoken dialogue system (SDS), ICALL, self-reported experiences.