Abstract:

Listening comprehension is widely recognized as one of the most challenging skills in second language learning because learners must simultaneously process pronunciation, vocabulary, grammatical structures, and contextual cues in real time. To support comprehension, subtitles are frequently incorporated into multimedia learning materials. However, previous studies examining subtitle effectiveness have produced mixed findings. While some research suggests that subtitles help learners segment speech and connect spoken language with written text, other studies report limited improvement in listening performance. In addition, most existing research focuses primarily on comprehension outcomes rather than the processes through which learners interact with subtitles. Consequently, relatively little is known about how learners allocate visual attention to subtitles and video information during listening.

Eye-tracking technology provides an effective method for examining learners’ real-time viewing behavior in multimedia environments. Nevertheless, eye-tracking studies on subtitle use remain limited, particularly those comparing learners with different proficiency levels. Moreover, although many language-learning videos now provide bilingual subtitles displaying both first-language (L1) and second-language (L2) text simultaneously, empirical research on how learners utilize bilingual subtitles during listening comprehension is still scarce.

This study investigated how subtitle type influences learners’ comprehension, visual attention patterns, and perceived cognitive load when watching English news videos. Participants with two proficiency levels—Low Intermediate and High Intermediate—watched a five-minute CNN news video under one of three subtitle conditions: Chinese subtitles, English subtitles, or bilingual Chinese–English subtitles. After viewing the video, participants completed a multiple-choice comprehension test and a free recall task in their first language. They also responded to a cognitive load questionnaire adapted from Kruger, Hefer, and Matthew (2014). Eye-tracking technology was used to record gaze patterns during video viewing.

The results showed that the High Intermediate group significantly outperformed the Low Intermediate group across subtitle conditions (F = 11.416, p < .05). For Low Intermediate learners, comprehension performance was highest in the bilingual subtitle condition, followed by Chinese subtitles, while English subtitles produced the lowest scores. In contrast, High Intermediate learners performed best with Chinese subtitles, followed by bilingual subtitles, and least well with English subtitles. A two-way ANOVA revealed significant differences in comprehension among subtitle types (F = 3.520, p < .05).

Eye-tracking results further revealed clear differences in visual attention across proficiency levels. Higher-proficiency learners allocated more attention to the video area and showed shorter gaze durations on subtitles, whereas lower-proficiency learners displayed longer gaze durations on subtitles, indicating greater reliance on textual support. These findings highlight the interaction between subtitle type and learner proficiency and provide pedagogical implications for selecting appropriate subtitle formats in multimedia-assisted listening instruction.

Keywords:

Eye-tracking, visual attention, bilingual subtitles, listening comprehension, proficiency.