Abstract:

This paper focuses on the integration of experimental research and teaching in the field of multimodal linguistics in Italian universities. The research general goal was to explore the meaning-making potential of hand movements in the TV series House MD.

The project started at the University of Messina (Arizzi, forthcoming) and was realized thanks to MWSWeb, a specifically designed online platform collecting the corpus of all episodes of House MD (Taibi et al. 2019). Students from several Italian universities virtually cooperated annotating the films on the platform.

A preliminary ‘hands model’ was developed and embedded in MWSWeb. Subsequently, each student was asked to apply the model to a significant sample of scenes paying attention to recurrent meaning-making patterns in the overall visual/verbal construction of the various episodes arising from the distribution between oral discourse and hand movements, including cases of co-occurrence as well as alternating and complementary distributions. The goal was to reveal distinctive conjunctive and disjunctive patterns of meaning-making that highlight the way in which hands accompany and integrate with oral discourse.

The students were virtually trained and provided with guidelines and detailed examples. Moreover, during the annotation process, they kept a diary to take notes of doubts, impressions, difficult decisions. There was a constant e-tutoring and randomized double checking.

The methodology is based on multimodal corpus linguistics (Baldry, Thibault 2006). MWSWeb allowed students to retrieve scenes in a way that shows side-by side-transcript and video. The annotation boosted students linguistic and interpretative skills, thus structuring their intercultural digital literacy (Sindoni et al. 2019). The students enjoyed their involvement in the project and showed solid critical thinking and awareness of how different semiotic resources integrate.
The paper concludes with some suggestions on the project’s expansion potential involving students from different countries, thus extending the reflection on meaning-making resources to different cultural backgrounds.

References:
[1] Arizzi, C. (Forthcoming). Exploring the narrative functions of hand movements in the Teaser phase in House MD. A corpus-assisted analysis. Proceedings of the XXIX AIA conference “Thinking out of the box in language, literature, cultural and translation Studies: Questioning assumptions, debunking myths, trespassing boundaries. Padova University Press
[2] Baldry A., Thibault P. (2006), Multimodal Corpus Linguistics, in G. THOMPSON, S. HUNSTON (eds.), System and Corpus. Exploring Connections, London, Equinox, pp. 164-183.
[3] Sindoni, M.G, Adami, E., Karatza, S., Marenzi, I., Moschini, I., Petroni, S. & Rocca, M. (2019). The Common Framework of Reference for Intercultural Digital Literacies. A Comprehensive Set of Guidelines of Proficiency and Intercultural Awareness in Multimodal Digital Literacies. cfridil-framework-linked-fin1.pdf (eumade4ll.eu).
[4] Taibi, D., Marenzi, I. & Ahmad, Q.A.I. (2019). Ain’t that sweet. Reflections on scene level indexing and annotation in the House Corpus Project. In A.P., Baldry, F., Bianchi & A. Loiacono (Eds.). Representing and Redefining Specialised Knowledge: Medical Discourse. Linguaggi specialistici e traduzione tecnica, 1, 151-181.

Keywords:

Online annotation, multimodal linguistics, meaning-making.