Abstract:

This PhD study focuses on the use of multimodal lecture screencasts for knowledge development in one Irish Higher Education institution. It extends an Inquiry Graphics (IG) Framework and proposes a Multimodal Inquiry Framework (MMI) to examine screencasts crafted by lecturers to explain core or threshold concepts, or ‘conceptual gateways’ (Meyer & Land, 2003) within their discipline. Inquiry Graphics is a theoretical and applied semiotic approach to communication and learning that places visual media at the centre of knowledge inquiry in teaching or research (Lacković, 2020). Multimodality is an approach that observes communication as including language but also encompassing other modes of communication, such as sound, image, touch, feeling, etc. (Jewitt et al., 2016). This proposed framework provides a lens to explore graphic-pictorial, linguistic, aural, and spatial-design modes and to analyse the semiotic organisation of the screencasts.

Qualitative interviews were conducted with HE lecturers, where the IG framework provided an opportunity to co-examine the underlying assumptions about how content is presented multimodally. An awareness of the semiotic dimensions of each mode was uncovered, in addition to the causal structures within the lecturers’ socio-cultural context, which influenced their decision-making processes.

The use of the MMI framework to analyse the screencasts revealed the semiotic purpose of the graphic-pictorial elements, primarily as unprobed representations of the chosen concept (Hallewell & Lackovic, 2017). Specific linguistic choices were revealed to conceptualise the concept, while prosodic features of the voice, along with music in some cases, were often used intentionally by the lecturer. The enactment of software features in the design of the screencast indicated lecturers’ embodied cognition of multimedia design principles and their digital fluency.

The MMI framework may be a helpful research tool for video and multimedia analysis to unpack the plurality of representations within multimodal screencasts. During the recent remote emergency online teaching period, an increasing number of educators curated and created multimodal resources such as screencasts (Belt & Lowenthal, 2021). However, the focus for professional development initiatives was primarily skills-based, rather than exploring teachers’ semiotic choices for the design of these videos, which could reveal implicit assumptions relating to knowledge creation and dissemination.
Finally, as a teaching tool, it could be used to help students ‘inquire’ the multimodal ensemble that is the screencast and develop their critical media literacy.

References:
[1] Belt, E. S., & Lowenthal, P. R. (2021). Video use in online and blended courses: a qualitative synthesis. Distance Education, 42(3), 410–440.
[2] Hallewell, M. J., & Lackovic, N. (2017). Do pictures ‘tell’ a thousand words in lectures? How lecturers vocalise photographs in their presentations. Higher Education Research and Development, 36(6), 1166–1180.
[3] Jewitt, C., Bezemer, J., & O’Halloran, K. (2016). Introducing Multimodality. In Introducing Multimodality. Routledge.
[4] Lacković, N. (2020). Inquiry Graphics in Higher Education: New Approaches to Knowledge, Learning and Methods with Images. Palgrave Macmillan.
[5] Meyer, J. H. F., & Land, R. (2003). Threshold Concepts and Troublesome Knowledge : linkages to ways of thinking and practising within the disciplines. 1–16.

Keywords:

Multimodality, screencasts, Edusemiotics, higher education.