Abstract:

The introduction of generative artificial intelligence (AI) technologies into widespread use from November of 2022 has created disruption in education. Tools such as ChatGPT enable generation of new content through dialogue with users, are used by higher education students increasingly frequently (Lo, 2023). Current discourse around the use of AI by higher educational institutions (HEIs) has concentrated on potential threats or on benefits of this technology (Dai et al., 2023). Due to recent widespread use empirical work on how AI is affecting teaching and learning innovations in specific institutional and disciplinary context is only starting to emerge.
The case study offered in this poster presentation investigates redesign of learning and assessment material on a large purely online module in Business Management at a UK university. The module offers introduction to management studies and follows an open-access admissions policy, attracting a heterogenous cohort of more than 3,000 students annually.
Students are provided with access to module material, synchronous and asynchronous tools and interactions and support from their tutor, who provides academic guidance to learners and marks student assignments. The module is assessed through a mix of formative and summative pieces of written assessment, each tapping into a different business or management topic.
From spring of 2023 the module team has been noticing novel trends in student work, which suggested use of generative AI, such as:

a) Use of theories or frameworks not covered on the module.
b) Use of non-existent reference sources.
c) Use of generic assertions in cases where more detailed treatment of the topic and associated case material was required.
However, in the absence of tools that can detect the use of AI, raising plagiarism cases proved difficult, since the extent and nature of AI use could not be established.

To counter the negative effects of students’ using AI. a significant redesign of assessment tasks is offered along the following lies:
a) Modifying written assessment tasks – e.g., writing bespoke case study material that reflect current trends and developments and are beyond the scope of AI remit.
b) Shifting emphases in written assessment tasks – e.g., allocating more marks for analytical skills compared to description and regurgitation of module material, which might include critical engagement with AI-generated content.
c) Diversifying assessment – giving greater salience to collaborative interactions and corresponding assessment tasks.
d) Introducing oral assessment tasks – allocating marks to oral presentation within tutor groups.

This poster presentation will seek to provide a more comprehensive overview of changes to assessment that are due to be introduced from 2024. During the conference feedback on innovative uses of AI to enhance assessment from educators in other institutional contexts will be sought with a view of developing a clearer perspective on the opportunities that the introduction of these technologies brings to large-scale, open-access online learning settings.

References:
[1] Dai, Y., Liu, A., Lim, C.P. (2023). Reconceptualizing ChatGPT and generative AI as a student-driven innovation in higher education. In Proceedings of the 33rd CIRP Design Conference, Sydney, Australia, 17–19 May 2023
[2] Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410.

Keywords:

Generative AI, ChatGPT, online assessment, assessment redesign, large-scale online courses, open-access online courses.