Abstract:

The development and increased accessibility of services such as ChatGPT, which provide access to advanced Large Language Models (LLMs) like GPT-4, is reshaping the landscape of educational assessment in K-12 education [1]. Historically, written examinations have been a fundamental method for assessing students' knowledge, with written language serving as a valid indicator of a student's understanding and thought processes [2]. However, the ease with which students can now utilize AI tools for assistance in generating text, challenges traditional assessment methods. There is a vivid debate among teacher concerning the extent to which technology functions as a support versus a replacement for actual performance.

This development has led educators to reconsider their assessment strategies to ensure academic integrity and the authenticity of students' work, reflecting a shift in the dynamics of educational assessment. This study investigates teachers' assumptions, expectations, and knowledge regarding LLMs and their potential influence on teaching and assessment. Our analysis adopts a socio-cognitive perspective, utilizing Orlikowski and Gash's [3] concept of technological frames to explore how LLMs might transform educational environments and assessment practices.

We report from 11 workshops with about 200 teachers from various schools and subjects in the west of Sweden. The workshop methodology was selected for its dual function as both professional development and a source of rich, authentic discussion [4]. The workshop format included demonstrations of ChatGPT, group discussions, and exercises in designing assessment scenarios both with and without the integration of LLMs. This paper primarily builds on the data from the group discussions where teachers deliberated on how students' use of LLMs affects the validity of traditional assessment practices.

Preliminary findings indicate a range of teacher perspectives, from concerns about academic integrity to curiosity about new pedagogical possibilities. Teachers report needing more controlled examination settings and express that they can no longer rely on texts produced outside their supervision as a valid source for summative assessment. Additionally, homework assignments no longer function as they used to. Several teachers also testify to an epistemological shift in that, due to technological advancements, they are trying to design assignments and tasks that they perceive systems like ChatGPT are incapable of handling. In this context, the theoretical perspective of technological frames can be used to illuminate how teachers' preconceptions and expectations of large language models create frameworks that shape their practice.

The study contributes to the emerging discourse on the role of advanced AI in education, highlighting the urgent need for empirical research and informed dialogue among educators, policymakers, and technologists.

Keywords:

Large Language Models, ChatGPT, Educational Assessment, Teacher Perspectives, Technological Frames, Swedish Education.