Abstract:

For many higher education practitioners, one of the most daunting challenges is providing meaningful and precise feedback on long-format written assignments, such as essays, papers, and reports. Such writing tasks are integral to higher education, as they simultaneously sharpen students’ higher-order cognitive skills (analysis, argumentation, synthesis, creation) and students’ ability to communicate their ideas cogently in an academically phrased and well-composed text. However, teachers in higher education are often confronted with large cohorts of students, which serves to curb their inclination to assign such writing tasks, crucial as educators know they are. The provision of meaningful feedback would require an investment of time that is simply unavailable, and without meaningful feedback, the writing task would do little to further student academic development.

This paper seeks to explore and assess the extent to which Generative Pretrained Transformers (GPTs), of which ChatGPT is the most well-known example, can aid teachers in providing feedback on complex and long-format academic writing tasks. Since November 2023, users can create a dedicated GPT agent using purely verbal prompts, making personalized design and deployment of neural network technology, at least in theory, within reach of any user.

This exploratory study is structured as follows:
1) gathering relevant prompt engineering examples from early adopters experimenting with GPTs as feedback copilots;
2) developing a dedicated feedback GPT for a scientific paper assignment that is part of the second bachelor of the joint program for Industrial Engineering Technology at the University of Leuven and Hasselt University (Belgium);
3) iteratively ameliorating the GPT and documenting the build process; and
4) comparing the quality, depth, and correctness of feedback provided by the GPT to that of the teacher for 20 student papers, using a rubric with evaluation criteria including content (veracity, logic, argumentation, use of source materials), structure on the text and paragraph level, phrasing, grammar, spelling, and adherence to academic formatting norms.

This paper contributes insights into the usability and potential of dedicated GPT feedback assistants for academic writing assignments. Furthermore, it offers a model for practitioners aspiring to experiment with neural network technology for feedback.

Keywords:

Academic writing, feedback, AI, neural networks, GPTs.