Abstract:

This work addresses a persistent challenge in introductory programming courses across engineering degrees: the high demand of manual grading, which often leads to delays, inconsistency, and limited opportunities for meaningful feedback. These issues reduce students’ ability to self-regulate their learning and generate frustration during their first encounters with programming.

Building on a previous technical validation where the JorGPT system achieved near-perfect alignment with human grades (R² ≈ 0.9994), this study presents the pedagogical evolution of the project. The focus now shifts from replicating instructor evaluations to understanding how Large Language Model-generated feedback can enhance learning, promote metacognitive engagement, and improve students’ confidence and curiosity while solving programming problems.

The intervention is implemented in the first-year course Introduction to Programming at Universidad Francisco de Vitoria with approximately 120 students.

A mixed-methods design guides the study:
(1) refinement of exercises and rubrics;
(2) development of a custom LLM-based assessment platform;
(3) a pilot phase assessing reliability, usability, and student perceptions; and
(4) integration into weekly lab sessions.

Students submit their solutions through the platform and receive immediate, rubric-aligned formative feedback whose structure was optimized using cross-model comparative analysis.

Beyond accelerating correction times, the feedback produced by JorGPT is designed to support metacognitive processes such as planning, monitoring, and self-reflection. Its conversational nature encourages students to question their assumptions, compare different solution strategies, and understand not only what went wrong, but why. Preliminary evidence suggests that this reduces grading anxiety and increases the perceived fairness and transparency of assessment. Meanwhile, instructors benefit from substantial reductions in repetitive grading workload, enabling a shift toward higher-impact interactions.

Expected outcomes include:
(a) over 90% reduction in grading time;
(b) strong convergence between LLM-based and human assessments;
(c) significant gains in students’ perceived usefulness and clarity of feedback; and
(d) measurable improvements in self-regulation and engagement.

The study aims to offer a scalable model demonstrating that AI-assisted assessment can be pedagogically meaningful, ethically responsible, and operationally efficient in programming education.

Keywords:

Programming Education, Automated Grading, Formative Feedback, Metacognition, Self-Regulated Learning.