DIGITAL LIBRARY
EMPIRICAL ASSESSMENT OF METACOGNITIVE OUTCOMES: EVALUATING THE IMPACT OF AI-DRIVEN SCAFFOLDING ON UNIVERSITY STUDENTS' LEARNING PROCESSES
University of L'Aquila (ITALY)
About this paper:
Appears in: EDULEARN26 Proceedings
Publication year: 2026
Article: 0144
ISBN: 978-84-09-88444-5
ISSN: 2340-1117
doi: 10.21125/edulearn.2026.0144
Conference name: 18th International Conference on Education and New Learning Technologies
Dates: 29 June-1 July, 2026
Location: Palma, Spain
Abstract:
The efficacy of learning processes during the first academic year is strictly correlated with students' capacity for self-regulation and cognitive monitoring. This study stems from the identification of a growing need for pedagogical tools capable of supporting executive functions as students navigate the increasing complexity of digital content (Ali et All., 2023; Chen et All., 2024). The investigation analyzes how the introduction of a device based on Generative Artificial Intelligence (GenAI), specifically configured according to a "cognitive friction" model, can influence and enhance information management and strategic study planning (Isidori et All., 2025). The theoretical framework challenges the trend of "algorithmic passivity," proposing instead a model where AI acts as a catalyst for reflective thought.

Methodology:
The research adopts a quasi-experimental test-retest design aimed at verifying the effectiveness of a structured pedagogical intervention. The sample consists of 79 first-year students enrolled in the Primary Education Sciences degree program. To measure cognitive processes, the Metacognitive Awareness Inventory (MAI) was administered both at the beginning (pre-test) and at the conclusion (post-test) of the intervention. The experimental phase involved the active use of a specialized toolkit (integrating Gemini, Copilot, and Diffit) as a mediator for the critical structuring of complex workflows. Specifically, students were tasked with the design of assessment instruments—structured, semi-structured, and unstructured tests—intended for fifth-grade primary school pupils. This docimological task required students to constantly validate the system's outputs, cross-referencing AI-generated suggestions with pedagogical goals and logical consistency, thereby preventing mere task delegation.

The data evaluation highlights a significant transformation in learning outcomes following the intervention. Post-test analysis indicates that 83.9% of the participants perceived the interaction with the AI toolkit as a "reflective stimulus," which prompted them to systematically question and refine their initial cognitive hypotheses during the design process. A notable improvement emerged in the "Information Management" subscale: 32.3% of the students reached the highest levels of awareness in using guiding questions to identify and organize key conceptual nodes for their assessment rubrics. Regarding the "Planning" function, 67.7% of the sample reported an increased ability to autonomously organize and sequence educational activities. This data marks a critical transition from a fragmented approach to a strategic, holistic vision of the instructional task.

The study confirms that the integration of advanced ICT aids into higher education requires rigorous evaluation protocols that transcend simple technological efficiency. The results suggest that, when properly mediated through a "cognitive friction" approach, GenAI can serve as a powerful ally for developing monitoring skills. This research provides robust empirical evidence for the design of multimedia learning environments. These environments are capable of transforming human-AI interaction into a sophisticated opportunity for metacognitive development and individual empowerment, equipping future teachers with the professional tools necessary for conscious and self-regulated instructional design.
Keywords:
Metacognitive Assessment, Empirical Research, AI-Mediated Learning.