Abstract:

The rapid diffusion of generative Artificial Intelligence (AI) in education requires structured frameworks capable of supporting teachers not only in tool literacy but in reflective and intentional instructional design. While many training initiatives focus on operational skills, fewer address how AI can function as a catalyst for metacognitive awareness and pedagogical decision making (Isidori et All., 2025; Tang et All., 2025; Zacharis & Papadakis, 2024). This study presents the first empirical validation of an AI based toolkit integrating Gemini, Copilot, Diffit, Canva AI and Quizlet within a structured training program for pre service and in service teachers, situated within the methodological tradition of experimental pedagogy.

The research adopts a longitudinal pre test/post test design with a three month interval, combining the administration of the Metacognitive Awareness Inventory (MAI) (N=294) with a re test on the subgroup that completed the full training cycle (N=215). A complementary questionnaire assessed the perceived impact of AI on planning, monitoring and evaluative reasoning. Participants were engaged in the design of a complete didactic unit, including learning objectives, sequencing of activities and construction of assessment tools. AI was used not as a generator of ready made products but as a mediating device to refine conceptual coherence, validate instructional choices and align activities with pedagogical intentions. The task required teachers to interrogate AI outputs, compare alternatives and justify decisions, ensuring methodological control over the cognitive processes activated.

Findings show consistent improvements across both MAI dimensions—Knowledge of Cognition and Regulation of Cognition—with increases between 8% and 12% in items related to strategic planning, monitoring of understanding, adaptation of learning strategies and management of cognitive load. The AI impact questionnaire confirms that teachers perceived generative tools as catalysts for reflective thinking, particularly in critical source evaluation (86% positive responses), visual planning of didactic sequences (92%) and refinement of assessment criteria (89%). Qualitative comments further indicate that AI supported the clarification of pedagogical intentions and the alignment between activities and assessment, reinforcing the internal and content validity of the toolkit.

The study demonstrates that AI, when embedded within a structured pedagogical framework, can strengthen teachers’ reflective agency and enhance the coherence between objectives, activities and evaluation. The validated toolkit represents a scalable and sustainable model for teacher education programs aiming to integrate AI ethically and pedagogically, promoting intentionality, self regulation and evidence based instructional design. These results highlight the potential of AI mediated scaffolding to transform teacher training from tool centered practices to reflective, strategically grounded professional learning.

Keywords:

Teacher Education, Instrument Validation, Generative AI, Instructional Design, Agency.