Abstract:

Understanding how reflection quality improves in project-based learning remains underexplored. While interventions combining human coaching with AI-facilitated reflection show promise, the developmental pathways of reflection depth and metacognitive awareness require systematic investigation. This study examines reflection quality development mechanisms in undergraduate project-based learning seminars.

This research investigates how LLM-enhanced reflection influences reflection quality development. We examine:
(1) the developmental trajectory of reflection depth,
(2) the metacognitive language patterns, and
(3) the relationships between engagement consistency, reflection quality, and learning outcomes.

Fifty-two undergraduate students, two faculty members, and five Action Learning coaches participated in LLM-assisted reflection across two periods (May-July and September-November 2025). Students submitted weekly reflections documenting project events, learnings, and action plans; the LLM responded with personalized encouragement and advice. Action Learning coaches facilitated team reflection sessions. Mixed-methods analysis combined:
(1) qualitative coding for reflection depth (descriptive, analytical, critical levels), and
(2) computational analysis for metacognitive language patterns (planning, monitoring, evaluation).

Engagement consistency was measured through submission frequency and temporal patterns. Complete interaction logs enabled systematic quality development tracing.

Students who maintained consistent engagement demonstrated progressive deepening, transitioning from descriptive to analytical and critical reflection levels. Computational analysis showed corresponding increases in monitoring and evaluation indicators. However, sustaining engagement remained challenging for some participants. Interaction logs revealed specific instances in which LLM prompts successfully scaffolded deeper reflection, though effectiveness varied by engagement levels.
Systematic mixed-methods text analysis reveals specific mechanisms of reflection quality improvement in LLM-enhanced interventions. Consistent engagement with structured AI-facilitated reflection supports progressive development of reflection depth and metacognitive awareness. Future research should investigate factors supporting sustained engagement and long-term retention of reflective practices.

Keywords:

LLM-enhanced reflection, Project-based learning, Metacognition, AI-facilitated learning, Action learning.