Abstract:

The effectiveness of AI-powered adaptive learning systems depends not only on the sophistication of large language models but fundamentally on the quality, structure, and governance of the knowledge bases that sustain them. In high-stakes standardized exam preparation contexts, poorly curated content can lead to misalignment, hallucinated explanations, conceptual gaps, and unreliable feedback. This paper presents the design and implementation of a Retrieval-Augmented Generation (RAG) knowledge architecture developed to support an AI-based exam preparation agent for senior Industrial Engineering students preparing for a national standardized exit examination.

The project integrates curriculum mapping, structured content engineering, semantic indexing, and quality validation into a unified pipeline for knowledge base construction. The process began with a systematic alignment between the official competency framework of the standardized exam and the course learning outcomes. Content sources included validated textbooks, prior exam blueprints, institutional syllabi, and ad hoc developed materials. These materials were decomposed into granular knowledge units tagged by topic, cognitive level, difficulty, and competency domain.

The curated corpus was transformed into semantically indexed embeddings and stored in a vector database to enable context-aware retrieval. Instead of allowing the language model to generate questions solely from parametric knowledge, the RAG architecture constrains generation to verified content retrieved dynamically based on the user’s prompt and performance history. This approach enhances factual reliability, pedagogical alignment, and traceability of generated questions, study guides, and feedback explanations.

A multi-layer validation strategy was implemented, including expert faculty review, automated consistency checks, duplication detection, and item calibration testing. The system also incorporates metadata logging to track which knowledge segments are retrieved most frequently and which generate higher student error rates, enabling iterative refinement of the knowledge base. Beyond technical architecture, the project emphasizes governance principles, including version control, source transparency, bias mitigation, and ethical data management. By combining retrieval-augmented generation with structured academic curation, the system balances scalability with academic rigor.

This study contributes to emerging research on knowledge engineering for AI in higher education by proposing a systematic framework for content preparation in high-stakes assessment environments. It demonstrates that robust knowledge-base design is a critical determinant of AI reliability, pedagogical coherence, and measurable improvements in student performance. The framework is transferable across disciplines and provides a replicable model for institutions seeking to deploy AI agents in standardized exam preparation while maintaining academic integrity and quality assurance.

Keywords:

Retrieval-Augmented Generation (RAG), Knowledge Engineering in Education, AI-based Training Systems, Content Curation and Validation, Standarized Exam Preparation.