Abstract:

This paper presents the design, implementation and evaluation of a specialized generative AI assistant that supports students in the Department of Electrical and Electronics Engineering at the University of West Attica during Diploma Thesis selection and administration. The assistant is implemented in no-code environments (a Custom GPT and a Perplexity Space) and combines prompt engineering patterns with Retrieval-Augmented Generation (RAG) so that its answers are grounded in official departmental regulations, curriculum structures, faculty lists and the institutional thesis repository. Its purpose is to provide clear guidance on available thesis options, supervisor selection, procedural steps and required documentation, while explicitly avoiding the generation of thesis chapters or ready-made submissions to preserve academic integrity and student autonomy.

Two versions of the assistant were developed and nine Large Language Models, including standard, Thinking and Reasoning variants, were benchmarked using a 19-prompt test suite aligned with four functional categories: factual queries, reasoning tasks, procedural instruction following and creative topic suggestion based on a student’s academic profile. Model outputs were evaluated on accuracy, completeness, relevance and security/compliance using a five-point satisfaction scale. Results show consistently strong performance on factual and procedural tasks, where the assistant delivers precise, reliable guidance on regulations, deadlines and administrative workflows, confirming that well-structured system prompts combined with RAG-based access to institutional documents can offer robust support for routine thesis-related queries.

By contrast, all models perform weaker on creative tasks that require synthesizing a student’s background (semester, track, preferred courses and working style) with faculty research expertise to propose realistic, well-aligned thesis topics. In a follow-up study, the assistant generated thesis topic suggestions and summaries of teaching and research fields for 15 faculty members, who then reviewed the outputs. Faculty generally agreed that their courses and broad research areas were represented accurately, but expressed lower satisfaction with the proposed thesis topics, which were sometimes generic or only loosely connected to their current work. Performance improved when using a Thinking Model together with explicit instructions to search the institutional thesis repository, indicating that tighter integration with structured, up-to-date institutional data can significantly enhance topic relevance and alignment.

Pedagogically, the assistant is conceived as a complementary support tool that reduces students’ uncertainty about a complex, high-stakes academic procedure, while keeping them in control of key decisions and encouraging direct interaction with supervisors and academic consultants. By offering just-in-time explanations, checklists and clarifications, it can lower administrative burden and allow students to focus more on the intellectual and creative aspects of their Diploma Thesis. At the same time, carefully designed guardrails, restricted scope and transparent communication about limitations are used to avoid over-reliance and to maintain the central role of human judgment.

Keywords:

Generative AI Assistants, Higher Education, Diploma Thesis Support, Prompt Engineering Patterns, RAG, LLM Benchmarking, Educational Technology.