Abstract:

Qualitative coding constitutes a basic process in qualitative educational research, yet it remains a resource-intensive task. While recent breakthroughs in Artificial Intelligence and Large Language Models (LLMs) promise to automate this process, their reliability in highly specialised fields remains under scrutiny. This paper presents a study that explores the performance and methodological implications of AI-assisted coding compared to manual coding within the specific domain of Engineering Education and mechanical 3D CAD life-long training. We designed a comparative analysis using a one-hour semi-structured interview transcript, which was coded using five distinct approaches: traditional manual coding, internal AI from ATLAS.ti computer-assisted qualitative data analysis software, and external generative models including ChatGPT-5, Perplexity Pro, and Gemini 3. Contrary to the prevailing narrative of AI efficiency, our results reveal significant limitations in current models; specifically, AI tools exhibited a lack of conceptual granularity and coding accuracy, struggling to interpret the nuances of mechanical 3D CAD terminology. Consequently, the findings suggest that the time and effort required to supervise, validate, and correct AI-generated codes in technical STEM contexts may outweigh the benefits of automation, advocating for a cautious integration of AI where human expertise remains indispensable for ensuring data integrity

Keywords:

Engineering education, AI-assisted coding, qualitative research, technical interview analysis.