Abstract:

In the rapidly evolving domain of computer engineering education, the integration of cutting-edge technologies into teaching methodologies presents a promising way to improve learning outcomes and pedagogical effectiveness. This manuscript describes an innovative approach within a Master's course entitled Embedded and Ubiquitous Systems, where Large Language Models (LLMs) are used to critically analyse and refine the course syllabus, assessment questionnaires and exercises, and coding assignments. The focus of our research is to assess the potential of LLMs to support teachers in designing a more effective educational framework, thereby indirectly benefiting the student body.

Our investigation unfolds in several distinct phases. Initially, we use LLMs to conduct a thorough review of the existing course syllabus, aiming to identify gaps and areas for improvement. This is followed by a careful analysis of the evaluation questionnaires used by the course instructors. By simulating the performance of students through the lens of LLMs, we ascertain the extent to which these questionnaires accurately reflect the intended learning outcomes and the depth of understanding required at the Master's level in Embedded Systems. Moreover, we scrutinise the design of coding and programming activities, evaluating their alignment with course objectives and the practical skills they aim to instil.

A critical aspect of our research involves assessing the accuracy of LLMs in solving evaluation activities designed for the course. This evaluation serves as a benchmark to determine the reliability of LLMs in providing correct answers to complex questions within the domain of embedded and ubiquitous systems. Through this analysis, we aim to unveil the potential of LLMs not only as a tool for improving the design and assessment of educational content but also as a means of measuring the cognitive and analytical demands placed on students.

Our findings suggest that LLMs hold substantial promise for revolutionising the pedagogical landscape of computer engineering education. By enabling a data-driven refinement of course content and assessment mechanisms, LLMs contribute to a more targeted and effective educational experience. Although the current phase of our research does not involve direct student interaction with LLMs, the implications of our work pave the way for future explorations into the integration of these technologies into student learning processes.

This manuscript advocates for the proactive adoption of LLMs by educational staff in course design and evaluation processes. It highlights the utility of LLMs in designing a curriculum that not only meets the academic and practical demands of embedded and ubiquitous systems, but also enhances the overall learning experience through the strategic use of technology. Our research invites a wider discourse on the role of artificial intelligence in education, particularly in the areas of syllabus development, performance assessment, and the dynamic adaptation of teaching methods to the ever-changing landscape of computer engineering.

Keywords:

Large Language Models (LLMs), Computer Engineering education, Course syllabus design, Artificial intelligence in education.