Abstract:

The aim of this paper is to describe the evaluation process and findings of an AI-based Adaptive Learning System for the Computer Science discipline at two different German universities and discuss an array of methods in regard to assessing such a system. The primary objectives have been twofold: firstly, to examine the reception of selected learning elements, which were conceptually outlined based on relevant literature, among the student body; and secondly, to investigate the efficacy of individualized adaptive learning paths. These paths were generated by employing a variety of algorithms to analyze students learning style tendencies, with a particular emphasis on adaptive navigational techniques. The used algorithms encompassed a modified version of a literature based adaptive mechanism, an Ant-Colony-Algorithm and a Genetic Algorithm, alongside a lecturer-recommended learning path for a non-adaptive comparison. While the system suggested suitable learning paths based on student data, it never forced the individuals to give up their self-directed learning. The evaluation criteria revolved around the evolution of student motivation, interest levels, and knowledge acquisition during the time they spent working in the system. The evaluation sought to facilitate comparative analyses and assess algorithmic fitness for proficient learning path generation. The methods included both quantitative and qualitative approaches to gather data, seeking to strike a balance between being student-friendly and scientifically informative. They ranged from Likert Scale self-assessments to screen and video observations with retrospective interviews. Since the purpose of adaptive learning systems is intertwined with personalized learning it seems imperative to already take the preferences and opinions of students into account while the system is still in development. This complexity underscores the challenge of evaluating such systems, as significant constraints on student choice - though simplifying evaluation - directly oppose the ethos of individualized, self-directed learning. Initial findings suggest that the underlying theoretical considerations on sequencing and structuring of learning elements are confirmed, coupled with providing adequate flexibility to meet diverse learning needs. Cross-site evaluation of the literature-based learning elements indicated a high comprehensibility and positive student ratings. While significant positive trends were observed regarding knowledge acquisition, they cannot be definitively attributed to a specific method of learning path generation. Motivation and interest analyses show no significant differences among learning path types, albeit heavily limited by sample size. Similarly, emotion measurements, though limited, hint at positive impacts from HASKI system use. Despite limitations, early indications suggest student acceptance and potential effectiveness of learning paths, highlighting the need for larger sample sizes for validation and expansion. Ensuring alignment with student needs and user-friendly design are crucial considerations.

Keywords:

Adaptive learning environment, tertiary education, personalized e-learning, evaluation.