Abstract:

Information technologies (e.g., e-learning tools) are crucial to education, especially for the accessibility of knowledge and learning. The importance of self-learning, especially video-based learning, is increasing, particularly for school students, due to teacher shortages and increasingly heterogeneous classes with a broad performance spectrum. Recently, assistive tools like Chat-GPT have progressed as support tools. However, individual knowledge gaps can only be addressed to a limited extent due to the prevailing oversupply of learning platforms.
Studies show that learning aided by explanatory videos increases learning and retention. The AI Education (AIEDN) research project investigates how an AI-based learning assistant can compensate for existing weaknesses by enabling a better understanding through video-based learning. AIEDN encourages a deeper engagement with the content, a more personalized approach to learners, and is expected to enhance learning outcomes more than traditional video-based learning. For this study, 275 students were selected in the age range of 14-20 from two secondary schools (N=137) and two grammar schools (N=138) in Baden-Württemberg, Germany, each with an approximately equal distribution of participants from grades 9 and 11, considered due to comparable psychological maturity shortly before their graduation.
A quantitative experiment was conducted from March to May 2023 to test the hypotheses: the extent to which learners solve more tasks, build broader (transfer) knowledge, and retain it. Students in both grades were given a fixed set of mathematical problems on a previously unknown topic to solve in 90 minutes. The students, divided into two groups, had to familiarize themselves with the topic. The test group used the AI assistant to ask questions, while the control group used only keyword searches. The learning assistant works based on semantic AI, understanding questions’ meaning and displaying matching passages from the videos of the math YouTuber Daniel Jung. Mathematics was chosen as the subject because of its objectivity and comparability. The test was repeated after 6-8 days with similar tasks of the same scope without an assistant.
To investigate the performance increase with the AI learning assistant, t-tests were conducted. Analyzing the results by the school types, within the test group of grammar schools, a slight change (Cohen’s d=.34) in the results between the two test days existed (T(65)=-2.73; p<.01) when the AI tool was used. In contrast, within the control group, no significant change was observed (T(71)=-.05; p>.05, Cohen’s d=.006). Within the test group of the secondary school that used the AI tool, a slight (d=.22) performance increase (T(71)=-1.88; p<.05) was also observed as opposed to the control group (T(67)=-1.42, p>.05). Thus, a positive correlation between the AI learning assistant and the student learning was observed, particularly for the grammar school students, in favor of the hypotheses. However, the results were not as significant for the secondary school students, as the given tasks were not designed for their level of understanding and were considered too demanding. The video content used was primarily designed for grammar schools. Further research is needed to determine AIEDN’s performance for other target groups, possibly in a longitudinal study, controlling for confounding variables such as the stressful environment of the classroom compared to a home environment.

Keywords:

Education, self-learning, AI, semantic AI, AI-based learning, video-based learning.