Abstract:

In educational innovation, learning spaces are increasingly recognised for their potential to improve skill acquisition and knowledge dissemination. Integrating chatbot systems into these environments offers a dual benefit, supporting learners throughout their educational journey and answering ancillary questions. Advanced chatbots have the ability to tailor their responses to individual learner profiles.

The FiSK project is investigating the effectiveness of such systems in the context of teacher training. Teacher trainees are assessed on their didactic ability to analyse students' answers and learning progress. A chatbot system is used to answer questions on educational material. In addition, the chatbot can proactively intervene to provide guidance on the learning path, thereby enhancing the learning experience. The simulated lesson follows a question-and-answer scheme, where predefined questions are to be posed to the virtual students. The answers are designed to align to different levels of learner progress. Three different subject areas have been developed, each by manually creating questions and corresponding simulated answers. The texts follow specific concepts to be identified by the learner, as in the case of evolution: evolutionary concepts, threshold concepts and students' misconceptions. Learners engage in this interactive scenario by selecting multiple choice options to identify the presence or absence of key learning indicators. This diagnostic process is then subject to a summative assessment. A comprehensive learning system architecture has been established that includes a user-friendly web interface, the classroom simulation backend, a cloud-based chatbot backend and a highly responsive middleware. This middleware not only facilitates the chatbot's proactive features, but also ensures seamless integration across all system components.

The studies conducted were designed to address several aspects of the research project, such as user acceptance: learner preferences for interaction density and amount of knowledge. For the chatbot system: personality, friendliness, usability, and quality of interaction.

Of the 175 participants, the majority expressed a preference for decreased interaction frequency (56%) and increased depth of information (41%). The persona of the chatbot was considered satisfactory by 49% of respondents, with a notable approval for its approachability (41%). The navigational aspects were criticised (59%), although these responses were highly polarised (mode: 5, std: 1.12), suggesting different user experiences. The relevance and clarity of the answers were considered average (32%, 35%). Notably, the system's ability to deal with errors was rated as average (46%), although overall usability was rated positively (63%).

To conclude, while the survey has shown the basic acceptance of interactive and guided learning with the help of chatbots, there is still considerable potential for improvement when it comes to communicating the value of such interactive learning tools. Future directions include refining learner adaptation mechanisms, enhancing the naturalistic engagement of the learning environment, and streamlining content generation to facilitate wider application. Advances in text input processing and the integration of automated, formative assessment could further enrich the learning experience.

Keywords:

Chatbot assisted learning, tutoring system technology, learner adoption, educational chatbot systems.