Abstract:

Chatbots are increasingly being used in various aspects of daily life, including shopping, customer support, and education. They have evolved significantly during the last decade, with many of them now incorporating some form of artificial intelligence. In the field of education, chatbots are particularly useful, since they can enhance learning experiences, provide individualized attention, and support online classes. Being accessible via smartphone, they furthermore help engage students outside of traditional classroom hours. And by utilizing chat platforms such as Telegram or WhatsApp, which are already familiar to students, the likelihood of prompt responses and increased motivation is higher than with standard learning platforms. Yet, building chatbots for educational purposes can be challenging, as most implementation frameworks require significant time and technical effort. To address this, we have developed a chatbot creation tool for the Telegram chat platform that allows for easy integration of custom multiple-choice questions. Teachers can quickly create chatbots and respective questions using Excel or XML files following the QTI (Question & Test Interoperability) specification, while students can simply subscribe to these chatbots and consequently receive periodic questions.
So far, we have not tested the tool-created chatbots with students, yet we have gathered feedback from n=8 teaching professionals concerning the creation process. The evaluation thereby covered two creation approaches. On the one hand, we instructed participants to create a chatbot and multiple-choice questions using a specifically formatted Microsoft Excel template. On the other hand, they were asked to create questions via the Tests & Quizzes tool offered by the Sakai Learning Management System (LMS) and to then use the therein provided export function to create a QTI compliant XML file.
A questionnaire instrument containing 13 questions was used to evaluate the experience with this chatbot creation process. The questions covered both the chatbot creation itself and participants’ background knowledge. Factors such as prior use of an LMS and familiarity with Telegram aimed to provide some information on participants’ technology savviness. Additionally, we wanted to know whether they think that using chatbots for their teaching would be beneficial. Finally, they were given the opportunity to provide suggestions for additional improvements.
Results show that the chosen Telegram platform is not widely used among teaching personnel (i.e. only three of our eight participants had used it before) and that people may have significant data protection concerns. Consequently, it seems crucial for future versions of the tool to support various other, privacy preserving chat platforms. Furthermore, it was shown that the familiarity with the QTI standard seems to vary among educators, with digital subject teachers being more knowledgeable than those in "management only" subjects. Still, all of our participants were able to create a working chatbot, and we received positive feedback concerning the short amount of time it took them to do so. Results are also encouraging in that all participants expressed willingness to use chatbots for their courses and that they felt that learning with such a conversational tool seems to be more efficient and less time-consuming than with other learning instruments.

Keywords:

Learning Technology, Chatbots, Tool Evaluation, User Study, Learning Support.