Abstract:

Growing potential of chatbots is getting more attention in education, both in application and in the research area. Using chatbot technology as a new channel of collecting feedback from students is an innovative approach that is less time and resource-demanding for both sides included in evaluation: teachers and students. Main purpose of this study is to present a result summary of the students’ reflection upon courses collected in an innovative way – using the educational chatbot Hubert. Hubert’s features include dynamic conversation, personal experience and better interpretation of student feedback. Built on the basis of user feedback, new features include a more dynamic chat conversation that allows for a more personalized experience and better comprehension of students’ input. Hubert's main advantage is engaging students and customizing flow of conversation about course perception and asking follow-up questions for additional explanation. The study was conducted at three courses at the University of Zagreb, Faculty of Organization and Informatics in 2020. Number of students involved in study was n=69. Thematic analysis was conducted to reveal patterns and themes within a data set of student-chatbot conversations. Following Braun and Clark's six-step recursive framework, main themes and categories were found upon positive and negative aspects of the course, suggestions for improvements and students' personal opinion. Since course evaluation was conducted in full swing of COVID-19 pandemic, some of the themes are related to this topic. In conclusion, guidelines and recommendations for implementing chatbots as a form of course evaluation in higher education are highlighted.

Keywords:

educational chatbot, course evaluation, higher education, thematic analysis