Abstract:

Conversational agents—commonly referred to as “chatbots”—have been shown to have the potential to support educators with a wide range of tasks, including answering frequently asked questions, encouraging learning, and providing personalized tutoring. Indeed, recent surveys have shown that chatbots have been incorporated into applications aimed at several educational domains, spanning science, technology, engineering, and mathematics (STEM), as well as languages, business, and the arts. Our work focuses on software engineering education and builds on a recent Wizard of Oz study that used chatbot identities as a way to illustrate the code review process. In that study, instructors could impersonate chatbots to annotate code snippets with sample comments. However, these annotations were static, meaning that students could not engage in real-time conversations with the chatbot, instead requiring manual intervention from the instructor. In this paper, we enhanced the static interface used in the Wizard of Oz study by equipping it with chatbots following an automated, interactive script. We then conducted a pilot case study aimed at integrating these interactive chatbots into an online lesson on programming best practices. Using an educational application simulating the code review process, students were introduced to Python style guidelines as a part of their practical work for a course on computational thinking aimed at non-technical students. Our mixed-method analysis sheds light on the opportunities and challenges affecting the implementation of our chatbot integration and its potential for adoption in programming education. While we identified opportunities to guide students through predefined conversational pathways, provide personalized content, request feedback, and incorporate humor into the learning process, we also encountered challenges involving configuration, repeated exposures, and limited scripts. These findings have implications that affect the design considerations of the technological and pedagogical aspects of our implementation, which are relevant to educators and researchers in digital education looking to incorporate chatbots into their practice. Strategies for harnessing these opportunities and addressing these challenges in future work are discussed.

Keywords:

Chatbots, software engineering education, code review, programming, best practices, digital education.