Abstract:

Context of this work:
This article presents the design of a chatBot-type tool to provide effective feedback to students in programming courses. The tool was created based on chatGPT and is intended to help computer science students at the Universidad Católica de la Santísima Concepción. This work is part of a larger investigation where we want to measure the usefulness of tools based on LLMs in the teaching-learning processes, specifically during feedback.

Methodology:
The research methodology consists of four stages. The first stage involves an immersion in the domain of Large Linguistic Models (LLM) through a review of the literature on the use of artificial intelligence in education, the evaluation of intelligent tutors, types of feedback for learning and mastery. chatGPT. The second stage focuses on the development of the chatBot, including a conversational and non-conversational version and its integration to an LMS (Learning Management System) platform and an open source programming online judge. The third stage involves the design of an experiment to evaluate the usefulness of the tool. Finally, the intervention stage involves requesting the University's ethics committee for approval of the study, implementation of the experiment, data extraction and evaluation.

Theoretical importance:
This research contributes to the expanding field of integrating artificial intelligence tools in education. By developing a feedback tool that leverages the capabilities of chatGPT, this study shows the potential of AI to facilitate learning processes. The research also highlights the importance of safeguarding academic integrity when using artificial intelligence technology.

Data collection and analysis procedures:
The data for this research will be collected through a quasi-experimental study in which students from the Programming and Database laboratories will participate. The study will measure the effects of using the chatBot created on the development of computational thinking skills. Data mining will involve collecting students' interactions with the tool, including their feedback and performance on programming tasks. The data will be analyzed using appropriate statistical methods to evaluate the usefulness and effectiveness of the tool.

Questions addressed:
The research aims to address the following questions:
1. How can chatGPT be used to provide effective feedback to computer science students in programming courses?
2. What is the usefulness of TutorBot+ to enhance the learning experience and development of computational thinking skills?
3. How can academic integrity be ensured when using AI-based feedback tools?

Preliminary results:
The preliminary results of this work are promising. A functional prototype called TutorBot+ has been developed and has been successfully integrated into an online programming contest platform and an LMS (learning management system). The research also explores the possibility of customizing a pre-trained model for the programming domain. These findings suggest that the development of an AI-based feedback tool for programming courses is feasible and has potential to improve the learning experience. Currently, having received acceptance from the UCSC ethics committee, we hope to be able to apply and evaluate the experience during the first and second semester of 2024.

Keywords:

Effective feedback, learning strategies, assessment, chatGPT.