Abstract:

Generative AI-powered chatbots have been increasingly adopted in various sectors, including education, where they hold the potential to reframe the way students interact with learning materials. In this context, many higher education institutions are exploring generative AI’s potential for education. Forward College, an undergrad higher education institution relying on personalized learning approaches (small class teaching, flipped classroom, tutoring, etc) is no different. Our objective is to develop a generative AI powered learning assistant (AIA), in the form of an interactive chatbot, to help students manage their learning tasks in a "flipped classroom" environment. Expected outcomes include improved critical thinking skills, higher engagement levels, and positive attitudes towards learning.

To do so, we divided our research and development process in 3 phases:
1) co-construction of prototype with students,
2) testing of prototype, and
3) impact assessment.

The first phase involved Forward College students in a series of interactive workshops led by a trained researcher to a) decide on the format, tone and personality of the AIA, and b) identify the most relevant learning tasks and problems the assistant could help with, in a flipped classroom environment. To do so, after a thorough training in prompt engineering, students were tasked with creating their own AIA prototype, under the format of a word-based document guide, describing how to create and use the AIA. A total of 11 prototypes were created by students. Out of these 11, 3 were selected based on expert and peer evaluation. The criteria for selection included comprehensiveness, variability of learning tasks, understanding of prompt structures, and output quality control. This participative approach ensured the prototypes were closely aligned with student needs.

In the second phase, an alpha version of the AI assistant was created that incorporated all student requests from the previous prototypes and added the principles of evidence-based learning t​​o foster fact-checking, reflective discussion, and intellectual curiosity. We asked a selected group (consecutive sampling method) of students to provide critical feedback on the assistant. Preliminary feedback indicated positive student reception.

The final phase will involve a controlled intervention to assess the impact of the AI assistant on the learning process. This phase will utilize a mixed methods research approach. Quantitative methods will be performed by pre- and post-tests using such validated tools as Watson Glaser Critical Thinking Appraisal and Motivated Strategies for Learning Questionnaire. The qualitative phase will involve semi-structured interviews with students to contextualize the quantitative findings and a semantic analysis of student-chatbot interaction transcripts for better understanding of efficient communication patterns. The intervention will last for six weeks, allowing sufficient time to do an initial assessment of the AI assistant's impact on learning outcomes and student engagement. Students (n=30) will have unrestricted access to the ChatGPT-4 and the user guide during the intervention.

The outcomes of this study have the potential to become translational research, assisting educators to integrate AI-based solutions into their teaching practice and foster students' critical thinking, motivation, and overall learning journey.

Keywords:

Generative AI, education, flipped classroom, co-construction.