Abstract:

The recent development of Artificial Intelligence (AI) tools is rapidly changing the way we communicate and interact in many fields, including in the educational sphere. ChatGPT (Chat Generative Pre-Trained Transformer) from OpenAI, is a Large Language Model (LLM) that uses deep learning models to generate human-like responses to a variety of user queries.

The incorporation of chatbot-based technologies in educational settings has recently gained significant attention, and as educators, we must guide our students to properly use AI tools, to become more productive, and to foster competencies such as data analysis, curatorship, and prediction.

The current state of ChatGPT, based on a statistical approach rather than a symbolic approach, still yields too many arithmetic and algebraic errors when the chatbot is asked to solve a typical end-of-chapter physics problem. Thus, this is an excellent opportunity to use ChatGPT to challenge and test students’ comprehension of physics concepts and their ability to apply them in well-structured conditions.

A sample (N = 62) of freshman engineering students was considered during the February-June 2023 term at Tecnologico de Monterrey, Mexico City Campus. Each student asked ChatGPT to generate and solve a well-defined physics exercise without showing the procedure. The topics considered were (a) linear momentum conservation, (b) fluid dynamics, (c) calorimetry, and (d) thermodynamic cycles. The students first solved the problems and then compared the final answers with those of ChatGPT, which was wrong almost 100% of the time. The students then asked ChatGPT to solve the same exercise, this time showing the procedure. After identifying the errors made by ChatGPT, the students started a series of feedback loops with the AI tool, sequentially pointing out its errors, so the chatbot could arrive at the correct answer after several feedback loops. The entire feedback process was supervised by the professor during the intervention. This feedback process reinforced students’ comprehension and helped them develop their algebraic and problem-solving skills. All this requires, of course, some previous student knowledge and mastery regarding the concepts and exercises of the course. Each student uploaded screen captures of their interactions to our Learning Management System, which were then reviewed by the professor.

After the intervention, the students were asked to answer an online perception questionnaire about the use of ChatGPT as a learning tool. The survey consisted of five ranking items using a five-step Likert scale, and two open-field items asking the students about AI and the general learning experience.

The survey found that (a) 63% of the students agreed that the use of ChatGPT helps improve their learning process; (b) 63% of the students felt that ChatGPT should be used more frequently in the classroom, and (c) 67% of the students responded that they do not regularly use ChatGPT for their assignments (yet). Regarding the open questions, most students agreed that AI tools have great potential and will profoundly affect our daily lives. The use of this novel technology in an educational setting certainly motivated the students by challenging their understanding of the subject at hand.

Finally, connecting ChatGPT with formal symbolic languages, such as WolframAlpha, will exponentially increase the capabilities of the AI tool, further transforming the landscape of education.

Keywords:

Artificial Intelligence; ChatGPT, Higher education, Educational Innovation.