Abstract:

ChatGPT, a technology that uses natural language processing, has received a significant attention since it launched in November 2022. Therefore, it becomes a hot research topic recently. As a result, several studies have investigated the accuracy, performance, and limitations of this AI model in education. This study compares the quality and accuracy of student responses to exam questions with those generated by ChatGPT 3.5. Students and ChatGPT were asked to provide answers to a set of exam questions in computing engineering. Diverse question formats such as Multiple Choice Questions (MCQs), fill in the blanks, numerical calculations, and comprehensive questions are incorporated in the set of exam questions for this study. Students' responses and the ChatGPT generated responses were evaluated by the authors using a range of criteria, such as correctness, clarity, grammar, and relevancy to the questions. The findings of this study shows a similarity between ChatGPT and students' responses in MCQ and fill in the blank questions. However, a significant gap arises in problems that require calculations across multiple stages, highlighting a gap in ChatGPT's mathematical skills. This gap requests more investigation into ChatGPT's computational abilities, emphasizing the significance of improving the model's training and algorithms to improve its performance in mathematical problem solving. In contrast, none of the student responses to comprehensive and general knowledge questions matched ChatGPT's systematic clarity. ChatGPT generated responses were not only clear and well-structured, but also showed a clear flow of ideas.

Keywords:

ChatGPT, Computing Engineering, Engineering Education.