Abstract:

Artificial intelligence natural language processing technologies, like ChatGPT, have been used in many areas and for different purposes. ChatGPT has great public attention and is one of the fastest growing applications. As most new and innovative technologies, it also has the potential to benefit the field of education. This paper analyzes a difference in students’ perception between questions written by human and generated by ChatGPT. The questions cover both code and theory of an introductory course of python programming language. Authors prepared a total of 60 questions, of which half was written by human and half was generated by AI. Questions are equally distributed in easy, intermediate and advanced level. Students were given an exam with randomly selected questions, but instead of solving them, they had to evaluate their relevancy, clearness, precision and difficulty. They were also given correct answers and had to assess their accuracy. Except for the above, students had to estimate whether the questions were written by AI or a human. The paper describes a sample of students that had taken the exam, the structure of questions, and presents results, especially correlating values for human and ChatGPT questions.

Keywords:

Artificial intelligence, ChatGPT, higher education, students’ perception, exam questions.