Abstract:

The debate over the role of Artificial Intelligence (AI) in Higher Education Institutions (HEIs) has evolved. Initially, HEI stakeholders were obsessed with banning it or keeping it out of the hands of their students. When they realized that this was an impossible task, many HEIs shifted their focus to exploring how AI could be utilized in their classrooms. However, the emphasis remained on security; what confidential information might we be sharing? Who owns the output? This led to the first step in adopting AI in the HEI classrooms, i.e., the drafting of AI policies designed to limit or control the use of AI. Some faculty members followed a green light policy, allowing all use in any manner, while others chose a yellow route, where AI could be used in a specific and limited manner. Still others chose the red route, which forbids any use of AI in the classroom. Once the policies were sorted, faculty members had to consider the competencies they were assessing in this new AI environment. Could they keep the same tasks and assessments? Do they align with the AI policy that has been implemented?

"Critical Thinking" has been defined as an analytical process that helps learners justify their responses and evaluate the choices they have made. Until now, the ability to think critically has been one of the greatest benefits of higher education. However, this, too, has been debated in the AI venue. Many researchers emphasize the importance of critical thinking and argue that AI can help students enhance their critical thinking skills. However, their studies rely on student self-reporting that skills have improved. Conversely, some studies claim that the use of AI hinders students and reduces their ability to think critically. In those studies, the picture painted is quite grim; they posit that students will not improve their critical thinking but reduce it through the use of AI. Nonetheless, for both sides of the debate, there is scant research that ‘proves’ that AI helps or hinders critical thinking.

Thus, if AI is accepted in HEI classrooms, how can it be utilized in a critical manner? This study addresses the following questions:
RQ 1: Can graduate students distinguish between the two writing levels and critique the AI output they receive?
RQ2: Can graduate students suggest effective methods to improve the work to a graduate level?

To examine this topic, the researcher, who teaches Academic Writing in the Graduate School of an international hospitality management school in Switzerland, gave students an AI task to complete in the free version of ChatGPT. They needed to run the same prompt twice (at the undergraduate and graduate levels) and compare the output of the two levels before comparing their output with that of their colleagues. As a final step, the students were asked to do a critical analysis of the exercise and the output they received.

Based on early results of 129 students, the depth and breadth of critical thinking about this AI task and its output are mixed. While the critique of the outputs remained relatively light in analysis, the recommendations for improvements proved much richer. Their recommendations could be used to help the faculty member develop more effective writing tasks for the next cohort. The novelty of this study lies in a student’s ability to analyze the output and make concrete recommendations for improvement. We intend to create a new Academic Writing course that includes these recommendations.

Keywords:

Higher Education Institutions (HEIs), Graduate Students, Generative Artificial Intelligence (Gen AI), Critical thinking, hospitality management studies, Switzerland.