Abstract:

Ophthalmology education in medical schools is diminishing globally, resulting in poor ophthalmic knowledge and confidence in graduate doctors. Due to the significance of eye health on patients’ overall quality of life, this problem has significant implications for effective patient healthcare and as such, prompts the crucial exploration of novel learning approaches to bolster ophthalmic knowledge. Critical appraisal, a form of active learning, promotes learner engagement and increases knowledge retention. With the rapid emergence of artificial intelligence large language models such as ChatGPT, this study investigated the effectiveness of critical appraisal of ChatGPT-generated answers on medical students’ ophthalmic knowledge and confidence. Furthermore, we gauged student acceptance to determine its practicality and guide future improvements. An educational evaluation study was conducted on 41 participants, with three key topics for appraisal: retinal detachment, glaucoma, and orbital cellulitis/abscess. Students were invited to perform a ten-question multiple-choice questionnaire (MCQ) before and after the appraisal of ChatGPT-generated prompts to assess changes in knowledge and confidence. A 5-point Likert scale was used to assess student confidence and acceptance factors such as enjoyment, perceived usefulness, and satisfaction. Mean MCQ scores demonstrated a 10% increase in knowledge, a 22.7% increase in MCQ confidence and a 15.61% increase in confidence regarding broader ophthalmic topics. Whilst participants generally perceived the activity as “somewhat useful”, feedback suggested it was “slightly less enjoyable than other online learning activities”. Overall satisfaction was heavily correlated with perceived usefulness and enjoyment. These findings bolster the benefits of critical appraisal in medical student education, a significant consideration in the context of decreasing ophthalmic education and emerging artificial intelligence large language models. As such, the integration of critical appraisal in conjunction with conventional teaching is suggested to provide valuable and enjoyable learning experiences for students.

Keywords:

Ophthalmology education, medical education, critical appraisal, active learning, education, artificial intelligence, large language model, medicine, medical.