DIGITALASSESSING THE VIABILITY OF GENERATIVE AI FOR CREATING FORMATIVE MULTIPLE-CHOICE QUESTIONS TO ENHANCE FORMATIVE LEARNING IN CARIBBEAN MEDICAL STUDENTS: A PHENOMENOLOGICAL STUDY
1 American University of Antigua College of Graduate Studies (ANTIGUA AND BARBUDA)
2 American University of Antigua College of Medicine (ANTIGUA AND BARBUDA)
2 American University of Antigua College of Medicine (ANTIGUA AND BARBUDA)
About this paper:
Appears in: EDULEARN23 Proceedings
Publication year: 2023
Page: 6503 (abstract only)
ISBN: 978-84-09-52151-7
ISSN: 2340-1117
doi: 10.21125/edulearn.2023.1721
Publication year: 2023
Page: 6503 (abstract only)
ISBN: 978-84-09-52151-7
ISSN: 2340-1117
doi: 10.21125/edulearn.2023.1721
Conference name: 15th International Conference on Education and New Learning Technologies
Dates: 3-5 July, 2023
Location: Palma, Spain
Dates: 3-5 July, 2023
Location: Palma, Spain
Abstract:
Generative assistive intelligence (AI) has been used in various fields, including natural language processing, computer vision, and speech recognition. Recently, researchers have explored the use of generative AI in passing U.S. medical board multiple-choice question (MCQ) exams. However, no research currently exists on using AI-generated MCQs for formative learning in Caribbean medical students. This phenomenological study used a convergent parallel mixed methods design for the purposes of triangulation and complementarity to enhance the significance of findings regarding using generative AI MCQs for formative learning.Preliminary results from direct content analysis revealed that content experts found that chatGPT-generated MCQs, based on curricular learning objectives, were around 85% accurate, had quite high relevance to the curricular content, and could potentially save content experts a lot of time, even if editing is needed. Interviews are currently being coded for emergent themes. Some of the limitations found were that chatGPT often uses MCQ formatting that does not follow USMLE board exam style, included multiple correct answer choices, or answer choice options not allowed by the USMLE exams, and duplicate MCQs or MCQs that are very close to previously created MCQs. Out of about 30 MCQs generated per learning objective, about two were considered by the content experts to not be useful without significant editing.
Overall, this study provides valuable insights into the potential of using chatGPT for MCQ creation based on curricular learning objectives. However, there are some limitations of the content created, and the study highlights the importance of the prompts used in chatGPT in creation of the generated content. It also highlights the need for further research to fully explore the potential of AI in education while addressing the associated limitations and ethical considerations.