Abstract:

Context & Background:
The rapid advancement of generative artificial intelligence (GenAI) threatens academic integrity, especially in online education. Unauthorized use during summative assessments can undermine work authenticity. Traditional proctoring methods fall short for capstone project evaluation, since they are often completed off-platform, take days or weeks to finish, and can even encourage the use of GenAI for skill development. Prior research has established the viva voce exam as a robust method for verifying the authenticity of academic work. However, its application has been limited to in-person settings, making it impractical for large-scale online education. This challenge necessitates a scalable solution to ensure the authenticity of student submissions.

Goal:
This research aims to apply AI to conduct viva exams, providing a scalable solution to academic integrity challenges in online education, enhancing trust, ensuring excellence, and increasing adoption of online credit-bearing programs.

Research Questions:
- How effective is the AI Viva exam in assessing students' skills?
- Can it deter academic misconduct?
- What is the user experience for students and educators?

Methods:
The study focuses on implementing and evaluating an AI-based Viva exam, simulating a scalable dissertation defense. A GenAI chatbot conducts live conversations with students, assessing their understanding and critical thinking. A pilot study was conducted with a university partner in degree courses hosted on Coursera. The AI Viva exam was integrated into manually-graded summative capstone projects. The AI conducted live questioning following the project, assessing students' depth of understanding, critical thinking, and ability to reference their own work. We assessed educator confidence, inauthentic flag rate, course completion rate, learner satisfaction, and performance metrics.

Findings:
Initial results are promising, with only a small subset of submissions flagged as inauthentic and the preliminary reviews from the course team agreeing with the vast majority of the AI viva exam scores. These results suggest the students receiving below-threshold AI viva exam grades may need additional support in their learning. Educators and students were largely pleased with the assessment experience and the rigor it added.

Significance:
If consistent utility is demonstrated, the AI Viva exam could be an effective, scalable solution for enhancing academic integrity in online education. Looking ahead, the broader team at Coursera is exploring the integration of audio and video into future versions of this academic integrity feature on the platform.

Keywords:

Academic integrity, oral exam, GenAI, chatbot, assessment, online education.