Abstract:

Introduction:
Medical history taking is a key teaching topic in the Objective Structured Clinical Examination (OSCE) for second and third-year medical students in the Vietnam-Germany program at the Center for Elaboration Competency and Innovation in Clinical Simulation of Pham Ngoc Thach University of Medicine (CECICS-PNTU). To help students develop communication skills with patients before clinical practice, we designed several common medical history-taking scenarios involving standardized patients (SPs), such as cough, fever, dyspnea, abdominal pain, and jaundice. To enhance assessment reliability and ensure multiple assessors, we conducted the study titled: “Inter-Intra Rater Reliability of Standardized Patients in the Formative Assessment of Medical History Taking Skills at a Clinical Simulation Center in Vietnam.”

Background:
This study investigates the inter-intra rater reliability of SPs through the formative assessment of students’ communication skills in a jaundice history-taking scenario. Successful outcomes will help optimize faculty resources, enhance the role of SPs, lay the foundation for teaching other clinical skills, and improve assessment quality at CECICS-PNTU.

Methods:
This cross-sectional study was conducted in December 2022. Before participating, SPs underwent a training program at CECICS-PNTU and met the required standards. SPs assessed students using a checklist comprising 10 items. Two SPs were selected for this study. The Learning Space system included 22 third-year medical students in the Vietnam-Germany program (M3VG), divided into two circuits: the first with 10 students and the second with 12 students. Each circuit comprised six OSCEs, each lasting six minutes. The jaundice history-taking OSCE was one of these. SP1 interacted with 10 students, and SP2 with 12 students. SPs completed the checklist after the OSCE session. Four days later, the same SPs reassessed the 22 students' checklists via video. Analysis included inter-rater reliability between SP1 and SP2, intra-rater reliability of each SP, and comparison of mean values between SP pairs (inter-rater) and each SP individually (intra-rater).

Results:
The overall results indicated good inter-rater and intra-rater reliability. For the group of 10 M3VG students, the Cohen's kappa coefficient ranged from near perfect to perfect, and the intraclass correlation coefficient (ICC) ranged from modest to excellent. For the group of 12 M3VG students, Cohen's kappa ranged from substantial to perfect, and the ICC ranged from modest to excellent. For the group of 22 M3VG students, the inter-rater reliability between the two SPs was near perfect for the Cohen's kappa coefficient and excellent for the ICC (95% confidence interval). The T-test and Mann-Whitney U test, with a statistical significance level of P=0.05, indicated no statistically significant difference in the mean values between the two SPs in inter-rater assessment and within each SP in intra-rater reliability.

Conclusions:
The SP training program at CECICS-PNTU resulted in reliable and positive inter-intra rater reliability outcomes. Thus, SPs can effectively participate in the formative assessment of medical students' communication skills in jaundice history-taking and similar scenarios. This study also enhanced the role of SPs, optimized faculty resources, and improved learning and teaching quality at CECICS-PNTU.

Keywords:

Standardized Patient, Jaundice, Medical history, OSCE, formative assessment, inter-rater reliability, intra-rater reliability, clinical simulation, medical education.