Abstract:

Introduction:
Objective Structured Clinical Examination (OSCE) is a well-established platform to assess the learners’ clinical ability in a safe and conducive learning environment. As an integral component of the University of Medicine and Pharmacy- Ho Chi Minh City- Vietnam (UMP-HCMC), the Center for Advanced Training in Clinical Simulation (ATCS) is committed to standardizing the assessment process to enhance the quality and especially the validity of the OSCE platform in assessing the learners’ competency in pre-clinical and clinical phases. We decided to initiate a pilot study to measure the content validity of the History Taking checklist of the OSCE station for second-year medical students.

Methods:
We used the 6-step content validation process that is commonly reported in the literature, applied the 4-level rating scale of relevance (not relevant, somewhat relevant, relevant, highly relevant) to prepare the content validation form, and sent it to the review panel that consisted of 6 faculty assessors who are experts in standard setting and clinical simulation. These assessors critically reviewed, refined, and independently scored the domain and its related checklist items according to their relevance. We calculated four parameters of the Content Validity Index (CVI): Item-CVI (I-CVI), Scale-CVI/ Average (S-CVI/Ave), Scale-CVI/ Universal Agreement (S-CVI/UA), and Content Validity Ratio (CVR), and interpreted the results according to the recommended cut-off score of CVI (=> 0.83) and CVR (=>0.8).

Results:
Of the original 37-item checklist, 14 (37.84%) had I-CVI under 0.83; most of them (10/14) belong to the competencies of taking present illness. The S-CVI/Ave was 0.80, S-CVI/UA was 0.24, and CVR was 0.60. The revised checklist was reduced to only 17 items, and S-CVIs increased dramatically (S-CVI/Ave = 0.98, S-CVI/UA = 0.88, and CVR = 0.96).

Conclusion:
The validation process for content validity of the OSCE’s checklists conducted by faculty assessor experts in standard setting and clinical simulation has greatly improved the quality and validity of the history-taking OSCE station. The positive results of this pilot study have prompted us to continue this validation process for all current and future OSCE checklists to improve the quality and validity of our assessment tools.

Keywords:

Validity, validation process, content validity index, content validity ratio, I-CVI, S-CVI/Ave, S-CVI/UA, CVR, ATCS, OSCE.