EVALUATION OF A PROFESSIONAL PROGRAM DELIVERED THROUGH INTERACTIVE TELEVISION TECHNOLOGY: IMPORTANCE OF FORMATIVE AND SUMMATIVE EVALUATION

Introduction:
In response to the shortage of health care services in rural northern Michigan, the Physician Assistant (PA) program determined that it could best serve the state’s education and health needs through an expansion of the PA program to northern Michigan. The expansion to northern Michigan was justified by literature which identified that there is a strong relationship between a student’s connection to a rural background and the student’s subsequent intention to train and work in a rural area. By delivering the program through an Interactive Television (ITV) distant-learning format, students are able to remain within their rural communities during the education, and eventually become an integral component in the delivery of health care to their community. The delivery of the program involves ITV between the two cohorts of students: parent location in lower Michigan and distant campus in northern Michigan. The program developed an extensive formative and summative evaluation process to assure that both the program goals and accreditation standards are being met. This paper describes the implementation of the evaluation process and the results of the first year of implementation.

Methods:
Evaluation tools were established based on weaknesses and strengths of distant education as identified in the literature: a formative evaluation tool was used weekly and a summative evaluation tool used bi-semester. Formative evaluation included lecture content, special ITV needs, orientation of new lecturers, student concerns, support staff updates, and student/faculty traveling between campuses. Summative evaluation included 19 statements designed to evaluate the following: quality of technology, quality of instruction and instructor interaction between the two locations, and availability of resources. Students were given the opportunity to write qualitative remarks. The goal of the summative evaluation was to have 80% or greater of the students respond favorably(agree/disagree) to each of the 19 statements. The summative evaluation was administered at two time points during each semester to both cohorts: mid-semester and end-of-semester. Data was analyzed according to the program’s goal.

Results:
Weekly formative evaluations allowed for identification of issues that could be addressed quickly: audio quality, and orientation to visiting faculty on ITV use. The results of the first summative evaluation, identified that 84% of the statements met the goal of 80% or greater in the favorable category when both cohorts were combined. When stratifying the qualitative comments between the two cohorts, the summative evaluation identified that students at the parent location did not have adequate access to printers, and students at the northern Michigan location did not have adequate access to library resources. These deficiencies were address by replacing printers, and additional orientation for students on library resources. In subsequent summative evaluations, the trend over time of meeting the program’s goal of 80% increased to 92%.

Conclusion:
Successful expansion utilizing ITV technology requires extensive monitoring using both formative and summative evaluations. Through use of the formative and summative evaluations the program was able to address the needs of students at each location and to improve in meeting the program’s goals and accreditation requirements.