Abstract:

In a large engineering university, introductory engineering courses typically have hundreds of students which makes running exams challenging. To meet this challenge, over the past 4 years we've developed a large-scale Computer-Based Testing Facility (CBTF). The CBTF is a proctored computer lab where students are offered individualized exams and these exams can include a broad range of question types (e.g., numeric, graphical, design, and programming; not just multiple choice). In Fall 2017, the CBTF served 21 courses from seven different departments in the College of Engineering and over 6,000 unique students. Over 52,000 exams were proctored, including 3,500 final exams.

In this work, we explore how this large-scale introduction of computer-based testing has impacted students and instructors. Specifically, after describing the basic operation of the CBTF, we discuss the results of multiple rounds of surveys completed by students and faculty.

Some highlights from our student surveys include:
(1) students generally are more satisfied with CBTF exams relative to traditional paper exams (45% satisfied or very satisfied, vs. 17% dissatisfied or very dissatisfied), but this preference seems to vary by major, with computer science and electrical engineering majors even more strongly preferring the computerized exams,
(2) students' favorite aspects of CBTF exams include the flexibility to schedule them at convenient times, that CBTF courses generally have more frequent, shorter tests, and the opportunities to take second chance exams, and
(3) some students prefer the partial credit mechanism commonly used in traditional written exams, where credit is granted for work shown for incorrect answers.

Instructors report having generally positive experiences with the CBTF. A large majority report that once the exam content is in place they perceive reductions in the effort to run and grade exams and to handle the exceptional situations that result from running exams in large classes. Instructors also like how the CBTF enables them to run small frequent tests, run second-chance exams, and test computational skills. Instructors also reported generally high satisfaction with operational aspects of the CBTF, including rules and systems, communication with CBTF staff and proctors. About 85% of the surveyed instructors plan to continue using the CBTF, think that it should be expanded, and want more resources devoted to improving it.

Keywords:

Computer, Exams, Testing, STEM, Engineering, Asynchronous, Proctored.