Abstract:

Almost a decade ago, we redesigned our first-year computing classes in an entirely competency-based manner: in order to preserve the constructive alignment according to Biggs & Tang, the course was designed around the central competency of understanding a problem statement, deriving an algorithm to solve it, implement and test using state-of-the-art programming tools (IDE) - with a stepwise enhancement of programming concepts and language features.
Consequently, the final exams have been held at the computer labs where students were allowed to use almost the same tools as in their practical classes – except only for uncontrolled internet access which would open the door to synchronous communication with peers or other “experts”, and thus for cheating. This type of competency-based exam did not only respond to formal university requirements, i.e. it can be regarded as an ordinary paper exam where students just use a computer, IDE and printer access to finally submit a bunch of paper; feedback from students was very positive (“we don’t need to learn for the exam”), however, the Gaussian grade-distribution curve got somehow inverted as is often observed with competency-based exams on advanced taxonomic level.

So far, everything went well – even during the first year of the COVID-19 pandemic when classes had to be moved to online. With the use of video conferencing (Zoom) and other tools for social-learning (Perusall), audience response (Mentimeter) or visual collaboration (Miro), we were able to offer almost the same learning experience as before the pandemic – as long as the final exams could still be held at the labs, thus in a controlled environment.

When COVID-19 incidence raised dramatically in early 2021, state regulations suddenly did no longer allow for any exams being held on campus. Thus, we needed to transform exams from labs to online at students’ homes at very short notice. While exams for other courses have been moved to open-book or take-home exams, automated tests and e-assessments or just any kind of unsupervised homework, these were no options to be used for our programming course.

Our main goal was to keep the competency-based approach having students demonstrating their competency in developing and testing a computer program to solve a given problem using state-of-the-art tools while at the same time preserving fairness and validity of the exams. Therefore, we needed to have the exam setting at the student’s home as close as possible as in the lab at university. As the programming environment (IDE etc.) is already available on the students’ computers (from the practical classes), the main issue came with uncontrolled use of internet access. Technical solutions like online proctoring with video capturing and dedicated software to be installed on the students’ computers have been no option at all due to serious privacy concerns.

However, using the same tools as in the online classes, esp. Moodle plus Zoom with break-out rooms and screen sharing, together with online audio and video supervision of the students by our own staff, we have been able to design the setting and process to conduct true competency-based programming exams that are both fair and as cheating-resistant as on campus. First results from pre/after-exam-surveys also prove acceptance by our students. Thus, we did not let COVID-19 ruin our competency-based exams and thus preserved the constructive alignment of our CS1 courses even in times of the pandemic.

Keywords:

Competency-based education, competency-based exams, programming exams, online exams.