Abstract:

The critical need for organizations to learn and change is well known. “The ability to learn faster than your competitors is your only lasting competitive advantage” as put by a HR manager. Thus, knowledge is vital, both as a resource and a competitive advantage. The importance of life-long learning and distance education is clear, just as the role of higher education. The aim of this paper is to share academic experiences and lessons learnt concerning performing high quality assessment on-line and at the same time offering a learning opportunity for both teachers and students. This is a big challenge.

We performed a case study in an on-going hybrid course. There were 10 students on the course, all professionals and employed and hence performing the course and working at the same time. We used the final course assessment as a case unit, which was performed as structured on-line dialogues between each student individual and the examiner. Data were collected by observations of three students performing and a deep follow-up interview with the examiner responsible. Each dialogue lasted about one hour. To be eligible to participate the student must have prepared a task based on two books and questions aiming to put the literature in comparison as well as in the context of their own employment. The examiner leads the dialogue based on graduate assessment criteria, developed by the examiner. The collected data were inductively analyzed.

The analysis reveals advantages that oral individual assessments have concerning quality assurance and at the same time enable learning. The examiner has natural opportunities to ask follow-up questions to stimulate reflections and further learning. The examiner also has opportunities to ask questions that reveal the students’ knowledge and understanding related to the assessment criteria and the syllabus learning outcomes, which is a critical aspect not at least from the perspective of AI and chat-GPT. Since the dialogues are individual, the time perspective is a matter of fact that is often lift as an argument against this type of examination. According to the examiner, this is not a problem if you take all the steps in the examination process into consideration. The total time is more or less equal compared to traditional written examinations. Furthermore, the examiner emphasizes that this form of examination is much more fun.

To achieve both quality assessment assurance and learning, the analysis also reveals the importance of a holistic approach, as for example putting the matter of course in the students´ reality as well as the critical interface between the task and the assessment dialogue. It also strengthens the importance of preparing for both the examiner and the student and emphasizes the importance of having a system for continuous follow-up and improvements. Finally, the follow-up interview with the examiner reveals the learning it means by having someone externally observing the performance and then asking relevant questions and having a dialogue afterwards. “It forced me to think and reflect on the work from new perspectives, which is critical for future improvements” as put by the examiner. One piece of advice from the examiner is to include this type of work in the quality assurance system for course development. To reflect on the work was an important input to further increase and develop the course.

Keywords:

Learning, quality assurance, on-line assessment, academic experiences.