Abstract:

Introduction:
In recent years, especially during the COVID-19 pandemic, online teaching has emerged as a popular alternative to traditional classroom-based teaching, offering flexibility, accessibility, and a variety of innovative learning tools. Evaluation of those experiences is become of great importance as it contributes to the learning experiences and effectiveness of online learning. However, quantitative assessment often fails to capture the complex nuances of the online learning experience, while qualitative assessment helps to understand students’ subjective experiences, perceptions, and behaviours of students’ in the online learning environment. The aim of this study was to develop and test a protocol for qualitative assessment of the online learning experience.

Methods:
The protocol was developed in several phases:
(1) planning of the methodological concept,
(2) development of a topic guide,
(3) pilot test,
(4) evaluation,
(5) retest,
(6) final conceptualization, and peer review.
A qualitative-descriptive design was used, which can be applied in different educational settings and used by experts with advanced or basic knowledge of qualitative research. The topic guide was developed based on an extensive literature review and pilot tested in an online focus group (n=5) involving students who had completed an e-learning course. Focus groups were selected because the goal was to stimulate the exchange of views and to provide different perspectives on the phenomena under study. The final testing of the topic guide was conducted with a population of students with online learning experience and collected in four focus groups (n=16). Data were analysed using thematic analysis. The final protocol was evaluated by three experts.

Results:
The overall evaluation of the protocol for qualitative assessment consists of three main phases: (a) preparation phase, (b) assessment, and (c) analysis and evaluation and was positively evaluated by the expert panel. The content analysis identified five main categories that represent fundamental themes of the topic guide:
(1) perceptions of organisation and content delivery,
(2) being part of an online community,
(3) perceived support, ongoing evaluation, and feedback,
(4) technology and learning tools,
(5) identification of learner engagement strategies, and
(6) knowledge translation.

Discussion:
Developing a robust qualitative assessment protocol for online learning is essential to collect meaningful data and make informed decisions to improve the quality of online learning. The proposed qualitative assessment protocol has proven useful in gaining deep insight into students' experiences with e-learning. It should be used in conjunction with quantitative assessment to provide comprehensive assessments that contribute to the continuous improvement of online learning practises.

Keywords:

Evaluation study, higher education, e-learning, distance learning, students.