Abstract:

E-learning now has the support of a very sophisticated information technology, an ever-growing AI, improved data analytics, and a better understanding of how online courses are designed and delivered. The provision of e-learning particularly, though online distance courses, is developing rapidly in small and medium sized enterprises (SMEs) where owners and workers cannot abandon their job to follow classroom-based education or training initiatives, while also coping with family and social responsibilities. Moreover, owners of SMEs are also at a disadvantage because they often do not have the financial resources to provide this type of training. On the other hand, digital learning is increasingly being considered as a more cost-effective solution to the training of basic skills. It is also increasingly being regarded to be a valid, if not better, alternative to face-to-face teaching and learning provision. However, one of the weakest areas of e-learning, is the evaluation of courses. Indeed, even though many enterprises are investing substantial financial resources to e-training, very few of these organisations – either through a lack of resources, or lack of interest - actually review the training’s impact on workers, the business, and their clients. There is evidence that, indeed, much training is not achieving its intended objectives. A rigorous and well-planned evaluation process is thereby critical. Poorly designed evaluations, however, do not help in improving an online programme.

This paper tries to partly fill this gap. Through a systematic review of the literature, it will first identify the models that are currently being used in the evaluation of training courses. It will then analyse the factors that work against these models and their practical implementation in the online dimension. It will then attempt to develop an original working model that can be used by e-trainers, designers and SME owners to measure the quality of the learning in online courses. The resulting model addresses various pedagogical and course design issues, including assessment, the scaffolding of learning, technical support, the availability of efficient technology at the workplace and student satisfaction. The reliability and validity of the model is tested through a qualitative investigation involving key stakeholders of Maltese SMEs. The results suggest that the evaluation process must address a number of methodological problems, including the need for measures other than standardized achievement tests, differences among students about their needs and expectations of learning, differences in learning and teaching styles, and program implementation. Above all, summative evaluation activities, including the popular ‘smiley sheets’ based on Kirkpatrick’s model, are shown to be less effective than formative methods, particularly if the course pedagogy is inspired by constructivist learning theory which requires constant online teaching presence.

The paper ends with suggestions and recommendations for future research. As the need to help workers lacking basic skills increases (both for economic and active aging reasons), and this can mainly be achieved through alternative learning initiatives, the evaluation model proposed in this paper may assist policymakers, planners, funding agencies, designers and providers in making reasonable and informed judgments with regard to the quality of learning in web-based distance education.

Keywords:

Evaluation, e-learning, basic skills, lifelong learning, SMEs, Malta.