Abstract:

We investigate the importance of user experience (UX) aspects for two different large-scale learning environments, openHPI and the HPI Schul-Cloud, and compare them to results from a survey conducted by Winter et al. (2017). Results show similarities regarding the prioritization of pragmatic quality aspects, but noticeable differences for hedonic quality aspects. Our study shows that the category of a learning environment – whether it is a learning management system (LMS) or a Massive Open Online Course (MOOC) platform – needs to be taken into account for the prioritization of UX factors to gain a solid understanding of important aspects for users in their respective context of use and learning experience.

We aim to measure the satisfaction of users regarding their personal learning experience through questionnaires that are integrated within a learning platform. For this, three steps are necessary:
(1) The identification and prioritization of relevant UX aspects for learning environments,
(2) the development of a questionnaire that covers the most important UX aspects, and
(3) the integration of the questionnaire within a learning platform.

This paper will contribute to (1), the identification of relevant UX aspects.

Our studies investigate how users rank different UX quality aspects in their specific context of use. We hypothesize that the prioritization of UX factors varies for different types of learning environments. We conducted two comparable studies on openHPI as an (x)MOOC platform and Schul-Cloud as a cloud-based LMS. Our approach is based on the work of Winter et al. (2017) in which a procedure is described to create an own list of UX aspects for a product to improve the decision-making and the overall product quality. We followed their experimental procedure to gain comparable insights.

For both platforms we prepared online surveys. No incentive was given. We prepared a list of 21 UX factors that cover pragmatic, hedonic and other qualities. To every factor, participants could attribute a value from -3 (of no importance) to 3 (very important) or mark an UX Factor as 'unsuitable'.

We received 105 completed questionnaires for openHPI and 36 for Schul-Cloud. We considered all UX factors with an arithmetic mean value > 1.5 as important.

Participants of the openHPI study rated 14 aspects higher than 1.5, including Stimulation and Fun, both hedonic qualities. Stimulation and Fun have not been considered as important by Schul-Cloud teachers. They rated 12 factors as important, but the results reflect that pragmatic factors are of higher importance, underlined by the generally lower standard deviation for pragmatic factors.

The results of our studies show that the importance of UX aspects can be clustered for learning platforms as a product category, but varies in specific aspects for different types of learning environments: LMS and MOOC platforms. Pragmatic qualities such as Controllability, Efficiency, Perspicuity, Clarity and Usefulness are important for users of both platforms, while hedonic qualities such as Stimulation and Fun are far more relevant for MOOC platform learners.

Winter D., Hinderks A., Schrepp M., and Thomaschewski J. 2017. Welche UX-Faktoren sind für mein Produkt wichtig? Mensch und Computer - Usability Professionals (Aug. 2017), 1–10.
Schrepp M. and Santoso H.. 2018. Has Culture an Impact on the Importance of UX Aspects?. In Mensch und Computer 2018 - Workshopband.

Keywords:

MOOCs, learning platform, user experience, usability evaluation, openHPI, Schul-Cloud.