Abstract:

MOOCs are becoming highly relevant and widespread learning tools in higher education institutes both in traditional and distance universities. Most of the top tier Universities in the World joined this innovative scenario in on line education. MOOCs are “based on video lectures, multiple-choice quizzes or peer-review assessments, discussion forums and documents” (Khalil, Ebner, 2016a, 2016b). This widespread dissemination requires new tools and strategies to assess the quality of the MOOCs. With respect to face-to-face courses, a huge literature has been developed to deal with the evaluation of the effectiveness and efficacy of the courses and more in general with the quality of the courses from all the points of view. A research challenge is now how to develop learning analytics (LA) customized to the MOOC framework (Johnson et al., 2014). The most popular definition of LA is “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Long, Siemens, 2011).

The aim of this paper is to propose a multivariate approach to assess the quality of MOOCs.
The analysis refers to the courses offered by the Platform Federica.eu (www.federica.eu) at University of Naples, Federico II. In 2015, the University of Naples, Federico II has entered the world of MOOCs launching a new series of online courses with more interactive features and more multimedia contents. More than 60 courses are offered and they range from basic introductory courses to topics of particular public interest.

The proposed analysis exploits different sources of information, from participant demographics to learners’ trends in usage (for example with respect to number and length of online sessions, downloaded materials), course organization (text slides, video-lectures, links to online sources), course outcome (gained score, certificate, % of lessons), and didactic approaches. Advanced statistical methods based on Multivariate analysis will provide a simultaneous analysis of all the sources of information identifying user profiles, course profiles and learning profiles.

All the analysis will be based on objective indicators but in the future it will be enhanced with data from surveys on the user-friendliness (look and design impression, navigation easiness, schedule of the course, video accessibility, quality of the slides) and on the teaching quality.

References:
[1] Khalil M., Ebner M. (2016a). What Massive Open Online Course (MOOC) Stakeholders Can Learn from Learning Analytics?. Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy. In J. M. Spector, B.B. Lockee, M.D. Childress (eds) Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy. Springer International Publishing.
[2] Khalil M., Ebner M. (2016b) Learning Analytics in MOOCs: Can Data Improve Students Retention and Learning? Proceedings of the World Conference on Educational Media and Technology (EDMedia 2016), Vancouver, Canada, Volume: 1, pp. 569-576.
[3] Johnson L., Adams Becker S., Estrada V., Freeman A. (2014) Horizon report: 2014 Higher Education, Austin TX, USA: The New Media Consortium.
[4] Long P., Siemens G. (2011) Penetrating the Fog: Analytics in Learning and Education, http://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education

Keywords:

Learning analytics, MOOC, multivariate analysis.