About this paper

Appears in:
Pages: 4334-4342
Publication year: 2015
ISBN: 978-84-608-2657-6
ISSN: 2340-1095

Conference name: 8th International Conference of Education, Research and Innovation
Dates: 18-20 November, 2015
Location: Seville, Spain


C.M. Steiner1, P.A. Hollins2, E. Kluijfhout3, M. Dascalu4, A. Nussbaumer1, D. Albert1, W. Westera3

1Graz University of Technology (AUSTRIA)
2University of Bolton (UNITED KINGDOM)
3The Open University of the Netherlands (NETHERLANDS)
4University Politehnica of Bucharest (ROMANIA)
Digital games constitute a major emerging technology that is expected to enter mainstream educational use within a few years. The highly engaging and motivating character of such games has the potential to support the creation of immersive, meaningful, and situated learning experiences. To seize this potential meaningful quality and impact measurement are indispensible. Although there is a growing body of evidence on the efficacy of games for learning, evaluation is often poorly designed, incomplete, biased, if not entirely absent. Well-designed evaluation demonstrating the educational effect as well as the return on investment of serious games may foster broader adoption by educational institutions and training providers, and support the development of the serious game industry. In the European project RAGE (http://rageproject.eu/) a holistic and multi-perspective framework for serious game evaluation is developed and applied and is presented in this paper.

The primary objective of RAGE is to make available and accessible advanced software tools, methodologies and expertise for supporting serious game development and take-up: Reusable, and interoperable software components (“gaming assets”) for creation are made available, with the ability to undertake various kinds of data analysis (e.g. competence assessment, emotion detection) and game intelligence (e.g. competence-based personalisation, natural language understanding). These assets will be provided via an Ecosystem as a central access point and affinity social space. These games technologies will be applied and tested in learning game development for six selected application scenarios.

The RAGE evaluation framework forms the basis for analysis of the effectiveness of the RAGE technologies, and for ensuring that they reflect the needs of industrial and educational stakeholders. The assets will be evaluated from the industry perspective in respect of their software quality and their added value in the authoring process. In addition, the Ecosystem will be assessed in terms of its usefulness, usability, and overall impact for gaming stakeholders. Further, a contextual cost-benefit analysis will be undertaken in each of the (six) application scenarios. The evaluation perspective in addressing the “educational effectiveness” of the games will be on analysis of their benefit for learning, as well as users’ perceptions. Unobtrusive data tracking and sensoring will complement more traditional instruments (such as questionnaires), thus providing a wide spectrum of evaluation methods capable of obtaining in-depth insights of players’ behaviours and experiences.

The RAGE evaluation framework will give rise to scientifically sound, iterative, and mixed-method evaluations of the project outcomes and may serve as best practice for future serious game evaluation, in general. The tools and methods that RAGE produces will be of interest to the wider serious gaming research and industry communities as they strive for improving the quality of serious games.
author = {Steiner, C.M. and Hollins, P.A. and Kluijfhout, E. and Dascalu, M. and Nussbaumer, A. and Albert, D. and Westera, W.},
series = {8th International Conference of Education, Research and Innovation},
booktitle = {ICERI2015 Proceedings},
isbn = {978-84-608-2657-6},
issn = {2340-1095},
publisher = {IATED},
location = {Seville, Spain},
month = {18-20 November, 2015},
year = {2015},
pages = {4334-4342}}
AU - C.M. Steiner AU - P.A. Hollins AU - E. Kluijfhout AU - M. Dascalu AU - A. Nussbaumer AU - D. Albert AU - W. Westera
SN - 978-84-608-2657-6/2340-1095
PY - 2015
Y1 - 18-20 November, 2015
CI - Seville, Spain
JO - 8th International Conference of Education, Research and Innovation
JA - ICERI2015 Proceedings
SP - 4334
EP - 4342
ER -
C.M. Steiner, P.A. Hollins, E. Kluijfhout, M. Dascalu, A. Nussbaumer, D. Albert, W. Westera (2015) EVALUATION OF SERIOUS GAMES: A HOLISTIC APPROACH, ICERI2015 Proceedings, pp. 4334-4342.