Abstract:

Complex skills, like analytical thinking, are essential in preparing students for future professions. Serious games hold potential to stimulate the online acquisition of such professional skills in an active and experiential way. Rubrics are proven assessment and evaluation instruments, but were never directly integrated into actual gameplay. We present our approach towards the integration of analytical rubrics into gameplay (Gamebrics), the implementation into existing (scenario-based) serious games, and main findings from an experimental study into learning effects and appreciation of the (Gamebrics) tooling (dashboard with progress information and reflective feedback).

Our experimental research setup randomly allocated eighty-four participating students into either a control group or an experimental group. The intervention for the control group consisted of an existing serious game with natural feedback as part of the scenario, supporting the gameplay itself. The intervention for the experimental group also provided an additional dashboard with reflective feedback, monitoring the acquisition of analytical skills through gameplay. Learning growth through gameplay was measured by pre-/post-test scores on a knowledge test, in-game performance by means of computer logging, and appreciation of the dashboard and feedback by means of a questionnaire.

Our participants were university students taking a distance education course that included a serious game. Fifty-six of them took a beginner course (and game) on management and organization (domain of management science), twenty-eight of them took an advanced course (and game) on experimental design of psychological studies (domain of psychology), allowing us to also generalize findings across domains and game types.

Participants receiving dashboard information show (significantly) higher learning increases (on a pre/post-knowledge test), and (significantly) more efficient gameplay behavior (on performance scores). Students overall appreciate the dashboard with reflective feedback as 'just above neutral', however we did find that studying it makes them more confident about competence development. These findings appear promising for this (Gamebrics) approach on integrating formative assessment in serious gameplay to support the acquisition of complex skills.

Keywords:

serious games; analytical rubrics; analytical thinking skills; formative assessment; reflective feedback; monitoring dashboard