Abstract:

In a modern world, anecdotes, unsubstantiated claims, and misinformation are abundant across all platforms of social media. Novel contexts present unique challenges to evaluate claims due to a paucity of shared understanding. The proposed paper establishes a process to a) identify claims from a digital game’s encapsulating social ecosystem and b) leverage data science techniques to quickly evaluate the merits of those claims. In this way, this work establishes broader understanding of the skills and knowledge associated with a novel context, as well as provide a model for hypothesis testing in adjacent environments. This work examines a video game and its encapsulating social media as a platform to: 1) derive socially constructed hypotheses, 2) establish patterns of play within games, and 3) create the means to evaluate those hypotheses. The League of Legends (LOL, Riot Games, 2009), a video game, was selected because it is massively popular and players discuss play mechanics, trade tips, and improve their performance across its associated media and discussion forums.

An explanatory-sequential mixed-method design was applied to this study (Creswell & Plano Clark, 2017). Initial steps involved an automated process to scour Reddit and Twitter, extracting commonly held assumptions about play. Players hypothesized that activities that secure information (i.e., contextualized in the game as “warding”), performance (i.e., successful game actions against other players), and resilience (i.e., staying alive or defensive actions) were regularly cited as the most important aspects of improving success. Each of these hypotheses are linked to authentic, observable actions that are logged by the game. These data were extracted from LOL using a second automated process. Educational Data Mining (EDM) and Learning Analytics (LA) techniques (see Baker & Inventado, 2014; Baker & Siemens, 2014) were applied to these gameplay data in LOL. An exploratory structural equation model was used to evaluate underlying constructs in LOL relative to the player-held hypotheses. Specifically, parameter estimations from the ESEM model were tested against emergent themes and specific hypotheses which garnered support (via upvote or through comments) by the gaming community. The entire process from data collection to inference was very quick (2-3 days).

The findings confirmed that the opinions and conjectures of players reflect their own anecdotes, rather than verifiable patterns of play. Alignment with success was statistically incidental and often individualistic. According to the data, defensive actions, which include mitigating opponents’ opportunities for advancement, were most instrumental in game success. These superseded acts associated with gaining information and offensive performance when examined in relation to winning the match. Implications of this research include developing intentional coaching strategies in this specific game. More broadly, the approach discerning community-based and using the performance context to evaluate them has implications for traditional online teaching contexts and non-traditional learning contexts, like social media. In either, researchers can strategically evaluate group assumptions and challenge them with authentic, performance data very rapidly. In this way, EDM and LA are demonstrably valuable in novel and non-traditional learning environments (Baker & Siemens, 2014; Siemens, 2013).

Keywords:

Video games, learning analytics, mixed methods.