Abstract:

The acquisition and use of so-called “soft skills”, including problem solving, resilience, and self-regulation have been associated with better performance at school and in the workplace [1], [2]. Problem-solving is defined as the ability to acquire or use prior knowledge in order to solve new problems. Strengthening this skill is a concern to educators and employers as the 21st century labor market is increasingly unpredictable and requires skills that go beyond mastering and executing familiar processes. Students need to identify and solve problems that they have never encountered before, formulate a solution plan specific to that problem, and execute the plan. Thus far, the body of research that has measured these relationships relies on traditional self-reporting measurement questionnaires.

This methodology is prone to bias since youth may respond in a way they know is desirable, rather than the way they actually behave [3]. Stealth assessment attempts to gather more authentic measurement of skills by asking children to demonstrate them in a structured environment where data collection is unobtrusive [4]. Digital games can be used for stealth assessment, with data on decisions and strategies collected in the application during game play. Since 2017, RTI has been developing games that target a range of soft skills by simulating real-world tasks in a virtual environment. The game designed to measure problem-solving skills gathers metrics on task completion, time management, accepting instruction, problem identification, solution identification, and self-regulation.

This paper describes the multi-year process of development and testing of this game, the results obtained from pilots in the Philippines and Morocco, and the implications for strengthening problem-solving skills among youth worldwide.

References:
[1] Duckworth, A. L. (2011). The significance of self-control. Proceedings of the National Academy of Sciences, 108(7), 2639-40.
[2] Zimmerman, B.J. (2000). Attaining self-regulation: A social cognitive perspective. In Boekaerts, M., P.R. Pintrich, M. Zeidner (Eds). Handbook of self-regulation, (pp. 13–39). San Diego, CA: Academic Press.
[3] MacCann, C., Duckworth, A. L., & Roberts, R. D. (2009). Empirical identification of the major facets of conscientiousness. Learning and Individual Differences, 19(4), 451-458.
[4] Shute, V., & Wang, L. (2016). Assessing and Supporting Hard‐to‐Measure Constructs in Video Games. The Wiley Handbook of Cognition and Assessment: Frameworks, Methodologies, and Applications, 535-562. Hoboken, NJ: Wiley & Sons.

Keywords:

Game-based assessment, problem solving, digital games, stealth assessment.