Abstract:

Digital competence is recognized as an essential 21st century skill for participation in contemporary society. Its educational framing includes computational thinking (CT), often defined as “solving problems, designing systems and understanding human behavior by drawing on concepts fundamental to computer science”. With curricular presence of CT, particularly in secondary education, there’s an increased need for effective assessment. Teachers require reliable tools to monitor progress; researchers need validated instruments to evaluate significance; policymakers depend on robustness for informed decisions. Several CT assessment tools are already available to meet those needs, like K-12 tools divided into categories for diagnostic, summative, formative-iterative and data mining assessments, which differ in evidence sources and cognitive objectives. Diagnostic, summative tools often use predefined lower-level skills items, while formative-iterative and data mining tools analyze student-generated work to capture higher-level processes. In this regard, cognitive complexity on which assessment focuses is a crucial consideration. Researchers consistently positioned CT as problem solving that resides at higher taxonomy levels, emphasizing importance of creativity as core component of CT pointing to creation as valued outcome. This emphasis on higher-order skills underscores importance of aligning assessment requirements with educational objectives. But gaps between high-level goals of critical thinking and requirements of current instruments pertain, as most existing tests focus on lower cognitive levels, particularly comprehension. Transitioning to advanced, high-level assessment is complicated because this depends on specific programming environments, limiting applicability in diverse educational settings. Therefore, instruments are needed to assess CT competencies at high cognitive levels while remaining adaptable to educational settings. Our study explored the extent to which the developed Creative Computational Competence test (Triple-C) measures CT for which creativity is a prerequisite. Triple-C is designed as semi-open, block-based assessment grounded in creative problem-solving theory, combining cognitive framing of CT with practice-oriented dimensions operationalized to the process definition of creativity. Semi-open tasks were scored for functional correctness and conceptual fit. Validation involved 3 phases: expert review, pretest, and a pilot study with 107 students aged 12–14 in secondary education. Content validity was partially supported: CT-related ratings exceeded for assessing content validity of single items, while creativity ratings were more variable. Face validity was confirmed by students' perceptions and timely completion. Internal structure analysis revealed students’ current versus proximal competence levels with high overall reliability. External validity showed greater creative engagement in difficult items, and consequential validity analyses revealed large, systematic differences across 4 competence levels. Our findings indicate Triple-C captures CT knowledge and its creative application in ill-defined contexts. Its design supports large-scale administration without prior platform knowledge, enabling formative feedback via competence-level mapping. Validity evidence substantiates Triple-C’s theoretical foundation and highlights its potential for investigating developmental interplay between CT and creativity.

Keywords:

Computational Thinking, creativity, competence, assessment, programming, education.