Abstract:

This study seeks to understand how undergraduate students can better understand threshold concepts in academic skills, such as integrating theory and applying theory to practice. Threshold concepts are complex concepts that are difficult to learn and when learned, transform learners (Meyer & Land, 2005). We hypothesize that students can cross threshold concepts in academic skills when they are able to identify and understand the cognitive processes to accomplish the tasks that are associated with these academic skills.

Whereas literature suggests 'decoding the discipline' as a method (e.g. Middendorf & Pace, 2004) to 'read the mind' of the expert - in this case the teacher -, we applied cognitive task analysis (CTA) (Hall, Gott & Pokorny (1995) as a method by asking students to break down tasks into activities and to assess its complexity beforehand, thereby 'reading students' minds'. To assess students' learning progress, we developed a new rubric to evaluate their cognitive understanding of these academic skills.

Our empirical data consists of two cohorts of bachelor and pre-master students of industrial engineering and management (IEM) at the University of Twente, the Netherlands. Both cohorts in 2017 and 2018 had to individually write a five page essay about a self-chosen research methodological topic (such as internal validity), to develop a basic theoretical framework, and to apply this model to four bachelor theses of former IEM students. Before starting the assignment, all students handed in an essay proposal, stating their learning goal (topic), motivation and action plan. The 2017 cohort of 50 students received little guidance on the assignment and all students failed the first attempt. After providing a detailed approach, most of the students passed the assignment's retake.

The second cohort of 2018 of 72 students of IEM also first had to hand in an essay proposal without further instruction. Then they received instruction on how to conduct a CTA and had to hand in an improved essay proposal with CTA. During the course we offered sessions on cognitive analysis and we asked them in class to use CTA to evaluate how they applied theory to practice. Finally, these students handed in a final essay with an extended CTA.

A two-step analysis is used to assess the effects of the CTA in enhancing cognitive understanding of academic skills. First, for both cohorts 2017 and 2018 a within-cohort analysis is applied to compare learning progress between the essay proposal and the final essay. Second, learning progress between cohorts 2017 and 2018 is analyzed to evaluate the effects of the CTA approach.

The preliminary findings suggest that cognitive task analysis by students themselves promotes a better cognitive understanding of academic skills. We expect that students' cognitive understandings are higher when they are probed to develop their own CTA (cohort 2018) compared to providing them with a pre-defined and detailed approach (cohort 2017). The teachers' experience (first author) is that applying a CTA method not only reads the students' minds, but also enables to identify new threshold concepts.

We contribute to learning theory by providing a new measurement tool to assess cognitive understanding in academic skills. We also show that cognitive task analysis is a useful method to identify threshold concepts and that it can be used as an educational tool to enhance learning and eventually to cross threshold concepts.

Keywords:

Threshold concepts, cognitive task analysis, academic skills, metacognitive skills, decoding the disciplines.