Abstract:

Proficiency in 21st century skills (also known by terms such as professional, transferable, or soft skills) ranging from the ability to communicate effectively with a range of audiences to engaging in lifelong learning, is critical for success in today’s knowledge economy. Studies have shown that employers around the world value these skills in graduates more than disciplinary knowledge. In a recent study in the US, 93% of employers indicated that an ability to think critically, solve problems, and communicate clearly, are more important than a student’s major. Many studies have shown that students around the world are weak in these skills.

While academic programs worldwide strive to develop a solid professional skills base in students, these skills are notoriously difficult to assess. Much attention has been focused on these skills in recent years in the disciplines of engineering and computing. ABET (the Accreditation Board for Engineering and Technology Education), the global leader in technical education accreditation, prescribes a number of outcomes that students should attain by graduation. Among them are six non-technical skills and in the area of computing they are:
• An ability to analyze a problem, and identify and define the computing requirements appropriate to its solution;
• An ability to function effectively on teams to accomplish a common goal;
• An understanding of professional, ethical, legal, security and social issues and responsibilities;
• An ability to communicate effectively with a range of audiences;
• An ability to analyze the local and global impact of computing on individuals, organizations, and society;
• Recognition of the need for and an ability to engage in continuing professional development.

Assessment methods in existence evaluate each skill separately, distinct from one another. In addition, these metrics are used to evaluate a skill indirectly through focus groups, interviews or surveys eliciting student opinion. Disparate measurement tools not designed to complement one another and that rely heavily on perceptions are inadequate for data-driven curriculum decision making. These constraints can be problematic for accurate and useful course and program-level assessment of attainment of student learning outcomes because they:
(a) do not provide direct measures of student learning and
(b) can make the assessment process resource intensive and cumbersome to implement.

In addition, academic programs face significant challenges in using such assessment data for meaningful curricular change focused on increasing student learning of core skills.

Here we present the first and only direct method and measurement tool in the literature to measure the six ABET Computing Accreditation Commission non-technical skills for both course and program level assessment. The Computing Professional Skills Assessment is a discussion-based performance task, conducted through asynchronous online discussions, designed to elicit students’ knowledge and application of the skills. The method was developed in a funded research project over a two year period. Our trial has shown that the method can successfully measure the skills. This paper outlines the method and its implementation and describes data collection and findings.