UTILISING TECHNOLOGY IN THE ASSESSMENT OF COLLABORATION: A CRITIQUE OF PISA’S COLLABORATIVE PROBLEM SOLVING TASKS

Technological tools are increasingly becoming embedded in learning, teaching and assessment. Advances in technology offer new opportunities for assessing collaborative learning and problem solving skills in areas and contexts where assessment would otherwise not be possible. Computer-mediated communication environments can provide a record of activity that can be kept, replayed, and modified.

This presentation divides into two parts. The first explores the extent to how technology can support facilitation and assessment of group collaborative learning, how technology can offer a more effective means for recording the capture of interactions between group participants, and how technology can be employed as an agent for initiating collaborative behaviour. The second offers an in-depth critique of how one institution (the OECD) through the Programme for International Student Assessment (PISA 2015) attempts to assess collaboration. In PISA 2015 students collaborate with computer-based conversational agents. These agents are designed to represent team members who exemplify a range of collaborative skills, knowledge and understanding, and are programmed to introduce a degree of conflict that needs to be negotiated by the human partner. Technology is used in an attempt to control interactional boundaries, with the intention of pinpointing collaborative behaviours and traits in students’ responses. Furthermore, PISA uses technology in the recording of responses. In this critique, we present the outcomes of an exercise that mapped the assessment approach of PISA 2015 to pertinent constructs of the collaborative process, and recent theoretical developments related to engenderment of collaboration within assessment tasks (Child & Shaw, in press). The fundamental constructs that comprise effective maintenance and progress of the collaborative state include: Social interdependence (when the outcome of individuals is affected by their own and others’ actions); Conflict resolution (group members may take ‘path of least resistance’ when given a group task, either by avoiding conflict, reaching early agreement, or by dividing the workload into discrete sub-tasks where little discussion is required); Introduction of new ideas (creation of new ideas, speaking respectfully, keeping an open mind to new ideas, perseverance); Sharing of resources (sharing thinking and reasoning with others in the group); Cooperation/task division (group members sub-divide tasks without reference or discussion with other members); and Communication (using linguistic and non-linguistic features).

References:
[1] Child, S & Shaw.S .D. (in press). Collaboration in the 21st century – implications for assessment. Research Matters 22.