Abstract:

This project attempts to observe and measure collaboration by examining the dialogue stream of students engaging in typed-chat while they solve problems. In this paper, we report on findings related to improving the collaboration script. This research is in service of creating better computer-assisted collaborative learning exercises, and it furthers the goal of producing technology which can sense the quality of the collaboration by reading the dialogue stream.

The students in this study were in a 2nd-semester undergraduate computer programming class, engaged in COMPS (Computer Mediated Problem Solving) exercises. Each group is aided by a Teaching Assistant. COMPS exercises instruct the students to come to agreement on successive aspects of the problem. Then the Teaching Assistant passes judgment and perhaps provides assistance before the students revise their answer or proceed to the next part. This script is constructed to promote mutual dependence (they don’t signal for the TA until they all agree) and accountability (students should exhibit understanding for each part of the problem).

COMPS records and stores the dialogues for further analysis. Using these student dialogue files, the main research activity consists of manually classifying student dialogue acts according to four categories: sharing ideas, negotiating ideas, regulating problem-solving, and maintaining communication. From these annotated transcripts we count the different behaviors and look for patterns of interaction. This reveals the conversational fingerprints which could be characteristic of successful and unsuccessful student collaborations.

This study shows that the script as executed in real exercises has a problem. Students begin by working collaboratively. Once they come to a conclusion, the students give their answer to the TA. The TA checks their written answer and optionally probes the students to verify understanding. If the group’s answer is incorrect and the TA points out a gap in logic, the students often stop working collaboratively. They will individually engage with the TA, while group cognition ceases. The TA tends to respond to this individual until they are correct. But occasionally the TA will explicitly direct the students to resume working together. Ideally, the TA should give a hint, and then allow the group collaboration to continue. However, this ideal scenario is defeated by normal conversational discourse obligations: it is hard not to answer questions. The role of the TA thus too easily turns into tutoring, and this means tutoring individual students while the rest of the group lurks.

Toward fingerprinting collaborative activity we have been examining the quantitative differences in participation and dialogue acts. Those students with the most sharing ideas and regulating problem-solving dialogue acts tend to be the most prepared students in the group. These students also tend to contribute more dialogue turns. In addition, groups with more TA involvement are shown to have fewer turns involving negotiating ideas.

This research advances toward promoting better student collaborative problem-solving exercises, more fully using student group cognition and collaboration skills, and advances toward computer assessment of student collaboration skills. Problem-solving collaboration is among the 21st Century Skills recently mandated for K-12 education in the U.S. and measured by the PISA international comparison of educational achievement.

Keywords:

CSCL, Collaborative problem-solving, problem-solving dialogues, assessing problem-solving.