Abstract:

Computer programming is a creative but complex task and findings have shown that it can be facilitated with collaboration. Advances in network and collaboration technologies allow the development of powerful collaborative tools, while communication is considered as one of the main factors for successful collaboration, and as such, chats provide rich information on the process of collaboration. Many tools have been created to analyze chat data. Epistemic Network Analysis (ENA https://www.epistemicnetwork.org/) has been used in chat analysis in many different disciplines. ENA measures the connections between the discourse elements or codes by quantifying their co-occurrence. The frequency of the co-occurrence of two codes in a discourse is used to compute the strength of their association in a network.

In this study we used ENA to analyze chat to see if we can detect differences between the connections made by students with different performance levels in an Object-Oriented Programming (OOP) course and their scores in a collaborative solving assignment.

The participants were undergraduate university students of the Department of Applied Informatics, UoM, Greece. Students formed teams of 3 to 4 members on their own. 9 teams of students (34 participants) were selected based on their performances on a 2nd year OOP course. They had to collaboratively solve an OOP assignment in Java using Eclipse for coding, Zoom meeting software for communication and Remote Control to alternately code. The participants of each team communicated only with written messages using Zoom chat. They had 90 minutes to solve the assignment and upload the solution and the chat.

The chat messages of all students were coded for 23 OOP frame elements. Codes relevant to OOP practice were developed based on the ACM Computing Curricula 2020. The codes represent Knowledge (fundamental concepts of OOP: class, inheritance, constructor etc.); Skills (Problem Solving, Collaboration, Design); Identity (Supportive, Leader, Expert, Quandary); Epistemology, Values (Compromise, Responsibility). Chat was manually analyzed to assign each message to an appropriate code. The coding process was appropriately validated.

In order to examine the types of connections made between different types of teams based on their performance, we considered their mean grade in the course (7.2/10) and their mean grade in the assignment (7.8/10). Students with a score above or below the mean course grade and teams with scores above or below the mean assignment grade were categorized as High (H) or Low (L) scoring, respectively. We had 3 different categories of teams based on members’ individual and team scores. There were 3 teams of H student course score and H team assignment score (H-H); 3 teams of H or L student course score and team H team assignment score (H/L-H); and 3 teams of H or L student course score and L team assignment score (H/L-L).

The following research questions were explored:
- What types of connections between codes are made by teams in the H-H category? What types of connections between codes are made by teams in the H/L-H category? What types of connections between codes are made by teams in the H/L-L category?
- Is there a significant difference between the discourse networks of teams in the H-H category and the other two categories?

The results suggest ENA provides information and affords new insights into the processes related to collaborative interactions in OOP assignment.

Keywords:

Epistemic network analysis, chat, collaborative learning, learning analytics, object-oriented programming.