Abstract:

In our paper, we shall analyze an occurrence of “doing realizing” accomplished by two university students who are jointly engaged in a tangible-user-interface-mediated bicycle physics problem-solving activity. Tangible user interfaces (TUI) allow co-located participants to simultaneously interact with digital information by making use of widgets to interact with the system. Widgets consist of a graspable object participants manipulate to interact with the system and a visual feedback element. Here, the problem-solving scenario is a computer simulation of a biker’s applied force and work done to the bike while driving over different road surfaces, climbing diverse hillsides, travelling over various distances, and while changing pedal and wheel gears. Participants are asked to solve several tasks to eventually find out which factors influence the dependent variables “Force on Pedal” and “Work”.

To evaluate the prototype of this bicycle scenario and to gain insight into the situated organization of joint problem solving processes, we videotaped three small groups of students while they were interacting among themselves and with the tangible-user-interface. By relying on a conversation analytic framework to conduct a moment-by-moment video based analysis, we shall draw here on a single case to highlight how two students accomplish the interactional work of realizing that a specific factor influences the force applied on the pedal.

Through our turn by turn analysis of the multimodally embodied conduct of two adult students engaged in a TUI-mediated joint activity, we reconstruct how they interactionally accomplish doing realizing in situ. While constantly displaying a joint orientation to their task, they are doing a multimodally embodied work of noticing, of directing the other’s attention, of seeking and of securing mutual understanding. Finally, a new insight is introduced in their shared world. More specifically, they do this in six steps: In response to collaboratively raised doubt (step 1), they get ready (step 2), establish and follow a trialing procedure (step 3), find out that one variable is influencing the target variable (step 4), verify their finding (step 5), and finally one participant extends the jointly displayed understanding by referring to an experience outside their current setting.

More particularly, our analysis points to the significance of multimodally embodied conduct. Indeed, participants’ actions are not organized solely in talk but through the simultaneous use of multiple semiotic resources. For example, during the first step of the analyzed phenomenon, doubt is expressed through an inter-elaboration of the first participant’s uttered question and a halting hand movement. The so cast doubt is co-elaborated through the second participant’s hand movements mobilizing the relevant widgets for the next step, whereas her verbal utterance accounts for a delayed response.

Our analysis also raises some challenges for designing TUI-mediated activities. Prior to the participants casting doubt (step 1), the system did not react ‘properly’ due to a design slip and a temporarily malfunctioning widget. Thus, in a sense, the doing-realizing-episode was triggered by a breach in the system. Hence, it is worthwhile to further investigate what kind of TUI-mediated breaching behaviors may trigger moments of uncertainty with ensuing coping procedures allowing the participants to gain new insights regarding a topic matter.

Keywords:

Tangible user interface, joint activity, conversation analysis, doing realizing.