Abstract:

Musical gesture draws upon a close interaction between human perceptual and motor systems, resulting in a unique gestalt that conveys emotion. This is important in improvisational music therapy where the interplay between musical sound and body movement is a main medium of the therapeutic relationship (Trevarthan and Malloch, 2000), and a prerequisite to the understanding of the client’s feelings.

This paper reports on the development and clinical evaluation of a video-based, dynamic, sonification device which tracks gestures of music therapy clients with severe physical disabilities, extracts their characteristic features and converts them to dynamic sound. Of particular interest was the extent to which the dynamic properties of the technology could assist in a better understanding of the relationship between movement and emotional expression. Because of that, the design of the system utilised real-time sound synthesis providing a framework for dynamic morphologies, which although constrained within the aesthetic parameters of the synthesis algorithm, could produce sonification specifically related to the momentary action of each individual movement. Informing the design was the concept of dynamic orchestration (Paine, 2007). Here, not only was the amount of movement and location of the centre of the movement mapped to synthesis parameters but also the underling voicing structure of the instrument could be changed dynamically in relationship to changes in the measure of motion detected.

Participants were five young adults with severe physical disabilities, including quadriplegia, paraplegia, hemiplegia, of spastic and athetoid forms, and a degenerative motor disease of unspecified origins. Each participant took part in 8, half-hour individual session conducted at the music therapy at the University of Western Sydney. During each session several sonic patterns were trialled and adjusted according to each participant’s movements and preferences. The following video clip shows an excerpt from the trial: http://www.youtube.com/watch?v=VFnsqt_NZFo

Results showed that providing dynamic, musically-based sonification feedback to clients increases their vocabulary of movement responses, sustenance and the level of engagement with the music therapist, all of which can be considered to be beneficial outcomes of music therapy. There were also indications that the spaciousness of the generated sound influenced the spaciousness of movements, and that the two were in positive correlation.

The development of a suitably sound synthesis engine for a project of this nature is in itself a difficult task. In the present project the system was designed as a free improvisation system, as task oriented activities were deemed not appropriate for the participants. Further development may include high-level analysis units, reflecting aspects of posture, such as expansion and contraction, as a further approach to constraining sound synthesis outcomes to reflect not only the gesture of a particular limb, but the way in which the generation of that gesture causes variations in the entire body.

References:
Paine, G. (2007). Sonic Immersion: Interactive Engagement in Real-Time Immersive Environments, Scan: Journal of Media Arts Culture, 4 (1).
Trevarthen, C. and S.N. Malloch (2000) ‘The Dance of Wellbeing: Defining the Musical Therapeutic Effect’ in Nordic Journal of Music Therapy, 9, 2: 3-17.

Keywords:

Dynamic, Sonification, Improvisational, Music, Therapy.