David Rokeby : The Harmonics of Interaction
 

(published in MUSICWORKS 46: Sound and Movement, Spring 1990)

While the 'sound' of the system and the 'dance' of the person within the space are of interest to me, the central aspect of the work is neither the 'sound' nor the 'dance'. It is the relationship that develops between the sounding installation and the dancing person that is the core of the work.

(In one piece there was a sound you could only find if you walked as though you were carrying a 40 pound weight.)

The installation watches and sings; the person listens and dances. But the relationship that develops is not simply that of a dialogue between person and system. Dialogue in its back-and forthing implies a separation of the functions of perceiving and responding. But for the installation, perception and expression are virtually simultaneous. As a result the installation and participant form a feedback loop which is very tight, yet very complex...

The installation when empty stands as a silent awareness, a fragment of a nervous system: cameras for eyes... computerized visual cortex... in software a complex set of filters and translators (a set of opinions, a rudimentary aesthetic?)... synthesized vocal cords... speakers for mouths...

With first movement of a body within the perceptual range of this system, this synthetic awareness is engaged and begins making sound.

The sound propagates through the air, enters the ear (and body) and travels through the transformations of human acoustic perception, the labyrinth of the individual's brain, and the tangled interface (psychological and physiological) linking mind and body, to manifest itself as further movement (or lack thereof) which completes a cycle and starts a new one...

But the cycles blur into one another because as soon as one moves in response to the sound of the system, the system is already modifying its response to reflect those movements and so on, so that body and system are drawn into constant readjustment and a sort of oscillation begins.

The minimum time delay of this loop is about 1/30th of a second. But the real delay depends on the exact path(s) through mind, body and installation. Both the system and the person operate as prisms, splitting the 'beam' of feedback into different components, travelling at different speeds, oscillating at different frequencies...

And so, just as acoustic feedback exaggerates aspects of the acoustic nature of a room, the interactive feedback loop in my installations reinforces particular aspects of the system and the person in it, producing resonances. These can be thought of as the harmonics of the interaction and are unique for each different combination of installation and individual. (These harmonics often come as a complete surprise to me; I have sometimes observed people creating sounds and patterns in interaction with the installation which I, knowing the program intimately, would have thought impossible.)

Because of the overall complexity of the system, there are many factors which determine its resonant characteristics. These factors can be grouped as functions of the body, and the character of the person in the space, and the sound, and the responsive behaviour of the installation itself.

Each body has its own resonant frequencies which are functions of size, shape, weight and musculature. Rhythms inherent in the interactive program interfere with and/or reinforce these. (It is interesting to watch a large number of people on a dance floor, each striving to find a way to bring their uniquely shaped body into resonance with the rhythms of the music. Each different popular dance style seems to have evolved out of the need to configure the body in such a way that it can resonate to a particular kind and rate of rhythm.)

The nature of the sounds that I use in the installation have a less direct, but no less powerful impact on the interaction. People seem to endow a sound that has a lot of texture with an external material reality, find themselves imagining the feel of it against their body, imagining the space filled with sound particles. Rounder, deeper sounds seem to manifest themselves more as a presence within or an extension of the body, something of the stomach, organs and muscles rather than of the skin... The body tends to adopt different relationships to the installation depending on whether the sounds are felt on the inside or the outside, and each relationship has its own harmonics.

Within the system's harmonic spectrum, different ranges are brought into play as the interactor responds with different degrees of mental reflection and spontaneous action. If they adopt a predominantly intellectual approach to the installation, they tend to reject immediate participation in the feedback, working more like a scientist, making experiments and 'objectively' observing the results. The aim of such an approach is often to learn to control the system in order to be able to play it like an instrument. In this sort of situation, the system resonates, if at all, at very low frequencies, because the person shapes their movements over time in response to a slowly accumulating body of experience. (The intellect operating as a low pass filter.)

(The body tends to betray those who try to adopt a purely intellectual approach with the aim of exerting control over the system. The first test that such people apply to the system is 'If I make the same movement twice, will I get the same sound?' In some of my simpler interactive pieces, one will get the same sound for exactly the same movement. Within such an installation, a pattern emerges. A person makes a gesture, as though asking a question of the space and notes the system's response. They make a second gesture and a third, again as questions, and both times receive the same response as the first. At this point, they pause, a change comes over their face as if to say 'Yes I understand I have control', and they make a fourth gesture, no longer as a question, but as a statement, almost a command. And they almost invariably get a different sound, the shift in attitude manifesting itself in a subtle but noticeable shift in the carriage of the body and of the instant-to instant dynamics of the gesture.)

If, on the other hand, they allow themselves to respond spontaneously to the music of the system, it is they who are played by the installation. This approach involves opening oneself to suggestion, allowing the music of the system to speak back through one's body directly, involving a minimum of mental reflection, and thus tightening the feedback loop as much as possible. This sort of situation reinforces the higher frequencies and dampens the lower ones...

Among those who approach the installations in this manner I often observe people taking responsibility for perceptual errors that the system's software makes, unconsciously justifying these mistakes by deflecting their intended gesture into harmony with the system's response; conversely, they will often attribute manifestations of their own intelligence to the system. (Having unconsciously fallen into synch with a rhythm in the music, people often express amazement at the system's ability to synchronize to their movements.)

Of course these are gross generalizations, and everyone who enters the system employs a combination of each approach in different measure. My ideal scenario is one in which the two approaches are balanced, resulting in a 'broad bandwidth' of interaction.

When I design the programs that define the reactive behaviour of the installation, I work somewhat like an sound engineer at a amplified concert who uses filters and other distorting devices to control the feedback of the sound system. However, my aims in doing so are different. The engineer seeks to cancel the acoustic idiosyncrasies of the room and system so that they don't distract from the performance, and to eliminate feedback. I filter and distort in order to highlight the feedback, tuning it to reinforce the hidden physiological and psychological resonances of the body and mind in physical interaction.

The music of the 'Very Nervous System' installations is not so much in the sounds that you hear, but in the interplay of resonances that you feel as you experience the work with your body.

Copyright 1996 David Rokeby / Very Nervous Systems / All rights reserved. 3/7/96