Rokeby : Home / Installations / Current Shows / Texts / softVNS / Links / e-mail me

Erkki Huhtamo

Silicon remembers Ideology,

or David Rokeby's meta-interactive art

(from the catalog for "The Giver of Names" exhibit at the McDonald-Stewart Art Centre) 


When David Rokeby handed me his namecard, some ten years ago, it read: "interactive artist". I wonder if it still does? During the intervening years the connotations of the word "interactive" have undergone a dramatic change of emphasis, at least if related to the changes in the cultural frame of reference. From a simultaneously appealing and suspicious oddball, interactivity has within a short span been promoted to the sweetheart of the mainstream media of the 1990's. "Interactive" has become a label, a sticker which seems to have the magic power to transform anything - make it more sexy, more potent, more creative, in a word: a better purchase. Unfortunately, even the art world (usually so resistant to new media and technology) has shown some signs of succumbing to the epidemic.

This should not be misunderstood: I have not turned from a supporter to an enemy of interactive art. Yet, I think that the time when the epithet "interactive art" per se was a recommendation (at least for those who were open to it) is over. We have seen quite an avalanche of "interactive artworks" in recent years; a few of them even in the temples of high art. Although there have been remarkable and innovative creations among them, we must face the fact that interactive art is not inherently more interesting than any other artform, or some other kind of application of interactive technology. Although the exploration of a new and unknown medium is (and will always be) a commendable endeavour, it does not automatically produce exciting results. It has to prove its value, like any other activity.

This being the situation, I strongly feel that interactive art has important tasks ahead: not merely to prove its real value as a form of art and creativity, but also to initiate a critical discourse about its own underpinnings. There are too many "interactive artworks" that iterate popular clichés about interactivity, without really questioning their premises. Does interactive media always "empower the user" and promote "freedom of choice"? Does it "release" the user from captivity in one's material body? Is it somehow inherently more intuitive and responsive than any other form of media? Does an interactive "multimedia" experience automatically lead to heightened perceptual and cognitive activity, producing mentally more coherent instead of more distracted subjects?

Interactivity is clearly at risk of becoming just another ideology, echoing what happened to television. The promises of immediacy, transparency and democratic feedback that were associated with early live television were rapidly turned into mere simulations, powerful means to condition television audiences by service providers and advertisers. Something similar is happening in the field of interactive media, as exemplified by the huge commercial success of addictive games like Doom or Quake, or the rapidly advancing re-definition of the Internet as a global shopping mall. Although it is probably true that many users of interactive media (like at least some television spectators) will develop critical and even oppositional ways of using the media, "interactive artists" could affect this process simply by becoming more conscious about their own doings.

This brings me back to David Rokeby, one of the handful of artists on whose namecards, I feel, the title "interactive artist" (whether he still uses it or not) is fully justified. Rokeby has spent almost two decades developing interactive systems, software and hardware. The engineering side of his activity has, however, been inseparable from his ambitions as an artist. Rokeby's numerous exhibitions and demonstrations have served as a testing ground for his creations, contributing to further refinements of his system. But, perhaps even more importantly, Rokeby has never been content with his impressive technical and aesthetic achievements. He has also struggled to conceptualize and to contextualize his own activity, constantly (re-)defining and problematizing its relationship with contemporary media culture in general, and interactive media in particular.

Rokeby is neither an opportunist nor an anarchist; his attitude is critical, but constructive. Besides offering challenging aesthetic experiences, his artworks can be interpreted as discursive explorations, invitations to ponder the changing implications of interactivity. Rokeby is not a preacher. The discursive dimension is often subtle and may go unnoticed in the heat of the interaction. I believe, however, that many "interactors" get at least a vague feeling about the "surplus" these works contain beyond the jouissance of the interaction. It could be claimed that the discursive element has become more evident in Rokeby's later works, such as Silicon remembers Carbon and Watch. Some critics have gone so far as to detect a change of direction, perhaps even a rupture, in Rokeby's career. Where Rokeby's first chef d'oeuvre, Very Nervous System (1983-94), bore evidence of unrestrained enthousiasm, an almost orgiastic celebration of the possibilities of interactivity, the later works seem much more introspective, perhaps with a shade of pessimism or doubt.

I do not think that such an interpretation can be accepted without qualifications. The impression about the increased amount of introspection in the later works could be an illusion produced by the fact that the active physical (inter)action of the interactor, which was so essential to the experience of Very Nervous System, has been toned down. While Very Nervous system kept the interactor busy, there is now less to do (physically) and more time to observe the situation. This could be a sign of a certain resignation from Rokeby's part, but it might also be interpreted in various other ways. I think the key issue is to have a second look at Very Nervous System. Its very fame may have perpetuated some fixed ideas about its "nature" and prevented critics from appreciating some of its dimensions.

Indeed, most accounts about Very Nervous System have been fairly simplistic. Rokeby's creation is usually seen as an interactive sound environment, which is simultaneously a kind of virtual music instrument. The interactor's movements are detected by a video camera, functioning as the "eye" of the system. The movements are analyzed by a computer which controls a sound synthesizer that responds to the interactor's input by sounds. In most cases these have been musical sounds, originating as if from an invisible orchestra. Many newcomers (including myself), encountering the system for the first time, seem to develop a peculiar "rite of passage" which Rokeby himself has identified as "The First Test of Interactivity". The interactor waves his hand to trigger a sound. He then waves again, in a similar manner, to find out if the same sound will be triggered again. If something else is heard, the interactor may conclude that the system does not function well, or that it isn't really interactive at all.

Underlying such a "test" seems to be the widely shared notion that interactivity somehow presupposes an unambiguous one-to-one relationship between the interactor and the system. The reactions of the system should be immediate and (at least in principle) controllable by the interactor. Such a notion also implies the existence of a "map" and what I have called elsewhere a "pedagogical subtext" - the requirement that an interactive application should guide the user at every step and lead to gradual mastery over it. This idea of interactivity is a very limited one, yet it is the one that reigns in countless practical applications from electronic games to automatic teller machines. Although a videogame may seem to play hide-and-seek with the player, it is ultimately a closed system which has a closure, a point at which interactivity reaches its end.

Very Nervous System, however, introduces a far more complex notion of interactivity, taking it to its limits, but in a very different sense. Very Nervous System is an open system in the sense that Rokeby has avoided simple one-to-one relationships between the work and the interactor. This principle is built into its very architecture. Very Nervous System registers changes in the intensity of light (caused by the interactor's movements) by means of rapidly updated "raster images". It detects movements rather than, for example, discrete outlines (such as the different parts of the body). Because of this Rokeby has compared his detection system to a frog's instead of a human's eye. This solution makes the incoming information deliberately vague; correspondingly, the system reacts in very complex ways which are only partly predictable. Thus the interactor can never have an absolute control over the system. Rather, he enters into an on-going and evolving dialogue, a "cybernetic feed-back loop" without a final resolution.

All this creates an ambiguous situation, which can be read in different ways. Rokeby seems to dispense with the usual metaphor of "master and slave" in terms of which the human/computer encounter is often conceived. The interactor is not an absolute master, while Very Nervous System cannot be said to have real "(will)power" either. Has Rokeby, then, created a model for utopian interactivity, an open-ended "creative conversation" (to borrow Gene Youngblood's term) between two more or less equal partners, one biological and the other cybernetic? Wouldn't such a conversation eventually lead to the tightening of the cybernetic feed-back loop - a merger, cyborg logic? Rokeby perhaps hints at this possibility by making it often so difficult to tell who is controlling whom during the interaction. Human action is simultaneously centered and de-centered. Appropriately, Rokeby has compared computer interfaces with "distorting mirrors". In the context of his work, the invisible "figure" they reflect is potentially both liberating and distressing.

Although fully functional on the level of "raw", unmedi(t)ated interaction, Very Nervous System also has a strong "meta-interactive" element. It is a laboratory-as-playground, which is left at anybody's disposal. No house rules are given and no goals pre-determined. Its equipment make it perfectly suited both for sheer fun and to probing some of the most pertinent questions related to contemporary media culture. Because of my emphasis on the meta-interactive element, I do not see the difference between Very Nervous System and Rokeby's later works, such as Silicon remembers Carbon and Measure, as very dramatic. Rokeby has continued his exploration of the ambiguities and paradoxes embedded in interactive situations. In Silicon remembers Carbon the interactor's presence affects the system, yet it is next to impossible to learn the "code" governing the interactions. While the interactor may find out, for example, that crossing the bed of sand on which the images are projected triggers the sound of rippling water, it does not effect the image. In other cases the image changes, but how and under what conditions? I believe most interactors are left with the bewildering impression that there is a relationship between their actions and the system's reactions, but an opaque, impenetrable one.

In Watch, Rokeby's attitude towards interactivity is even more ambiguous. The instantaneous one-to-one relationship between humans and the system, so central to Very Nervous System, is there but displaced almost beyond recognition. The visitor is re-defined as an observer, like a guard keeping an eye on surveillance monitors, or the domestic TV spectator. Instead of any conscious interactor, the real-time detection system registers the movements of unsuspecting people (passers-by on the street) and things (leaves of the trees, raindrops). The images are seen in the gallery on two adjacent, yet mirrored screens in enigmatic, transformed form (due to Rokeby's complex treatment of them, based on light changes in the scenes). Figures appear and disappear, they seem to move faster or slower, their outlines get sharpened and blurred in turn. At least in some versions of Watch, the observers occasionally see their own altered images flashed on the screens, captured by a video camera hidden in the gallery space.

Silicon remembers Carbon and Watch scrutinize aspects of interactivity, which are often left unnoticed, or perhaps even deliberately suppressed. On one level, I see Silicon remembers Carbon as an invitation to re-consider the relationship between cybernetic interactivity and those, often banal, processes of interchange that take place in our daily lives - our relationships to all kinds of technical, conceptual and "natural" entities surrounding us. Our daily existence is only made possible by such interchanges, many of which are "automated" - we hardly pay attention to them. Should we become more or perhaps less conscious about our intercourse with interactive systems? Should interfaces develop towards more and more perfect transparency, or should they, instead, emphasize their role as pre-coded constructions? For Rokeby the nature and the role of the interface are key issues. Even if an interface may be immediate and transparent, it is never neutral. It imposes its own restraints on the process of interaction. It becomes a question of power and ideology.

By referring to video surveillance and the imbalance of scopic power in contemporary media culture, Watch raises the issues of agency and control. On the one hand, it deliberately restricts the possibilities of the visitor to affect the situation by replacing the physical interaction with an invitation to stop and look deeper into the image. On the other hand, it places humans (the passers-by) in a position, in which they both trigger a cybernetic system and are (potentially) "triggered" by it without being conscious about their role in the process at all. If interactivity implies an active involvement, a conscious interplay with a cybernetic system, both of these subject positions (the observer/the observed) could, indeed, be characterized as antithetical to interactivity. Arguably they are the reigning ones in a society dominated by broadcasting and surveillance.

Does Rokeby really claim that there is no alternative? I think he rather suggests that the road to a "culture of interactivity" is much longer than the self-proclaimed media prophets are ready to admit. Fancy slogans alone do not transport us to any interactive paradise; on the contrary, they may contribute to making us an easier prey for false promises. What is needed is a lot of work, but also a change of consciousness, an ability to discern the outlines of the hidden media machineries that purport to control our lives, sometimes in the guise of interactivity. Maybe Rokeby has, indeed, come to think that the seductive exctasy of interaction with Very Nervous System is not enough; a more determined look behind the scene is needed. Such a goal is always at risk of becoming didactic; even Watch makes one think if underlining the comparison between the image of the passers-by and that of the visitors themselves is really needed. As usual, Rokeby solves the problem by adding complexity. While persuading the visitor to become a peeping tom, the system simultaneously manipulates the images making it impossible.

Against this background Rokeby's latest project, The Giver of Names, raises great curiosity. Although a full analysis has to wait until the system will be more developed, I will venture some preliminary observations. Years in the making, The Giver of Names is an artwork which recognizes and verbally describes material objects. A visitor chooses any object and places it on a pedestal. A video camera observes the object; the visual information is passed on to a computer. Utilizing a complex knowledge base (mostly designed by Rokeby himself), the system proceeds by associating the input to a huge amount of data. The outcome of the process is a metaphoric statement about the object in question. In an early experiment, The Giver of Names for example described a small yellow rubber ducky as "Semicircles, so assymetric that ill-proportioned pears occurred to their informed bodies, can demonstrate no second edible fruits." A small disney female duck figurine: "You were lecturing on changed eggs."A small yellow volkswagen beetle car toy: "Lemons, more eyeless than other beady sectors, would pardon no optical drops."

Although The Giver of Names is based neither on artificial intelligence nor is it a full blown "learning system", it easily associates with the tradition of the automata - the (semi-)autonomously acting, speaking and "thinking" machines. Harold Cohen's painting robot Aaron or Joseph Weizenbaum's fake artificial intelligence ELIZA easily come to mind (although the programming principles and goals of Rokeby's creation are entirely different). I do not think Rokeby has any interest in "anthropomorphizing" his creation. Neither is he interested in emphasizing its "miraculousness", marketing it as a technological wonder (it would certainly have potential for that). He plans to exhibit it "bare", so that the individual components of the setup, the video camera and the computer, are clearly visible as themselves. Besides the room will only contain junk: potential objects to be described.

If much recent work on interactivity has emphasized the human side of the interface, Rokeby resolutely highlights the other side. Where Very Nervous System emphasized the interactor's constant physical activity, The Giver of Names only requires occasional human intervention - choosing an object and placing it on the pedestal. Limiting the interactor's input seems to be in line with Rokeby's other recent works. Like the invisible cybernetic "creator" of visual metamorphoses in Watch, also The Giver of Names functions largely on its own. On the other hand, in spite of the apparent differences, it is not difficult to detect connecting links between Very Nervous System, Rokeby's first major project and The Giver of Names, his second. Both aim at developing the recognition and interpretation capabilities of a cybernetic system, although humans have been replaced by inanimate objects as input.

On the other hand, Rokeby has already years ago (when describing Very Nervous System) pointed out "contrariness" as one of the main impulses behind his work with the computer: "Because the computer is purely logical, the language of interaction should strive to be intuitive. Because the computer removes you from your body, the body should be strongly engaged. Because the computer's activity takes place on the tiny playing fields of integrated circuits, the encounter with the computer should take place in human-scaled physical space. Because the computer is objective and disinterested, the experience should be intimate."

Although some of these goals can be applied to The Giver of Names only with difficulty, if at all, the main objective, the pursuit of "contrariness", is still the same. Beginning from the fact that he has worked throughout his career alone and independently on a field mostly characterized by group work and corporations, Rokeby has always run counter to the popular consensus. There is a thread running through his entire artistic output, although his individual works have also reacted to changes in the state of the media. This makes his own opinion about the change of emphasis between Very Nervous System and the Giver of Names understandable: "I feel as though the transition from Very Nervous System to the Giver of Names is a transition naturally paralleling the shift in the sense of what was being most challenged by the computer. In the 80's it seemed to the the material body. In the 90's it seems to be the notions of intelligence, and consciousness."

© Erkki Huhtamo 1998

Erkki Huhtamo is an unaffiliated media scholar, writer and curator. He has published widely about media culture and media art, directed television programs and organized several international exhibitions of interactive art. He lives in Turku, Finland.


Rokeby : Home / Installations / Current Shows / Texts / softVNS / Links / e-mail me

Copyright 1999 David Rokeby / Very Nervous Systems / All rights reserved. 5/18/99