Part 2

Translating Sound To Image

The spectrum of possible translations of a work from one mode to another stretches from the completely subjective representation of one's own personal experience with a work, to completely automated algorithmic translations. It is clear that no one approach is best across the board, and in fact, it may be that it is best to have a combination: one translation that offers a direct experience of something that reflects the actual material of the piece in some way paired with one or more subjective reinterpretations.

Beyond giving examples of the issues that complicate the translation process, and attempting to point out some of the pitfalls one might encounter as I have attempted to do, there is only so much one can say about the subjective reinterpretation process. But I think it is very much worth looking more closely at the possibilities and challenges of systematic translation processes because even partially satisfactory solutions could potentially have a big impact on overall accessibility. This is particularly true for work delivered across the web, as it is already in some sort of electronic form, and is therefore theoretically already prepared for processing.

It seems to me that the area of greatest potential for this sort of translation is in the realm of translation from sound or music to moving image. While I am far from sure about well this might actually work, this intermodal translation seems to have some factors which make it potentially ripe for this sort of approach. The bulk of this inquiry will focus on the issues, challenges and possibilities I can see for this.

Clearly sound can be digitized into information, and once it is in this form, it can be translated into images in any number of ways. But it is not sufficient nor satisfying to think in terms of a literal presentation of sound data in a visual form. It is not necessarily true that any translation is better than nothing. If the heart of the work is not present in the translation, then the effort of producing and digesting and deciphering the translation is wasted effort.

Music and works of sound art are almost inevitably at least in part constructed with a listener in mind, and take the nature of the human auditory system as a major consideration. This tight relationship with a particular sensory modality implies that this translation must carefully account for that manner in which music is customarily perceived and the ways that images are customarily perceived. On the other hand, it is important to take a look at the question of what exactly music might be.