To get a sense of whether a more complete translation is possible we need to start by looking comparatively at the human sound and vision systems and how they carry their content.

Introduction to Hearing

Harmonic theory

In order to fully grasp what the ear is doing, it is necessary to understand something about harmonic theory. Musical tones have a clearly identifiable pitch, a particular frequency of vibration. But any such tone is actually built up of a collection of vibrations at different but related frequencies. The frequency we hear as pitch is called the "fundamental". We call the additional pitches that are part of the tone "harmonics". Harmonics have a strict mathematical relationship to the fundamental. The frequency of a harmonic is always a perfect multiple of the fundamental's frequency. And harmonics always have the simple waveshape of a sine wave. The first harmonic is a sine wave with a frequency of exactly twice the fundamental. The second has a frequency of exactly 3 times the fundamental, etc. A full set of harmonics would have multiples of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12... up to infinity. This set of frequencies is called a harmonic series.

Any particular musical tone has a specific mixture of harmonics. Some tones, like those of the flute, are dominated by the fundamental, with small amounts of higher harmonics. Others, like trumpet have a greater presence of higher harmonics, and the result sounds "brassier". Reed instruments, like the clarinet, have a predominance of even harmonics (2, 4, 6, 8...) and that gives a clarinet its characteristic sound. In naturally occurring sounds, the strength or amplitude of the harmonics decreases significantly as we ascend the harmonic series. The vowels of human speech are all defined by specific harmonic profiles.

In the following demonstration, you can adjust the levels of the first 8 harmonics of a harmonic series by clicking and dragging on the red bars. To generate a sine wave, press '0'. To generate a sawtooth wave, press '1'. To generate a square-wave, press '2'. To soften the timbre, press 's'.

This browser does not have a Java Plug-in.

Sounds without an identifiable tone do not exhibit this sort of harmonic coherence, but can likewise be seen as mixtures of many different sine waves as different frequencies.