The Sight Of Sound

Or how your brain needs your eyes to help trick your ears into hearing what’s not there.

- Zachary Chan

In the past few months, I’ve experienced two different audio technology demos that have given me more insight into audio processing than I have had in my entire career. And I’ve come to the conclusion that you really have to “see” audio in order to believe in the experience that you’re hearing.

Of course, this concept isn’t really new. Think of blind taste testing, or the popular mystery box game where people stick their hands into a box and try to guess what they’re touching. As humans, we need visual confirmation of the input from our other senses in order to process it.

I knew this of course, but I misjudged the extent of the difference that sight brings to the table. This is why it’s so hard to appreciate audio technologies such as surround, binaural, 3D or physically-based sound. You can read about it and understand what is happening, but you may not always be able to put a finger on what you’re hearing.

Many surround sound systems try to mimic multiple audio sources to trick you, commonly called HRTF processing, or Head Related Transfer Function. Object-based surround sound like DTS:X and Dolby Atmos take the concept further by treating sound like an object that can be moved around in a 3D space, so you don’t just hear sounds front, or left, or back, but all around around you.