I went through some photos the other day, which made me think of a very happy period of my life, which I feel like sharing some thoughts about, because the subject of my work from that time is still very interesting to me.
Radio France, Paris
I was an intern at Radio France in Paris in the spring of 2014. Radio France is a massive organization, where a lot of audio history has sprung from. Pierre Schaeffer, Pierre Henri, Musique Concrète, Groupe de Recherche de Musique and the recording technique ORTF being some. ORTF was the short for the old name of Radio France, Office de Radiodiffusion Télévision Française.
I was lucky to work with Hervé Déjardin, a really interesting guy who lead the current research team at Radio France within the field of spatial sound e.g. multi channel mixing, binaural.. crazy projects with a lot of loudspeakers and the perception of sound in a space. Hervé and his colleagues create many good sound experiences and make them available at www.nouvoson.fr
Internalization / externalization
My work at Radio France was part of a research project in the ‘Internalization / Externalization’ of audio events while binaural listening. Internalization meaning the
sensation of perceiving an audio event as originating from within the listener’s head. One could say the localization of the auditory event being either inside or outside the listeners head. I will mention audio events later in this post. It basically just means a short something that happens, e.g. a single hit on a snare drum, or the sound of someone entering the room and closing a door.
The work also included trying to read and understand research literature and translate what I found into usable mixing techniques, made easily accessible for people who are more keen about doing than reading, and hereby looking into sound research papers e.g. some sound designers, mixing engineers etc.
The reason for this, is that I feel some some researchers and authors of papers I’ve read, to eagerly wanting this concept to be overly complicated. The way all the signal processing works is probably quite advanced (being a sound designer and not an engineer I don’t really care), but the concept is not, so don’t loose your breath! 🙂
There is a lot of information online if you want to geek out about binaural sound. There are some concepts which are good to know – Interaural level difference, – time difference, Head Related Transfer Function (HRTF) etc. It’s a read so I’d prefer to reference other sources e.g. this one from CIPIC and this one.
The concept of binaural sound production is the idea of capturing or re-creating an auditory space with all the perceptual factors required for e.g. correct determination of direction and distance to an auditory source. A source could be a guitar playing, a voice speaking or a car driving by. The latter is very interesting because it is an moving object. Also because the perceived auditory event (‘sound’) of a car consist of much change in how the spectrum and ‘volume’ develops – e.g. the auditory events spectral and amplitude envelopes are complex.
All of this assist to a greater precision in determining distance and direction to the source – a nice thing to think about when doing sound design / mixing, because this information can help one place sources in a composition 🙂
This concept is not new many people have experimented with recordings using artificial ‘dummy heads’.
Binaural recording – dummy heads
One can perform a binaural recording, placing microphones inside (or close) to the sound recordist or dummy heads ears. This recording will of course contain whatever has physically happened in the room / place in which the recordist was located, but it will in fact also contain information about the anatomy of the recordist, since spatial information or the way we humans perceive direction and distance, is personal to each individual, and is described as the Head Related Transfer Function or HRTF.
I have great success using a set of miniature DPA4060 microphones, and place them just behind the Tragus (the fleshy part, at the entrance of the earcanal). It is important to get them close to the earcanal, because the entire shape of the (among other things) outer ear has influence on localization.
A good reference to binaural audio is Jens Blauerts ‘Spatial Hearing’ (although the pages seem to fall out quite quickly - a shame when it’s bought on a student’s budget! 😉
Binaural – the virtual surround sound
Many commercial companies aim to sell their product on ‘virtual surround sound’ or ‘3D sound in headphones’. The basic concept is the before mentioned. The understanding of human perception of sound in a space with all what it includes of localization (direction, distance etc.), and then reproducing this sound experience while listening using headphones.
One of the reasons I think the people at Radio France and their many collaborators are really interesting, is because they work with the subject of using synthesis to artificially recreate the perception of a 3 dimensional auditory space. This basically mean that you can record the way one specific person hears, by recording the impulse response of that person’s anatomy that influences the perceived spatial information the persons HRTF.
HRTF – Head Related Transfer Function
One basically put a pair of microphones at the entrance of one’s ears, playback noise bursts or a sine wave sweep, record the audio with the microphones, and using some clever software to extract the difference between how the noise sounded coming out of the loudspeaker and how it sounds in the recording to get an understanding of how that unique person perceives an auditory source from a specific direction. Then you move the loudspeaker playing noise to a new position and go through the procedure once again.
After finishing a lot of measurements from a lot of different positions, one can combine all these measurements into what is known as a Head Related Transfer Function (HRTF), for that unique person, which can be used in a software, to artificially move a source around a listener’s head. One software which allow you to do that is Ircam SPAT which one can pick up from Ircam Forumnet and use inside Max/MSP, or as a VST plugin from the software developer FLUX. One has to be aware though, that the VST version can act a bit unstable, so don’t make your live-set depend on it 🙂
The sum up
I worked at Radio France on a binaural research project, with thesis that the human ability to perceive distance and directionality of an auditory event is not stimulated, when music, sound design, radio or similar is presented in headphones using traditional (non-binaural) stereo or mono format.
Modern technology such as binaural synthesis provides a theoretical foundation for reproducing the way humans hear, but actual tools such as mixing techniques needs to be derived from research literature, and made available for mixing engineers, musicians and the general public within the audio production community, for stimulation towards a more widespread production of content in the binaural format.
The candy, tips’n’tricks!
If you want a stronger binaural listening experience, then use headphones with head tracking. You want your binaural landscape to keep a fixed position in space, when the listener moves his or her head. Wireless headphones are preferred, but my friend Louis and I made some demo’s using wired headphones and a DIY headtracker.
Fingers off the cookie jar.. jar!
Binaural stereo is basically just a stereo signal with strange things happening in regards of phase, amplitude and filtering differences between left and right channel (ear, when using headphones). This mean that a binaural signal is sensitive to any kinds of filtering that change the phase, compression, harmonizing etc. These effects should be applied on a synthetic signal before the binaural encoding.
Mixing binaural and traditional stereo
Since binaural is also ‘just a stereo signal’, one can with great success blend binaural stereo, with nonbinaural encoded stereo. Meaning, if you want to keep punch in snare drums and bass sounds, then keep them e.g. centered mono or i traditional stereo, and if you have spheric sounds that need a lot of space then encode it to binaural, and mix the signals.
Mono != inside the head, yo!
Some literature describe that any sound which is mono will naturally seem like appearing inside the listener’s head, when played back using headphones. This is in my opinion definitely not true. The sound designer have to work with filtering, level and possibly the semantics of the source to actually succeed in internalization. Especially check out what happens when you play around with the lower mids (around 250 Hz), for the sensation of internalization.
There is a lot more to write, but this has already become quite a read, so get in contact if you want a chat about binaural – I will be more than happy to share more thoughts 🙂