Hear it in 3-D!

With a busy summer winding down, I’m staring down an equally busy fall. Unfortunately this summer produced fewer blog essays than I intended, as I was busy conducting interviews and developing other research for my doctoral project. I hope to remedy that in the coming months, as I prepare for another round of festival-going at TIFF and take on a teaching challenge at my alma mater, the University of Toronto, where I was offered a full-year course on the history of American filmmaking in the studio era. So, along with my usual thoughts on film style and technology, there will be plenty of film for thought to keep this blog busy.

Today, a few more thoughts on 3-D.

In transcribing some interview material from my conversations with sound practitioners, I came across an interesting point made by a notable re-recording mixer. In addition to finding the right balance among sound effects, this effects mixer is often responsible for placing sound effects in the 5.1 sound space. This means, for example, spreading explosions across the front channels or sending a variety of “bys” or passes from the front to the rear surround channels.

Before the re-emergence and hype surrounding modern 3-D presentations, 5.1 audio represented one way to immerse the audience in the space of a film. Sound mixers routinely envelop audiences in different sound spaces as a way of conveying the spatial geography of a scene; to provide clues to the location of a scene; and to embellish the sound signatures of a particular locale. Sound was a natural choice to convey immersion, since it fills the entire theater space with speakers not only behind the screen but along the side and back walls of all modern auditoriums. But now with 3-D all the rage, it seems that immersive 5.1 audio may not be immersive enough.

Speaking casually about sound technology and the aesthetics of modern film sound, this mixer — who has some experience mixing for recent 3-D fare — expressed frustration with the state of sound in relation to the 3-D format. Here’s the whole quote:

“At the moment I feel it is just a strictly visual experience. With 5.1 you can’t make it sound like special venue sound. In a standard theater, you just can’t do it. You try. You try to exaggerate surround. You try to get more special with things. But you can only do so much because you don’t have a speaker over your head. You don’t have a wall lined with them, like they do in some theme parks.”

With so few films being prepared for release in 3-D, the reality of this situation has yet to be felt by the majority of Hollywood re-recording mixers. Many have noted that norms and conventions have yet to be augmented to better suit 3-D because time, resources, and the small number of actual 3-D films prevent people in sound editorial and mixing to re-conceptualize sound style. While visual effects departments get the time and financial resources they need to refine and complete shots, sound departments are routinely told by post-production supervisors that there isn’t extra time or money for fringe benefits like reconfiguring the 5.1 layout.

The traditional press, along with the blogosphere, have had a lot to say about 3-D (including my own essay on the subject), but few have noted how the sound track might work in relation to eye-popping imagery. I naturally assumed that the 3-D sound track must be exploring new avenues of immersion, or responding in some way to the illusions of depth. If hardware firms like IMAX and Dolby are spending millions developing 3-D technology and studios are readying a growing number of 3-D releases, then surely sound must be part of the innovation party. Well…yes and no.

No one can say for sure what the future will bring, but at this moment the state of the 3-D sound track is unchanged. That is not to say that mixers aren’t working with sound differently than with traditional 2-D films. As the effects mixer noted, they will sometimes push more “hard” effects into the split surround channels to mimic an action that sends the image “into” the theater. Backgrounds (also known as ambiences) may also be treated with more gusto. In this sense, mixers are pushing more sound into the theater space to complement the visual push. It also helps that most 3-D movies has been animated, a genre which often affords mixers greater play with sound level and placement. Things can be more lively, full, and bright with films like Monsters vs. Aliens and Coraline.

Mixers are, therefore, tweaking current practices to suit the new image. They are not, as one might imagine, re-configuring the sound of sound. Why not? Well, a new delivery system for sound would be costly for exhibitors, who have already had to install digital cinema projectors to offer films in 3-D. A new sound system may involve not only new processors, but also more loudspeakers behind the screen and along the side and back walls. For years Tomlinson Holman has been arguing for 10.2 surround sound, which adds a pair of left and right overhead channels, a pair of wide left and right channels, a second subwoofer, and a center rear channel (which Dolby introduced in 1999 with the release of Star Wars: Episode One — The Phantom Menace). This layout would add the overhead channel desired by our effects mixer, and gives more latitude to the behind-the-screen channels to localize sound more precisely.

This report from Audioholics points out that psychoacoustic experiments suggest that human sound localization is far greater on the horizontal plane and front hemisphere than on the sides or rear. Unfortunately, the home theater industry continues to emphasize the importance of surround channels, and continues to add rear channels to the home array as some 7.1 systems demonstrate.

In Japan, NHK has proposed a 22.2 system to complement their super high-def television technology. Another re-recording mixer introduced me to a new sound system out of Germany called IOSONO. Here’s a brief excerpt from their description of its cinema applications:

An IOSONO system really comes into its own with IOSONO-encoded material. Sound designers can place up to 32 independent sound objects anywhere outside of, or within, the theater, either far behind the walls or right next to any member of the audience. What’s more, these sound objects can be made to move along any given path, at any desired speed. Ever experienced a helicopter slowly flying into the middle of the theater and hovering right above your seat? With IOSONO, you can.

Here is a description of the technology itself:

A computer controls each loudspeaker separately and actuates it the moment the desired wave front would pass through it. To synthesize a spherical wave front originating from a point behind the speakers, for example, the speaker closest to the virtual source is actuated first, followed by the speakers to the right and left of it. This results in a wave front with a relatively large radius and a virtual source point outside of the listening space. Reversing the order of actuation (where the speakers closest to the virtual point source are actuated last) results in a wave field corresponding to that of a source within the listening space.

The result is a stable wave field in which the listener can localize the virtual sound sources as if they were emanating from actual objects. The loudspeakers themselves, however, cannot be localized. Wave field synthesis thus creates a stunning illusion of acoustic events in a space, adding a whole new dimension to audio in the entertainment and other industries.

Beyond the marketing language of the IOSONO system, it is easy to see how this type of application could be attractive to Hollywood sound mixers who seek to augment the soundscape of a 3-D film by adding more localized channels. As the effects mixer stated, 5.1 is not broad enough to pin-point sound in space. In fact, several mixers agree that 5.1 is a digital compromise between exhibitors and industry practitioners. More channels may represent greater creative control, but it inevitably costs more. And, according to Tom Holman, 5.1 achieves the minimum number of discrete channels required for an immersive sound field.

In practice, mixers often avoid pin-pointed sound in 5.1 sound space because of its potential to distract audiences from the screen. Which is why many supervising sound editors and final mixers aim to fill the rear channels with rich but undefined backgrounds. Very rarely is dialog placed in the rear for the same reason. With 3-D, mixers are faced with an image that calls attention to itself, so why can’t sound do the same thing? If an arrow is shot out from the screen and lands somewhere to the left-rear of the viewer, why not indicate the arrow hit with a sound effect in the left-rear of the auditorium? As I mentioned earlier, mixers already use the rears as transport channels for fly-bys or car-bys or other moving objects.

As much as mixers are frustrated with the financial constraints to 3-D sound, there are some theoretical issues that still lurk in the shadow of 5.1. Mixers may want more channels capable of reproducing localized sound, but they must first overcome the conventional logic of surround sound mixing: avoid localized sound in the far left, far right, and rear. Tom Holman once quipped, “In Top Gun, when jets fly left to right across the screen and then exit screen right, what may be perceived aurally is the jet flying off screen as well, right into the exit sign.”

That is why Holman, among others, has opted for immersive film sound not localized film sound. Unlike a theme park ride which often directs your attention through sound cues placed in a 360 degree fashion around a room (think of the Hall of Presidents in Walt Disney World), cinema sound must contend with a two-dimensional screen on which audiences must stayed focused, even with 3-D presentations where your eyes remain fixed on a general axis, where any movement outside that axis might reveal the images to be cardboard cutouts — a phenomenon all too familiar to me.

The call for special venue sound for 3-D presentations seems to be an unlikely reality given the cost and small base of films released in the format. The desire for special venue sound also hides a fundamental aspect of Hollywood filmmaking that James Cameron continues to emphasize, even as he touts his upcoming Avatar as a veritable “game changer” in the way we experience our movies. In an interview with the Daily Mail he stated,

The irony with Avatar is that people think of it as a 3D film and that’s what the discussion is. But I think that, when they see it, the whole 3D discussion is going to go away…That’s because, ideally, the technology is advanced enough to make itself go away. That’s how it should work. All of the technology should wave its own wand and make itself disappear.

He is emphasizing story clarity and intelligibility, two of the most fundamental building blocks of American cinema. As much as the technology can wow our eyes and ears, the experience is in service to something else: the story. So as much as Cameron is prepared to awe his audience, he’s acutely aware that the illusion will fail if the audience isn’t taken on a journey that means something more than eye-popping visuals.

Filmmakers, including sound professionals, have always had to reconcile the spectacular nature of technology with the need for narrative invisibility. This is especially the case with sound mixing, where their art is based on the fine balance of story comprehension and environmental immersion. It is, therefore, hard to imagine sound acting any other way than it currently does in 3-D environments, especially if directors like Cameron subscribe to the story-is-paramount ideology.