Object-Based Audio Mixing Seeks Traction in Sports Broadcasting

Fraunhofer Institute, the German software company that helped develop the ubiquitous MP3 format, set up shop in Aspen in January, about the same time as a major winter sports event. In a rented remote truck, Fraunhofer’s A1s took in several audio feeds from the host broadcaster, including three announcer feeds, as well as inputs from a pair of homemade eight-transducer microphone trees — one composed of high-end Shoeps microphones, the other cobbled together from spare mics in various trucks in the compound — intended to gather the ambient sounds of the extreme winter-sports course. These were run through the company’s recently developed MPEG-H encoder and recorded to a Pro Tools system.

All this might have been just a slightly more elaborate setup for a conventional 5.1-surround mix of a live sports show. Except, that is, for the four extra Genelec speakers mounted in both the remote truck and the postproduction studio at Fraunhofer headquarters in Erlangen, Germany, where a final mix was done a short time later. These four speakers, mounted above the front and rear left-right speakers in the conventional array, create what Division GM for U.S. Operations Robert Bleidt calls “5.1 plus four” but what several large players in the broadcast and other media spaces hope home viewers will ultimately embrace as object-based audio.

Objects vs. Channels
Simply put, instead of a speaker array on a horizontal plane that an audio mix can be panned through as the viewer listens passively, object-based audio can assign individual sounds to individual speakers, which can be configured with an extra four (or more) speakers to add the dimension of height to the horizontal surround plane. In fact, ATMOS, a cinema sound system from Dolby, the other major entity in object-based audio, provides for 62 discrete audio outputs and one channel of LFE, plus two sub outputs, the second of which can handle redirected bass frequency from the auditorium surround speakers.

The “objects” in object-based audio could also be compared with the individual elements in a conventional mix, although its advocates suggest that might be mixing apples and oranges. That’s because the channel-based–surround mix is static on the viewer’s end. On the other hand, the codecs offered for object-based audio include the ability, on the broadcaster’s or the viewer’s part, to put individual audio objects (or elements; the nomenclature will take time to settle) into specific speakers, to turn individual channels on and off, to change their volume levels relative to the other audio components, and, in some cases, to choose between alternate channels, such as multiple announcer streams.

If object-based audio catches on, it could change the technical topography of the living room in much the same way that 5.1 surround did a decade ago. But it’s a big if: only a relatively small percentage of homes have the ability to decode discrete broadcast audio — just 15% of homes own an AV receiver with a surround-sound processor, according to the CEA’s 2013 survey — and even the term object-based audio has implicitly limited geek appeal. Extra speakers are the most obvious additional cost. However, at NAB 2014 last month, Fraunhofer demonstrated a new wireless-soundbar concept that delivers 3D audio without the need for multiple external speakers.

But object-based audio proponents understand that the idea will have to start on a smaller scale, both in the truck and in the home. “Unless you have an A2 doing nothing but [the additional three-dimensional mixing], it’s impractical for broadcast sports at this point,” says Bleidt. “But it’s important to distinguish between dynamic object-based audio, where things are constantly moving around, and static object-based audio, where individual objects are in a specific place but can be turned on or off or have their volume varied by the viewer. The technology is there for full dynamic objects, but there is also a lot of potential for static objects, such as announcers. Our proposal for four ‘height’ speakers on the four corners is the best compromise for now.”

Sports Demos
Dolby also offered a broadcast-sports demo at NAB 2014, in the form of a Red Bull-sponsored German hockey match. Listening with headphones, viewers were able to call up the announce track in different languages and could listen to “biased” audio configurations, with the crowd noise from either the home or away team amped up in the mix. They could also listen to a track of ambient sounds, pulled in part from extra microphones they put into the venue.

Dolby is putting its emphasis on what Ken Hunold, technical marketing manager for broadcast audio, calls “the personalized audio experience”: letting the viewer pick elements and their placement. He says the concept is compatible with Dolby Digital Plus but will be more so with Dolby’s new AC4 codec, which will eventually be the successor to AC3. In the home, metadata will query the television audio system via an HDMI “handshake” to determine the number and location of speakers in the system and how objects are distributed to them.

In the Field
From the A1’s perspective, says Hunold, live mixing is the “acid test” of any new sound format. “Based on the testing that we have done so far, one of the most encouraging signs I’ve seen is that most of the content that could be used in an object-based mix is already being captured today; in many cases, separate announcer, crowd, effects, and music sub-masters or groups are already being created,” he explains. “Mixers may have to structure the audio paths through the console a little differently than they do now to preserve the individual objects that they currently mix to a single stereo or 5.1 program. The task may be as simple as keeping some of those groups separate on their way through distribution to the home.

“In our testing,” he continues, “just re-busing the existing sources can create an effective object-based mix. In most cases, fader moves from the existing channel-based mix do not need to be altered. In effect, the work performed in the act of mixing is unchanged.”

In fact, he points out, much of the work of object-based mixing is actually preproduction: determining what elements will be available, such as multi-language announce feeds, and where they’ll be able to go, comparing it with sound design for films.

“Tools are being developed that allow the mixer to check these different combinations as often as desired during a live show,” he says. “Settings for a particular sport or event would be loaded into or recalled in the monitoring/authoring tool, similar to how a saved console snapshot is recalled. During the live show, a mixer could listen in one particular format and occasionally check other listening formats for compatibility or confidence purposes, as often as they wish.”

The concept of object-based audio is well-developed for film, where Dolby says it’s becoming embedded in cinema environments, and is becoming so in game audio. The Conservatory of Recording Arts & Sciences near Phoenix has a career track that includes object-based mixing, and Broadcast Audio Department Director Robert Brock says he is “keeping an eye” on the evolution of object-based audio for its possible convergence with sports broadcasting. (Fox Sports audio consultant/lead mixer Fred Aldous has helped structure the broadcast-audio curriculum at the school.)

Object-based audio for broadcast is still in its infancy. Neither Fraunhofer nor Dolby have yet fully productized its respective system, although both will be part of larger strategies as they promote their newest codecs for broadcast (they seem to be going about their trial work similarly: both the Dolby and Fraunhofer sports tests combined broadcasters’ conventional feeds with the company’s own ambient mic-tree sound-capture assembly). Standards will need to be created and focus groups parsed. But, if you thought 5.1 discrete surround audio was the last stop on the audio railroad, object-based mixing wants to add another station.