Sports Graphics Forum: How AR, Virtual Studios Are Changing the Game for ESPN, Fox Sports, Turner, Univision

Sports-content creators are rethinking the studio, taking it beyond the physical

Augmented reality and virtual studios have allowed sports-content creators to rethink what is possible inside the studio. Whether it is expanding the physical footprint, creating more-flexible multi-use sets, displaying interactive analysis tools, or teleporting on-air talent from the field into the studio, AR is allowing studio shows to go beyond the limits of the physical world.

Augmented Reality in the Studio: Expanding Beyond the Physical World: (from left) Turner Studios’ Zach Bell, Fox Sports’ Craig Chupinsky, Univision Deportes’ Alexis Salinas, Disney DTCI’s Aaron Thiel, and Ncam’s Phil Ventre

Recent projects like Fox Sports’ NASCAR virtual studio in Charlotte, NC, and ESPN’s Around the Horn set at its New York City Seaport District Studios, along with Turner Sports’ ELEAGUE efforts and Univision’s AR deployments, illustrate how these technologies are dramatically changing the face of sports studio shows. Along with Ncam — one of the key vendors in the AR realm today — execs from these broadcasters took the stage at last month’s SVG Sports Graphics Forum to provide a deep dive into their recent AR and virtual-studio projects.

Around the Horn Goes Virtual for Move to Seaport Studios
When ESPN decided to move Around the Horn from its existing location in Times Square to its new Seaport Studios (opened in April 2018 and home to several other ESPN programs, including First Take and Get Up), the ESPN Remote Production Operations unit, ESPN Creative Services Motion Graphics Design, and Disney DCTI Technology group teamed up to create a fully integrated, three-dimensional studio environment. The studio, which launched in November, completely reimagined interaction between host Tony Reali and the four daily panelists.

Courtesy of ESPN

“When we knew [Around the Horn] was going to move, we wanted to take this opportunity to repackage the show and give it a fresh look,” said Aaron Thiel, staff software engineer, Disney DTCI. “We had a meeting with Tony and the showrunners, and Tony said, ‘We want this show to look like it’s from 10 years in the future.’ That freed us up to take chances and break the mold of what Around the Horn was.”

To accomplish this, ESPN revisited the depth-monitor technology it first deployed at the 2019 FIFA World Cup in Johannesburg. This technology allows ESPN to feature virtual monitors within the set (showing the four panelists) that dynamically adjust when a camera pans around the studio (for example, when the camera pans left, the virtual monitor adjusts its perspective to look more lifelike). As the show progresses from its opening First Word segment through its signature Showdown and Facetime ending, the panelists’ images appear in-studio with Reali and can be enlarged, shrunk, isolated, and more. In addition, the higher-scoring panelists who advance through the daily competition will transition forward closer to Reali.

Although the studio footprint is just 16 x 25 sq. ft., the virtual technology (which leverages a Mo-Sys camera tracking system, Vizrt rendering engines, and computer-animated design) allows ESPN plenty of freedom — in terms of both space and flexibility to use for multiple shows.

“The facility is very tight, so we really wanted to extend it out and make it feel bigger,” said Thiel. “We freed up Tony, got him out from behind the desk, and gave him more control over the talent. It was really a game-changer for us. We also knew. when we were done, that, if Tony could use it, then we could pop in another show afterwards. Katie Nolan or someone else could come in, and we could flip that set over in 30 seconds and reuse all this technology. It’s a big technology cost, but, in the long run, it could be a cheaper production.”

Fox Sports Off to the Virtual Races With NASCAR Set
Fox Sports launched one of the largest virtual-studio projects ever seen in the sports sector with a new NASCAR studio at its Charlotte, NC, production facility. Launched in February ahead of the Daytona 500, the ultra-customizable 60- x 60-ft. facility allows instant 3D analysis of racetracks, cars, race shops, and more, with the ability to create unique and changeable spaces within the high-tech studio environment in a matter of hours.

“You can flip that set around so that, with the click of a button, you have a completely different set. You can put in a completely different environment for each individual show,” said Craig Chupinsky, senior designer, Fox Sports. “That same physical footprint can house three, four, five shows in a day. Right now, we’re just doing one set, but we’re using it seven days a week, and the potential is there to [expand]. Yes, there is significant up-front cost, but you end up saving [in the long run] by being able to do all those different shows.”

Fox’s new Charlotte, NC-based virtual studio is playing a major role in its Daytona coverage.

The virtual set features graphic elements powered by Epic Games’ Unreal Engine (the technology behind the popular online videogame Fortnite) suite of tools.

“[We needed it to be] photo-realistic enough so that there is the illusion that it is grounded in this physical space. If we couldn’t achieve that illusion, then we wouldn’t have done it,” said Chupinsky. “And game engines and hardware have gotten to the point when, even when I go home and watch the show, I’m fully immersed in that illusion and have that suspension of disbelief. And if it wasn’t for the quality of these game engines, it wouldn’t have that same impact.”

Fox Sports’ virtual studio for NASCAR coverage will debut at the start of the racing season.

The studio features five Stype RedSpy-encoded cameras, including one Steadicam and a jib. In addition, the facility is equipped with Zero Density studio software, including the advanced keying features. The keying will make use of a 50- x 47-ft. green-screen area, while maintaining the ability to shoot 360 degrees. 3D tracking technology from Blacktrax is used to automatically trigger events on the set. Whether it’s automated lighting following an analyst on set or AR activated by an analyst, this technology opens the door for unique storytelling opportunities.

“Directors and producers are more used to having to move cameras to frame their shot,” said Chupinsky. “But now we can actually move their position and rotate the set to get the shot they want. Or, if they want [to reposition] the talent, instead of the talent walking downstage and having a new lighting position, they can [stay in the same position], and we will move them [virtually] to the new position. Talent adapted pretty quick, and even guests adapt very easily. It was more or less production changing their mentality of how to shoot it.”

For Esports, AR Brings the Videogame Into the Real World
Turner Sports’ ELEAGUE productions have blazed the trail in the use of in-studio AR elements over the past few years. Working with The Future Group and others, the Turner Studios team created life-like incarnations of videogame characters and objects from Street Fighter V, Injustice2, and Rocket League, bringing them to life in the studio.

Zach Bell, senior CG artist, Turner Studios, opined that the power of gaming engines like Unity and Unreal has opened an entirely new world for content creators.

The Turner Studios team created life-like incarnations of videogame characters like Superman for its Injustice2 ELEAGUE coverage.

“Game engines are made to push an enormous amount of visual data through its pipe,” he said. “None of this would have been possible without the robust power of these game engines. Now we can have that sense of realism by using dynamics and having these fluid simulations. All of these things that are typically done only in a postproduction process are now available in a real-time solution for broadcast, which steps up the possibility of the types of experiences we can create for our audiences.”

Univision Virtual Replay: Marrying AR and the Real World
Last year, the Univision Deportes production department and its Technical Creative Services team began working on a project to create lifelike virtual replays in the studio in less than two minutes. These sequences featured a virtual projection of a soccer field and players within the studio, allowing analysts to dissect key plays in a 3D environment during halftime and after matches. Initially, Univision planned to deploy this as a full-screen technology but eventually took the extra step of deploying AR elements within the physical-studio environment using the Unity gaming engine.

Univision Deportes leverages augmented reality to create virtual replays in the studio of key plays in its soccer coverage.

“Some of the real-time render engines were just not capable [of creating these elements in under two minutes], and so we used Unity 3D for these,” said Alexis Salinas, senior director, Technical Creative Services, Univision Deportes. “Our experience was great, and I think the result was what production was expecting. But I would like to see these engines develop control applications for us broadcasters, instead of our having the responsibility and development teams to develop all these control applications.

“You can develop particular solutions — like the one we did for virtual replay — but, to end up using this in a consistent production workflow, we need help creating these control applications,” he continued. “Any technology used in our production pipeline, we need to make it easy and reliable. Otherwise, if they start having problems on the air, they’re not going to use it. Our job is to make these technologies easy to use and consistent and reliable every day.”

Up-Front Investment Is Worth the Long-Term Rewards
Although virtual sets and AR installations can mean pricey up-front costs, the panel agreed that the long-term flexibility and adaptability of these facilities make for a solid ROI.

“We’ve got a broadcast partner we’ve been working with in South America, and they are using eight different [virtual] sets for eight shows every day [from the same facility],” said Phil Ventre, VP, sports and broadcast, Ncam. “So, yes, there is an initial [up-front investment], but it’s saving so much time and cost. We’ve tried to work closely with partners to find out how that works and what’s the best thing for them. From what we’ve seen, as long as you’ve got buy-in from creative, technology, and purchasing [departments], it generally tends to work out well. The kit needs to be robust and needs to work every day. I think we’re getting to that point and [the technology is] a lot stronger than it was a couple of years ago.”

Bell seconded that sentiment: “When you think about up-front costs, you have to think about the long game. When you build out a practical set, you spend all this time and money and craftsmanship to create these assets, and then you’re tearing it down and rebuilding over and over again. … But, with a virtual set, the sky is the limit. You can make it whatever you want. If one day you want to add a football field to the side of your set, you have that flexibility. So it is a front-load, but, in the end, there could actually be most costs savings from a fiscal standpoint.”