Live From MLB All-Star: Intel 360 Replay Technology Steps Up to the Plate
There’s a new kid on the tech block, and it’s generating excitement this week at the MLB All-Star festivities in San Diego. Intel 360 replay technology, which creates a 3D rendering of the field and a virtual view of a play from almost any angle, was deployed by MLB Network for Sunday’s Futures Game and by ESPN for Monday’s Home Run Derby, and Fox will use it for tonight’s All-Star Game.
At Petco Park, the Intel 360 system has 28 5K cameras stretching from foul pole to foul pole, covering roughly 275 degrees of the stadium and all the action on the diamond (the outfield is not covered). The proprietary camera systems feature JAI 5K sensor, a Birger lens-control system, custom power and fiber/copper conversion systems, and various Canon Prime and Super Telephoto lenses (depending on camera position).
Each camera is connected via fiber (more than 20 miles of fiber in all) to a dedicated server in Intel’s rack room at Petco Park. Simultaneous camera data is processed together, composing every point in the ballpark into individual volumetric pixels (or “voxels”), which are used to build a true three-dimensional reconstruction of the scene.
In addition to the individual camera servers, “management servers” provide the computer power to build the replay packages, which display a 3D reconstruction of the play and can be paused, zoomed in or out, and viewed from anywhere on the field.
Two operators are located inside the Intel replay-control room at Petco: the pilot, who creates the camera angles and selects individual angles and moments to zoom or pause, and the navigator, who monitors the system to ensure that all the cameras are shaded and working correctly.
The pilot works with the broadcaster’s production team inside the truck to select “freeze moment” highlights of a play from any angle. Monitoring all 28 cameras, the pilot can select an individual camera to pause or zoom as needed. The clips can be created in anywhere from 2½ minutes for simple packages to 10 minutes for more complex packages with multiple zooms and pauses.
“I work with the production team from MLB and ESPN and Fox, and they will tell me to grab something specific,” says system pilot Jon Helton. “I will then go into [the system], and I have cameras 1 through 28 on a monitor to pick my frames. I hover over my 28 cameras and pick my cameras and movements. Then I hit a button, and it renders out automatically.”
A number of completed clips are sent to the production truck for integration into the telecast (via EVS playout server), and all clips are uploaded to a cloud-based archive for VOD posting online on MLB.com. For example, four clips were used in Sunday’s Futures Game telecast on MLB Network, and two were used on the videoboard inside Petco Park, but more than 30 packages were built during the game.
“[Use of the clips] depends on the production crews and the stories they want to tell for that particular game,” says Kevin Hanson, project manager, immersive sports experiences, Intel. “For instance, ESPN emphasized that they’d like to see certain looks for left-handed batters and others for right-handed batters to analyze their swing during the Home Run Derby. Fox would like to do some of that, too, but they are also looking to capture moments outside of the game itself, such as a player tipping their hat or a memorable moment after a big home run or David Ortiz waving to the crowd after his final at bat.”
MLB All-Star Week marks the highest-profile appearance on the diamond to date for Intel 360, which was created by Replay Technologies (acquired by Intel in March) and previously branded freeD. The system was previously installed at Yankee Stadium (New York Yankees) and is currently in use at Dodger Stadium (Los Angeles Dodgers) and U.S. Cellular Field (Chicago White Sox). Five weeks’ installation time was required at Petco Park to prepare for Intel 360’s use for All-Star Weekend.
Previously, Intel 360 replay was used at the NBA Finals and Conference Finals, as well as for Super 50, the NBA All-Star Game, NCAA Final Four, and US Open tennis. Other installations include AT&T Stadium (Dallas Cowboys); M&T Bank Stadium (Baltimore Ravens); Levi’s Stadium (San Francisco 49ers); American Airlines Center (Dallas Mavericks); and Quicken Loans Arena (Cleveland Cavaliers).