ESPN Emerging Technology At Home in New Jersey

In an office park steps away from the train station in Hamilton, NJ — a town situated between Trenton and Princeton — lives the unlikeliest of companies: ESPN. ESPN Emerging Technology, to be exact.

ESPN acquired the company formerly known as PVI (and owned by Cablevision) in 2010, changed its name to Emerging Technology, and moved its 12 staffers from Lawrenceville, NJ, to Hamilton for easier access to the NJ Transit commuter-rail line.

Nikhil Deshpande (left) and Jay DiGiovanni showed off the vast number of technology tools the team has created to serve ESPN’s varied needs.

Since then, ESPN Emerging Technology has focused on developing and deploying virtual-graphics enhancements for every property televised by its parent company, including football, basketball, baseball, and soccer.

“This is a group that’s made up of some highly skilled computer-vision engineers, so we had traditionally done things differently,” says VP of Technology Enhancements Jay DiGiovanni. “We didn’t live on instrumented cameras. We worked solely [using] algorithms. So, when we became part of ESPN, their typical workflows had games that were second and third tier that our vision solution — which doesn’t even need to be done on-site, can be done from the broadcast center — made a lot of sense [as] a very cost-effective way to enhance a game.”

The Hamilton remote office is one of three Emerging Technology groups under the ESPN umbrella. The Bristol, CT, group — the largest of the three — handles onscreen overlay graphics (including score bugs and studio shows), and the Florida group — located at the ESPN Wide World of Sports complex — handles research and development.

DiGiovanni’s group consists of 12 employees — “with the largest concentration of PhDs in ESPN,” he quips — who handle research, development, validation, and deployment of tools to create virtual first-down lines, shot clocks, advertisements, and more.

Ximin Gong stands with an instrumented camera, on which he experiments with different graphics tools and algorithms.

Because of ESPN’s multitude of sports properties requiring on-field virtual enhancements, the Hamilton team worked to create scalable solutions that can be controlled from the studio or truck or implemented directly on the cameras themselves. Chief Emerging Technology Engineer Ximin Gong creates the algorithms that allow ESPN to draw information directly from calibrated cameras and broadcast onscreen continuously, even when the camera needs to pan, tilt, zoom, or focus.

“The key is, how do you do it efficiently and fast?” says Gong. “Because you expect the operator to be there running multiple cameras, they need to finish it in a very short time window. So the calibration process needs to be efficient and quick.”

While the tools are currently limited to fixed tripods, Gong is experimenting with a push to handhelds in order to better serve the varied needs of ESPN. “They’re not just using fixed tripod cameras in the broadcast,” he says. “[In] their studio environment or other environment, they’re using different kinds of [cameras] — completely movable or Steadicam — so we want to expand our application to support all of this.”

Emerging Technology engineer Gene Rossi demonstrates the different solutions the Hamilton team has created for football broadcasts, including field-goal–range and down-and-distance lines.

In addition to graphic placement, the Hamilton team works to ensure that the graphics blend seamlessly into the broadcast. In other words, that a first-down line or on-court advertisement doesn’t interfere with the fan’s ability to watch the game.

“The one big piece of this puzzle is, how do you make it look like it’s lying on the field and have players walk over it?” says Nikhil Deshpande, associate director, systems. “The chroma-key technology [that we use] exists, people know how to do it, but not many people have figured out how to do it in a dynamically changing environment. You have different field colors, textures, clouds [creating shadows], lighting conditions changing. … The level of automation is high enough that the operator can concentrate on the game, where the line is and make that look nice rather than have to worry about all the ancillaries.”

The tools created in Hamilton serve to repurpose and repackage content within the broadcast, whether using the score bug to create down-and-distance markers on the field or shot-clock information to create a countdown on the court.

“If you’re watching the game at a restaurant where you’re looking across a bar… the large on-field enhancements that we put in are how we tell people what’s going on,” DiGiovanni explains. “A research firm did some investigation of where eyes remain, and the high-value places are always where we try to target. When you’re watching a football game, your eyes are right in the middle of the field watching the play. That’s where we try to enhance right there, bring the value to where the eyes are.”