Non-Linear Editing in Sports Production
By John Rice
What does it mean when we say non-linear editing?
Is it software? Is it hardware? Or are the words more indicative of workflow?
Today’s sports-production environment is increasingly dependent on non-linear “editing applications, especially in a live-broadcast scenario, to turn around highlights in-game and for postgame review. And that’s coupled with a growing marketplace for non-broadcast use of game and event footage: posting on the Web or sending to cellphones.
Non-linear solutions in today’s marketplace are proprietary systems or open software and hardware solutions.
SVG surveyed some of the leading companies in today’s non-linear marketplace about their product offerings, workflow benefits, and how they see themselves serving this industry. Opinions (and products) differ in such areas as codecs, integration with third parties, and workflow scenarios. As expected, each offered a compelling argument about the benefits of its offerings.
One theme, however, occurred consistently in every conversation: in sports production, it’s all about speed.
In October, Adobe released Creative Suite 4, the latest generation of its popular software bundle, which comprises After Effects, Photoshop, Flash, Premiere Pro, and other production-oriented software.
The non-linear centerpiece of CS4 is Premiere Pro, which “fits into the center of a lot of production workflows,” says Giles Baker, Adobe group product manager for editing workflows. “The focus of Premiere Pro is on speed and turnaround for video production.
“Premiere is all about native editing,” he continues. “From a philosophical standpoint, we don’t want our customers to spend time converting content from their camera to some other version of that content so it can be edited.”
Premiere supports a broad range of signal and camera formats, he points out. For fast-turnaround, relatively straight cuts, it can work directly off file-based camera-recorded media. “If you want to do compositing, it’s better to copy the content to your machine,” he says. “Even then, we ingest that content natively. We don’t have to do conversions.”
Baker also points to benefits of Premiere Pro: “We focus on the integration between the products and eliminate rendering wherever we can as we move content thru the workplace.” For example, an After Effects composition can be dragged into the Premiere Pro timeline without rendering. That timeline can be dragged directly into Adobe’s Encore authoring application to create a BluRay disc or Flash version. “And you never have to render it in After Effects. That speeds up production, but it also allows people to experiment.”
Premiere Pro’s ability to work with native formats is a key benefit for work in a live environment as well. “In a live situation, you want to get things out to multiple platforms,” says Baker. “Different systems require different encoding. What we’ve done with CS4 is expand our encoding features so they now batch all processes that run in the background. It will upload to an FTP server, create Flash videos. All those things enable fast turnaround of content.”
Currently, Premiere Pro can export directly to Omneon servers via Matrox hardware. “We are working on integration in general,” he says. “What we find is, most of the time, the exported content from the editing timeline will work directly with most playout servers.”
Another important aspect of Premiere Pro, according to Baker, is integration with other non-linear software offerings. “We see a lot of customer need to move content around to different systems. Customers are using Avid, Final Cut, Premiere Pro; they want to move content between them because so many people have different systems.”
Creative Studio 4 includes the ability to import Final Cut projects and exchange content with Avid. “What that means,” Baker explains, “is, you can do work cuts in the field and bring it back for more heavy-duty production work, and that might be using a different system. We are very mix-and-match “oriented.”
“On the editing side,” he says “we see a lot of similarities between systems. What we focus on is precision and speed in editing.” CS4 includes approximately 60 new features oriented toward speed of workflow. “It’s really eliminating clicks, making things more streamlined, and really understanding where customers spend time doing redundant tasks.”
Baker also sees metadata as a key component in current and future editing and distribution applications. “We’re extremely focused on generating and using it in production because it helps people save time. As important, we’re interested in generating useful metadata that can be used on the distribution side to enhance the way viewers experience video.”
Baker points to MSNBC’s coverage of the president’s inauguration speech. “Not only do you get the video of the speech, you also get a transcript of the content and the ability to search through the content using words and go directly to that part of the video.
“As a content owner,” he adds, “that means we can tie advertising and information to the video much more effectively because we know exactly what is going on.”
He sees a great advantage to this type of metadata application in sports. “Sports is all about stats, right? If you can tie the stats directly to what happened and when, then it becomes much more compelling as a viewing experience. That is something we are trying to enable all the way through production.”
“When Final Cut was developed, at no point did anyone sit down and say, ‘Let’s build the best sports editor we can build,” says Richard Townhill, director of pro video applications marketing for Apple. “What we wanted to do was build the best editorial application we could build.
“In some respects,” he adds, “it’s designed as a Swiss Army knife for editorial content.”
Townhill suggests that one of the reasons Final Cut Pro has become successful is its open architecture. “We have a thriving network of over 100 different developers that are building tools and applications, which allow Final Cut to get plugged in to various environments and work seamlessly in those environments. Sports is one of those.”
In today’s live and production market, he says, “the whole industry is taking on a much more IT mindset. As long as you are working in open standards and as long as we’re providing connectivity, you’re not taking a risk anymore by buying hardware from different vendors and connecting them together.
“Obviously,” he adds, “what you are trying to do is make the pipeline as seamless as possible. Especially when you are covering sports, you don’t want any delays in the process.”
Townhill suggests that the success of Final Cut Pro in the sports environment is due, in great part, to third-party developers. British company The Gallery has created Picture Ready, a plug-in that allows nearly simultaneous editing as live video is ingested. “What happens,” he explains, “is that you get a clip inside Final Cut that effectively grows as the content fills up.”
During the recent World Cup in Europe, Picture Ready was used as part of a process to deliver highlights to cellphone users. “When an event of significance happened” a foul or a goal or something on the soccer field that was of significance” it was captured on the Final Cut timeline. The material was edited into a highlight clip and “using a compressor,” Townhill says, “it was encoded into a 3G format and distributed to millions of cellphones throughout Europe.” Highlight clips, he adds, were delivered within five minutes of the event’s occurring on the field.
For Apple, the philosophy of “openness” extends to its codecs. The 2007 introduction of Final Cut Pro 6 included ProRes 422, a new codec that provided HD quality at standard-definition file sizes. The ProRes codec is a 10-bit HD/SD compressed format that provides 4:2:2 chroma sampling and enables conversion of HD, HDV, DV, and other formats. “Our partners have thoroughly embraced us, including Omneon, which is adopting ProRes as part of the playout process,” says Townhill. “The only level of connectivity you need between the editorial station and playout server is an Ethernet cable. It makes the process very straightforward.”
The goal is consistency of format from input to output, he says: “The notion of centralized storage and tools like Picture Ready allow you to create QuickTime files from the moment the feed hits your broadcast environment.
“Our project formats are published as XML,” Townhill says. “The idea that we have APIs that allow third parties to plug into both our ingest and output facilities allows our developer community to take Final Cut into places we never imagined it would go.”
There are three fundamental issues in sports production, according to Avid Director of Product Marketing Patrick McLean: “Quality, reliability and workflow. Obviously, one of the most important things about sports is the immediacy” the ability to respond quickly, to act quickly.
In June 2008, Avid released Media Composer 3.0, Symphony 3.0, and a new family of hardware. Using new DX architecture, the 3.0 products are based on a number of important changes, says McLean. “The prior generation of products were based on a Firewire interface to an external box, which proved limiting.” The new PCI Express interface connects to an external box at 10 gigabits. “We have an extremely fast connection to the external hardware,” he explains. “Because it’s PCI Express, it puts the video hardware right adjacent to the processors of the host. We also reworked all of our software for multicore post-based processing. On the current nine core machines, you can see enormous increases in performance.”
Equally important to a system’s speed is the “aspect of performance that rotates around workflow and optimized workflow,” says McLean. Avid works in native MXF, similar to camera formats P2 and XDCam. “The speed of getting started at work is instantaneous,” he says. “We handle multiple time codes. If you are working with different formats, you can track the different metadata associated with these things directly.”
Within Avid’s product lines, McLean points to advantages of working in a Unity or Interplay environment. “We share products. You can move your user settings from machine to machine. And you can start editing while ingesting.”
Avid also integrates with server systems like EVS and Omneon. “DNxHD has been standardized by SMPTE, and we have direct workflows with EVS where that server is integrated directly.” Avid also integrates with Omneon, and McLean hints that “we’re going to be announcing an even tighter integration with Omneon very soon.
“The aspect of openness and integration is very important to Avid,” he adds. Noting that there have been a number of changes within Avid over recent years, he says, “One of the key things the new management is emphasizing is to focus on open interfaces. People are building heterogeneous environments. It’s very important that we fit in those as smoothly as possible.
“I think you’ll find we have very broad codec support,” he says. “On the same timeline, in real time, you can mix Sony’s XDCam and Panasonic’s DVCPRO, and we have support for Panasonic AVCi. You can put in HDV, Quicktime files, and native MXF.”
While EVS is generally considered a major source of server systems finding increased application in sports production, the company also offers non-linear solutions.
In 2006, the company unveiled IP Director, a Windows-based “video-management suite” that handles ingest control, metadata management, and playout scheduling. At last summer’s Olympics, a new addition to IP Director was unveiled called IP Edit.
According to EVS General Manager of Operations Greg Macchia, IP Edit fills “a niche to be able to give [non-linear] support directly from an EVS product.” While products like Final Cut Pro and Avid integrate easily with EVS servers for quick-turnaround editing, “in some cases, you want to bring a program in and quickly turn stuff around,” he explains. “If they wanted to fix a graphic, add a voiceover, do some quick fixes, that’s when they used IP Edit.”
IP Edit does not replace the need for higher-quality editing. “This isn’t meant to be a high-end product like Avid or FCP. When higher capability is needed, we have the capability to select pieces we can send to Avid, for example, to do that higher end,” says Macchia. “In some cases, that is the solution. In other cases, it can be part of a combination.”
IP Edit and IP Director grew out of a desire to simplify and accelerate workflow, especially in a live-production environment. “Rather than control [the media] with our slo-mo remote, IP Director is a Windows GUI that allows drag-and-drop, database-searching capabilities, and metadata logging,” says Macchia. “From that came the desire to be able to do more in the way of timeline editing: keyboard shortcuts and things like that.”
While IP Edit works directly with material on EVS servers, the company is also looking to expand its codec capabilities on the server level. “We are working with DNX, working with Avid,” says Macchia. “We’ve got two or three other codecs on the horizon,” he hints. “AVC looks like a good one.”
“From the macro level, while there is still a lot of video, that is video in the professional and broadcast space; there is a strong movement toward file-based, tapeless workflow,” says Mark Narveson, director of product development for desktop and enterprise solutions for Grass Valley. He sees a need for “an editing system that handles both standard video and audio inputs and outputs but also is able to deal with all the expanding number of file-based formats that are out there.”
Grass Valley offers both hardware and software non-linear solutions. On the hardware side, the EDIUS NX Express and SP-SDI handle standard-definition video, while the new HD Spark, HD Storm, and HD Thunder offer different levels of high-definition capabilities. At the center is Grass Valley’s EDIUS software, which was released in version 5.0 at last fall’s IBC, along with the new HD hardware systems.
One of the key benefits of EDIUS, according to Narveson, is that “it can handle natively all these different formats, decode them on the fly, add them together without having to go to an intermediate format, or transcode them until you are ready to commit to a final product.”
These formats are supported by the Grass Valley lines: Canopus HQ, Canopus Lossless, Infinity JPEG 2000, DV, DVCam, HDV, AVCHD, MPEG-2, AVC-intra, and uncompressed video. “From a workflow perspective, you just grab the file and start editing,” Narveson says.
EDIUS is “meant for very high-turnaround editing,” he explains. “EDIUS’s codec can take advantage of the parallelism of multi cores and multi CPUs. So, for example, the more cores you have and the more CPUs you have in a system, the more streams you can do at a time. You can do more than just cuts editing. You can layer up a fair amount and be able to do that in real time, finish your edit, and commit it.
“The key,” Narveson adds, “is the fact that you can begin editing right away without having to do format transformation and rewrapping on the input side.” The EDIUS export module “exports in most of these formats as well.”
Today, he says, EDIUS is being used for highlights, repurposing video for Web-based programming, and editing features for broadcast.
Although EDIUS 5.0 was released just last fall, Narveson says version 5.1 will be released at this year’s NAB with additional workflow improvements.
“The point of our entire workflow is that we are able to optimize it because we are a soup-to-nuts provider,” says Harris Senior Product Specialist Bob Bolson. “We can provide it from ingest to edit to playout all from one provider.”
The centerpiece of Harris’s non-linear capabilities is its Velocity line. Velocity ESX is “our high-res system,” says Bolson. “It connects to our shared storage via either Fibre Channel or Gigabit Ethernet, depending on how it’s configured. You edit your HD or SD material directly off the shared storage. There’s no push-pull. There’s no transfer going on. It’s a direct-editing process that is being used in production that requires a fast turnaround.”
Velocity is also available as XNG, for laptop field editing, and PRX, a proxy editor.
One of the prime applications of the Velocity system is for game highlights, such as the current installation at TNT for NBA coverage, which integrates logging software from Dixon Sports Computing. “We’re able to edit material within 5 seconds of the beginning of ingest,” Bolson explains. “You can actually drag directly out of the Dixon highlight selector into the Velocity timeline. You can define the handles. You can even drag multiple highlights at the same time, trim up the edges, clean up your handles, drop in some dissolves, and even drop in some ‘slo-mos’ or some spot shadows. All of that is very easy to do. You can store those all as macros.”
He adds that, once macros are stored, “you can start publishing that. Then, 5 seconds after the publish process begins, they can be playing that out on-air, even if the publish isn’t complete.”
Harris systems accommodate a variety of current formats, including MPEG-2, DV, DVCPRO 20/50/100, XDCam, and P2. “We wrap it in a proprietary format: an LXF file, our version of MXF,” says Bolson. “Essentially, we do that so we can stuff the header with stuff that makes sense to us. It’s really just a wrapper. The codecs all stay in their native format.”
He also notes Velocity’s strength in handling metadata. “The Velocity has an interface with 12 or 14 different categories on the XDCam and P2 with drop-downs. So you can choose each piece of metadata that the P2 or XDCam offers, and it has a second column for all our LXF data. So you choose how to map it.”
While the Harris offerings are complete, proprietary systems, they do integrate with Final Cut Pro. “We realistically have to be cognizant of the fact that Final Cut Pro is a hugely popular editor,” says Bolson. “We’ll be the first to admit that the Velocity editor is not as solid at doing high-end, craft editing. It’s not going to be able to do film-glows, five or six layers of compositing. That’s why we offer the Final Cut option.”
But, he continues, “if you using a long-GOP format, we do something called a ‘smart render,’ or long GOP. In Final Cut, if you start chopping up long GOP, when you publish that at the end, it’s going to render the entire thing to create a predictable GOP structure. What we do, if it’s cuts-only, is copy all those pieces, put them in a single container on the server, and the transmission server is able to play back these variable-GOP-structure clips. It’s very quick and very efficient, and you’re not taking up additional space.
“What we think we do very well is fast-turnaround highlight video,” Bolson adds. “It really is speed.”
“When it comes to sports production, where Quantel technology and services are increasingly applied is with its server-based production that allows simultaneous production,” says Danny Peters, creative services director for Quantel North America.
The centerpiece of Quantel’s offering is a range of products under the sQ Server label. “While you are recording a game, within frames of hitting the server, you can start playing it out. Multiple users on desktop applications can have simultaneous access to that incoming material,” Peters explains. “In practical terms, while we are recording in HD, simultaneously, a low-resolution copy is being made. Anyone sitting with a browse application can view and log and make shot selections for playout. That’s what’s important for a live-game application.”
For editing in that environment, Quantel offers a range of sQ Editor products.
sQ View is for “shot selection, viewing, and logging,” Peters explains, adding that anyone moderately familiar with non-linear systems can be taught sQ View “within 15 minutes; if they are really a newcomer to non-linear, in an hour, maximum.”
For timeline-based, cuts-only editing, Quantel offers sQ Cut; for effects, compositing, and titling, there are sQ Edit and Edit Plus.
“The interface is the same,” says Peters. “If you learn your skills on sQ View, you can easily move to the higher-end editor and do your job.
“The process is,” he adds, “while you are capturing, as soon as you start to capture, everybody that is tied in on the application side” sQ View, sQ Cut, sQ Edit” can access material, and you can play it out. As a sports producer, I can assign different tasks to the process, different people to log various points of interest from the game. It allows me to get to highlights faster while the game is going on. Because you’ve added a non-linear “workflow process, your speed to air and your flexibility happen tremendously quicker.”
Quantel systems currently can work with DVCPRO HD, AVC, and IMX 30. For field material, “one of the good things about how we work with that kind of source material is speed,” says Peters. For example, using Quantel’s Essence driver with a P2 source, “once the drive is plugged in, we can see that material as if it’s in the Quantel library. We can access the material. We can make an edit. We can mix it with material that’s already in the Quantel library. At the time of publish, you are only taking what you need from the P2 Essence and sending it across. It’s a much quicker way of working.”
Peters acknowledges that the Quantel systems do not work directly with Avid “at this point.” He sees Avid’s coming into the Quantel environment for postproduction, when an AAF or EDL can be transferred “and we pick up the decisions.”
But there may be news forthcoming regarding Final Cut Pro. At last fall’s IBC, Quantel provided a technology demonstration integrating Final Cut. “We started an edit using our sQ product, then we picked up that edit on Final Cut Pro that we integrated into the sQ server. It had the exact same flexibility.”
Looking Forward” Stereoscopically
While non-linear providers are at different levels of development and exploration, it is certain that all eyes are on 3D or stereoscopic TV (pun intended). Recent experimentation such as Fox’s 3D bowl-game broadcast” using EVS servers” and commercials and promos during the Super Bowl are early forays into what many believe is the next technological wave in sports broadcasting.
“This is an emerging new area,” says Avid’s McLean.
Harris’s Bolson agrees. “The feeling in the industry is, that’s the next thing. But there’s so much that has to happen: infrastructure, monitors, cameras, and everything else.” Although he won’t say what Harris may be working on, he confirms that “it can’t have that much buzz without us and everyone else working on it.”
Avid, Quantel, and EVS are among the companies that have been visible in the stereoscopic arena” both in support of recent 3D broadcasts and in demos.
“We showed [in a technology demo in 2008] that you can do editorial on a Media Composer, even Media Composer software.” says Avid’s McLean. “You can get a commercially available $3,000 monitor, hook it up to your Media Composer, and do editorial.” Avid’s involvement to date has been “more on the feature-film side of things,” he says. “We are interested in seeing what is going on in the television side of things.”
Adobe and Apple are looking at the stereoscopic future as well. Says Adobe’s Baker, “We have some partners who are looking to create plug-ins for [3D].” But he cautions that “we don’t have anything extremely detailed to talk about yet.”
Apple has been working with the Hollywood community and “some people who are doing experiments,” says Townhill. “Will Quicktime and Final Cut work? It’s such a new field, and there are no specific features in Final Cut Studio II.”
Grass Valley’s Narveson acknowledges that “it’s something that we’re looking at. One of the things we’d like to potentially do is make it more automatic so you can have left and right channels and, when you do effects, it will automatically create the proper angles for left- and right-eye viewing.” He adds, “I don’t have a promise date on that, but that’s something that we’re looking at.”
Quantel “came out of IBC with six or seven awards in stereoscopic [TV],” Peters reports, adding, “There is huge interest amongst sports broadcasters. It can be offered as a premium channel, a way of getting new revenue.”
There are issues to be resolved. Among them: SMPTE standardization, extension to the home, and the financial costs and benefits to broadcasters.
Nevertheless, all of those we spoke with agree that is likely the next big thing.