Smellyvision and Associates

 What is reality? And is it something we want to get closer to? Take a look at the picture of a cat above, as printed on a package of Bell Rock Growers’ Pet Greens® Treats <http://www.bellrockgrowers.com/cattreats.html>. Does it look unreal? Distorted? Is it?

At this month’s HPA Tech Retreat in Indian Wells, California (shown above in a photo by Peter Putman as part of his coverage of the event <http://www.hdtvexpert.com/?p=1804>), there was much talk about getting closer to reality by using images with higher resolution, higher frame rate, greater dynamic range, larger color gamut, stereoscopic sensation, and even surround vision. The last was based on a demonstration from C360 Technologies. Another demo featured Barco’s Auro-3D enveloping sound technology. In the main program, vision scientist Jenny Read explained how stereoscopic 3D in a cinema auditorium can’t possibly work right and why we think it does. And then there were the quizzes.

All of them related to the introduction of image and sound technologies at various World’s Fairs. Although the dates ranged from 1851 to the late 20th century, more than one quiz related to technologies introduced at the 1900 Paris Exposition. It stands to reason.

At that one event, people could attend sync-sound movies and watch large-format high-resolution movies on a giant-screen. They could also experience reality simulations: an “ocean voyage” on a motion-platform with visual effects called the Mareorama (depicted at left), a “train trip” on the Trans-Siberian Railway using spatial motion parallax (with one image belt moving at 1000 feet per minute!), and a “flight above the city” in the surround-projection-based Cinéorama (shown below, with synchronized projectors under the audience). At the same fair, they could also hear sound broadcasting of music (with no radios required) and even try out the newly coined word television.

Well over a century later, we still have sound broadcasting (though receivers are now required), we still watch sync-sound movies, and we still use the word television. There are still large-format large-screen, surround vision, and moving-platform experiences, but they tend to be at, well, World’s Fairs, museums, and other special venues.

There was a time when at least 70-mm film was used as a selling point for some Hollywood movies and the theaters where they were shown. And then it wasn’t. The audience’s desire for quality didn’t seem to justify the additional cost. The digital-cinema era started at lower-than-home-HD resolution but is now moving towards “4K,” more than twice the linear resolution of the best HD (the 4K effects and workflows of The Girl with the Dragon Tattoo were discussed at the HPA Tech Retreat).

Back in the publicized 70-mm film era, special-effects wizard, inventor, and director Douglas Trumbull created a system for increasing temporal resolution in the same way that 70-mm offered greater spatial resolution than 35-mm film. It was called Showscan, with 60 frames per second (fps) instead of 24.

The results were stunning, with a much greater sensation of reality. But not everyone was convinced it should be used universally. In the August 1994 issue of American Cinematographer, Bob Fisher and Marji Rhea interviewed a director about his feelings about the process after viewing Trumbull’s 1989 short, Leonardo’s Dream.

“After that film was completed, I drew a very distinct conclusion that the Showscan process is too vivid and life-like for a traditional fiction film. It becomes invasive. I decided that, for conventional movies, it’s best to stay with 24 frames per second. It keeps the image under the proscenium arch. That’s important, because most of the audience wants to be non-participating voyeurs.”

Who was that mystery director who decided 24-fps is better for traditional movies than 60-fps? It was the director of the major features Brainstorm and Silent Running. It was Douglas Trumbull.

As perhaps the greatest proponent of high-frame-rate shooting today, Trumbull was more recently asked about his 1994 comments. He responded that a director might still seek a more-traditional look for storytelling, but by shooting at a higher frame rate that option will remain open, and the increased spatial detail offered by a higher frame rate will also be an option.

That increased spatial detail is shown at left in a BBC/EBU simulation of 50-fps (top) and 100-fps (bottom) images based on 300-fps shooting. Note that the tracks and ties are equally sharp in both images; only the moving train changes. The images may be found in the September 2008 BBC White Paper on “High Frame-Rate Television,” available here <http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP169.pdf>.

Trumbull is a fan of using higher frame rates, especially for stereoscopic 3D (his Leonardo’s Dream was stereoscopic). Such other directors as James Cameron and Peter Jackson have joined that approach. And at the SMPTE International Conference on Stereoscopic 3D in June Martin Banks of UC-Berkeley’s Visual Space Perception Laboratory explained strobing effects that can occur in S3D viewing.

A hit of the 2012 HPA Tech Retreat, however, in both the main program and the demo area, was the Tessive Time Filter, a mechanism for eliminating (or at least greatly reducing) strobing effects without changing frame rate. It applies appropriate temporal filtering in front of the lens — essentially any lens. Because the filtering is temporal, it does not affect the sharpness of items that are stationary relative to the image sensor. Above right is an image illustrating a “compensator” plug-in for Apple’s Final Cut Pro “to achieve the best possible representation of time in your footage” (when the green word “Compensated” appears at the bottom right, the compensator is on <http://www.tessive.com/>).

That’s frame rate and resolution. Visual dynamic range (from brightest to darkest) and color gamut were also topics at the 2012 HPA Tech Retreat, primarily in Charles Poynton’s seminar on the physics of imaging displays and presentation on high-dynamic-range imaging, in a panel discussion on laser projection, and in Dolby’s high-dynamic-range monitoring demonstrations.

Poynton noted a conflict between displays that can “create” their own extended ranges and gamuts and the intentions of directors. He also noted that in medical imaging, where gray scale and color can be critical, there are standards that don’t exist in consumer television. But that doesn’t mean medical imaging is closer to reality. In fact, it might be nice for a tumor otherwise invisible to show up very obviously, like a clown’s red nose.

Above left is another scientific image, the National Oceanic and Atmospheric Administration’s satellite image of cloud cover over the U.S. this morning, at a very clear time. Rest assured that the air did not look green, yellow, and brown at the time. Sometimes reality is not desirable.

Consider the cat at the top of this post. Its unusual look is intentional, something to grab a shopper’s intention. But it’s actually not unrealistic.

Try holding your hand about a foot in front of your face and note its apparent size. Now move it two feet away. It looks smaller, but not half the size. Yet the “real” image of the hand on your retina is half the size.

Reality is even more complex. We track different moving objects at different times, changing what looks sharp or blurry. We focus on objects at different depths in a scene, unlike a camera (regarding stereoscopic 3D perception, at the HPA Tech Retreat Read noted that although a generation that grows up with S3D imagery might not experience today’s S3D viewing difficulties neither might they find S3D exciting). We can see 360 degrees in any direction (by moving our heads and bodies, if necessary). We can also hear sounds coming from any direction. And then there are our other senses.

At the 2010 International Broadcasting Convention in Amsterdam, the Korean Electronics and Telecommunications Research Institute demonstrated what they called “4D TV” (diagram above). When there was a fire on screen, viewers felt heat. When there was the appearance of speed on screen, viewers felt the rush of air across their faces. During an episode reminiscent of a news event in which an athlete was struck, viewers felt a blow on their legs. And there were also scents.

“There may come a time when we shall have ‘smellyvision’ and ‘tastyvision’. When we are able to broadcast so that all the senses are catered for, we shall live in a world which no one has yet dreamt about.”

That quotation by Archibald Montgomery Low appeared in the “Radio Mirror” of the (London) Daily News on December 30, 1926. Much more recently (June 14 of last year), the Samsung Advanced Institute of Technology and the University of California – San Diego’s Jacobs School of Engineering jointly announced the development of something that might sit on the back of a TV set and generate “thousands of odors” on command. But that raises the reality issue, again. Do we really want to smell what the sign above left depicts?

Archibald Low was an interesting character. He was inducted posthumously into the International Space Hall of Fame as the “father of radio guidance systems” and was one of the founders and presidents of the British Interplanetary Society, but he was also (among many other posts and appointments) fellow and president of the British Institute of Radio Engineers, fellow of the Chemical Society, fellow of the Geographical Society, and chair of the Royal Automobile Club’s Motor Cycle Committee (he built and arranged the demonstration of a rocket-powered motorcycle, above right).

Besides that motorcycle, he also developed drawing tools, a well-selling whistling egg boiler, and what was probably the first drone aircraft not carrying a pilot. But two other aspects of Low’s long and varied career might be worth considering.

In 1914, he demonstrated, first to the Institute of Automobile Engineers and later at Selfridge’s Department Store, something he called “televista” but probably better described in the title of his presentation, “Seeing by Wireless.” And, in a 1937 book, he wrote, “The telephone may develop to a stage where it is unnecessary to enter a special call-box. We shall think no more of telephoning to our office from our cars or railway-carriages than we do today of telephoning from our homes.” So he wasn’t too bad at predictions.

“Smellyvision”? Who knows? But, if we’re lucky, it won’t bring us any closer to reality.

Tags: 4D, 4K, Archibald Low, Cineorama, color gamut, Douglas Trumbull, ETRI, high frame rate, high resolution, High-Dynamic Range, HPA Tech Retreat, Jenny Read, Mareorama, Poynton, reality, Showscan, Tessive,