How Different Is 3D?
When you watch a televised advertisement for an automobile, do you fear there’s a moving car in the room with you? I didn’t think so. But more on that later.
This post is about human perception of 3D imagery. It’s also about how we see moving images in general and about color, sound, carsickness, and the idea of smashing open a TV set with a hammer to allow the tiny people inside to be seen more clearly.
That last suggestion probably first appeared in 1961 in an age-inappropriate alphabet tome called Uncle Shelby’s ABZ Book, written by Shel Silverstein. In it, T was for TV. The book indicated that small performing elves lived inside the television set and an adventurous child reader using a hammer to break open the tube “will see the funny little elves.”
That same year, Colin M. Turnbull of the American Museum of Natural History published “Some observations regarding the experiences and behavior of the BaMbuti Pygmies” in the American Journal of Psychology. One of the observations seems related to those little elves in the television set.
“As we turned to get back in the car, Kenge looked over the plains and down to where a herd of about a hundred buffalo were grazing some miles away. He asked me what kind of insects they were, and I told him they were buffalo, twice as big as the forest buffalo known to him. He laughed loudly and told me not to tell such stupid stories and asked me again what kind of insects they were. He then talked to himself, for want of more intelligent company, and tried to liken the buffalo to the various beetles and ants with which he was familiar.” http://www.wadsworth.com/psychology_d/templates/student_resources/0155060678_rathus/ps/ps06.html
Those of us who grew up with television and open spaces might find both stories equally ludicrous. We know the people we see on a TV screen are full size (and don’t live inside the television set) and so are distant animals. But why do we know that?
Based on the angles their images form on our retinas, we should think the people we see on a small TV screen are tiny. We don’t only because we’ve learned what TV is. Kenge, a life-long forest dweller, had never been exposed to distant vision, so he’d never learned how small things might look when viewed from far away.
What does that have to do with 3D? Take a look at the diagram at the left. It was created by Professor Martin Banks of the Visual Space Perception Laboratory at the University of California – Berkeley. The vertical axis represents viewing distance from a movie or TV screen, the “accommodation” or eye’s-lens focusing distance. The horizontal axis represents the depth within a stereoscopic 3D image where something appears to be, the “vergence” or “convergence” distance, the distance to which the two eyes point (“vergence” is used because eyes can both converge and diverge).
The dark-colored area represents a comfortable viewing zone — a depth range where 3D viewing should not make viewers feel sick. The lighter-colored area represents a potentially uncomfortable “fusion” zone, where viewers can combine the two eye views into a single object or character, though they might not like doing so. Outside that zone, even fusing the two images into one can be a problem.
At viewing distances of at least 3.2 meters (easily achieved in cinema auditoriums; less common in homes), the comfort zone appears to extend out to an infinite depth behind the screen, and only very close vergence depths are a problem. At shorter (home) viewing distances, even significant depth behind the screen can cause discomfort, as well as in front of it.
There’s an easy solution to the problem, one put forth in the white paper “3D in the Home.” It was previously available on the web site of the 3D company In-Three. http://in-three.com/
In accordance with the comfort-zone plotted above, the In-Three white paper said depth could extend to an infinite distance behind the screen for movie-auditorium viewing, with restriction only for imagery extending in front of the screen. As shown in the diagram at right, however, for a home-theater viewing distance of six feet, the white paper suggested restricting depth behind the screen to just four feet and depth in front of the screen to less than two feet. That depth range, too, seems well within the vergence-accommodation comfort zone.
It might be possible to restrict shooting to that depth range in a talking-heads-style public-affairs discussion. But that’s an extremely limited range.
It’s unlikely to be sufficient even for a variety or reality show, let alone for most sports. Two football players standing side-by-side perpendicularly to the camera might exceed the range all by themselves.
Another alternative, therefore, is to shoot the natural scene depth but adjust homologous points in the two eye views so that the depth presented on a home display does not stray beyond the comfort zone. Unfortunately, the shrunken depth might cause those football players to be perceived as being tiny, like the supposed buffalo insects or mythical TV-set elves.
Professor Banks is well qualified to discuss discomfort associated with viewing stereoscopic imagery. He designed an impeccable experiment that proved that a vergence-accommodation conflict could cause discomfort (one experimental subject even aborted the sequence due to extreme queasiness). At right a subject bites a bar to ensure accurate distance measurements. But Banks was by no means the first person to note the consequences of a vergence-accommodation (V-A) conflict.
The zone of comfort is often called Percival’s zone in honor of Archibald Percival, who published “The Relation of Convergence to Accommodation and Its Practical Bearing” in Ophthalmic Review in 1892 (and even in that paper, Percival attributed ideas to prior work published by Franciscus Donders in 1864). The reason eye doctors have been concerned about V-A conflict relates, in part, to eyeglasses. If you wear them, you might have noticed a queasy feeling when you put on your first pair or when there was a substantial change in the prescription. But that feeling probably faded as you became accustomed to the V-A conflict.
Another group that was interested in V-A conflict was the original National Television System Committee (NTSC), which began meeting in 1940, the year this off-screen photo was taken. WRGB was named in honor of Dr. Walter Ransom Gail Baker, the engineer who became the head of the NTSC (the initials also stand for white-red-green-blue color systems).
The first NTSC came up with the standard for American black-&-white television, but they were also concerned about color. One of their concerns was that simple lenses (like those in our eyes) cannot focus red and blue in the same place at the same time. The change in focus is a change in accommodation, potentially leading to a V-A conflict. In other words, color TV, in theory, could have made people sick.
In fact, the NTSC concluded that it wouldn’t, based on such work as a paper by Technicolor research director Leonard Troland published in the 1926 American Journal of Physiological Optics specifically related to color motion pictures and the V-A conflict. But, even if color TV would have made viewers sick in 1926, would it always have done so?
Consider, for example, a short movie shot by the Lumiere brothers in 1895, L’arrivée d’un train en gare de La Ciotat (The Arrival of a Train at the Station of La Ciotat). The original looked a little better than what’s shown here, but it was black-&-white and silent. And it’s clear that the train is not heading straight towards the camera.
Nevertheless, here is a report (translated from the original French) from Henri de Parville, an audience member at an early screening. “One of my neighbors was so much captivated that she sprang to her feet… and waited until the car disappeared before she sat down again.” The same reaction was not reported from screenings of other movies, such as one of workers leaving the Lumiere factory. In other words, it seems as though the crude, silent, black-&-white movie made at least one audience member react as though there were a locomotive in the screening room.
About a quarter-century later, Thomas Edison conducted what he called “tone tests,” at which audience members were blindfolded or placed in a dark room and asked if they could tell the difference between a live opera singer and a mechanical phonograph recording of one. Here’s a contemporary account from the Pittsburgh Post in 1919 about a test conducted at a concert hall. “It did not seem difficult to determine in the dark when the singer sang and when she did not. The writer himself was pretty sure about it until the lights were turned on again and it was discovered that [the singer] was not on the stage at all and that the new Edison [phonograph] alone had been heard.”
It might seem ridiculous to readers today that a viewer could be scared by a silent, black-&-white movie of a train or that a listener couldn’t tell the difference between a live singer and a mechanical recording of one (in fairness, I should point out that one of the singers revealed, many years later, that she’d taught herself to sound like a phonograph recording). But that’s because we’ve learned to perceive the differences between those recordings and reality.
There are many examples of such perception education. You might have outgrown your childhood carsickness, for example, just as sailors get over seasickness.
In 3D, research into the amount of time it takes subjects to fuse stereoscopic images has found not only improvement with experience but even the ability of those who underwent the experiments to fuse stereoscopic images more rapidly when tested again after a very long period of no exposure to stereoscopic images. 3D perception, it seems, comes back, just like riding a bicycle. And some eye doctors specialize in training people with stereoscopic perception problems.http://www.vision3d.com/
There are two pages of health warnings in the manuals of Samsung 3DTVs, and at least some of them may be very well justified by such issues as the vergence-accommodation conflict. But that doesn’t mean viewers will always have problems watching 3DTV.
Tags: 3d, accommodation, BaMbuti, carsickness, Colin M. Turnbull, convergence, Edison, football, health, history, illness, In-three, Lumiere, Martin Banks, NTSC, perception, Percival's zone, Samsung, stereoscopic, Technicolor, Uncle Sheby's ABZ Book, vergence, WRGB


Some stereoscopic camera
rigs are said to be so precise that correction is not necessary. Although some had seen it previously, 3ality’s small, relatively lightweight TS-5 rig (shown at left) was officially introduced at IBC 2010. Zepar introduced an even-smaller stereoscopic lens system (shown at right) for a single camera, reducing the need for correction. Such 3D-lens systems normally raise concerns of resolution and sensitivity loss, but Zepar’s is intended to be mounted on the Vision Research Phantom 65, which has plenty of each.




Whereas at IBC 2009 some 25 new camera models were introduced, at IBC 2010, besides the V3i stereoscopic camera and the AF100/101, the main introductions were Canon’s XF100 and XF105 camcorders and IDT’s palm-sized, 2K, high-speed NR5. There were also compact versions of NHK’s 8K Super Hi-Vision cameras from Hitachi and Ikegami. But there were significant wide-angle lens introductions from Polecam (HRO69, at left) and Theia (MY125) for 1/3-inch-format cameras, offering horizontal acceptance angles of 69 and 125 degrees, respectively. 
Other acquistion-related introductions at IBC included a video whiteboard system from Vaddio that does not require a computer, a version of Sennheiser’s MKE-1 lavalier microphone in which every part, from cable to connector to windscreen, is paintable to precisely match costume color, and a wireless tally system from Brick House Video. Capable of dealing with up to eight cameras, the Tally Ho! handles both on-air and preview/iso tally, and the charger for the tally modules doubles as the system transmitter.
If IBC 2010 wasn’t about new cameras, it did offer many new introductions in storage and distribution. There was, for example, AJA’s new, small, lightweight, camera-mountable Ki Pro Mini (left). Then there was the even smaller and lighter Atomos Ninja (right), intended specifically for use with certain types of cameras. And Cinedeck Extreme v. 2.0 allows direct use of Avid’s DNxHD codec. Sonnet’s Qio MR brings the ability to play essentially all popular types of camcorder flash cards (including Panasonic’s P2 and Sony’s SxS) to Windows-based tower computers.
Then there were transportable systems, bigger than those above but still usable in the field. One was the Globalstor Extremestor Transport. Comparably sized but serving a very different function was Marvin (left), from Marvin Technologies. It accepts almost any form of field recording and then, according to preselected options, automatically makes copies, including archival tape cartridges and DVD screening copies.


It is, however, a 3D camera, but unlike any other. Its image sensors (the prism optical blocks with chips attached) move horizontally.
[The photo above, by the way, like the others in this post from Canon Expo, was shot by Mark Forman <
The Internationale Funkausstellung (IFA) in Berlin is an example of one of the latter. It’s an international consumer electronics show.
Perhaps such glasses-free 3D leads to a greater sensation of immersion, but there are other ways to create (or increase) an immersive sensation. Consider, for example, the CAVE (Cave Automatic Virtual Environment), a room with stereoscopic projections on at least three walls and the floor (sometimes all surfaces). The photo here is of a CAVE at the University of Illinois in 2001 (it was developed there roughly 10 years earlier). SGI brought a CAVE to the National Association of Broadcasters convention shortly after it was developed.
The result is definitely mixed reality, a combination of stereoscopic imagery with unprocessed vision, with the 3D virtual images conforming to objects and views in the “real world.” Virtual images can even be mapped onto real-world surfaces, with the cameras in the headgear telling the processors how to warp the virtual images appropriately. This photo shows a complex version of the headgear; other mixed-reality viewers at Canon Expo looked little different from some 3D glasses. Canon’s “interactive mixed reality” brochure showed people wearing the headgear walking around and collaboratively discussing an object that doesn’t exist.
Another form of immersion involves capturing 360-degree images. At left is the Immersive Media Dodeca® 2360 camera system, combining the images from 11 different cameras and lenses into a seamless panorama. At Canon Expo, a 360-degree view was achieved with a single lens, a single imaging chip (8984 x 5792, with 3.2 μm pixel pitch) and a mirror shaped like a cross between a donut and a cone that is, in the words of one high-ranking Canon employee, “the single most-precise optical component the company makes.” The whole package forms a roughly fist-sized bump.
Across the room from Canon’s 360-degree system, however, was their version of ultra-high resolution, with roughly eight times the detail of 1080-line HDTV in both directions.
The hyper-resolution image sensor had a roughly full-frame 35mm format (comparable to that in the Canon EOS 5D Mark II DSLR), already roughly four-and-a-half times taller than a 2/3–inch format image sensor. A few feet away was another new sensor that was larger — much larger. It was made from a semiconductor wafer the size of a dinner plate, and the sensor itself was the size of an old 8-inch-square floppy disk — huge!
Canon Expo demonstrated advances in both immersiveness (aside from the 360-degree and mixed-reality systems, there was also the 9-meter dome projection shown at right) and in spatial sharpness (the hyper-resolution and giant image sensors, the latter because it can deliver more contrast ratio, which affects sharpness). There are also temporal sharpness (high frame rate) and spatio-temporal sharpness, both of which affect our perceptions of sharpness. I found no demonstrations of increased temporal or dynamic resolution at Canon Expo, but that doesn’t mean they’re not being developed.
The images at left are portions taken from BBC R&D White Paper number 169 on “High Frame-Rate Television” published in September 2008. It’s available here:
Ever since its earliest days, color video has been based on three color primaries. As this chromaticity diagram shows, however, human vision encompasses a curved space of colors, whereas any three primaries within that space define a triangle, excluding many colors.







When
sound was added to movies, something similar happened. At the left is the four-units-wide by three-units-high (4:3 or 1.33:1 aspect ratio) frame of silent movies. When a soundtrack was added, it impinged on the frame, giving the picture shape a much squarer 1.16:1 aspect ratio. The so-called Academy aperture shrunk the picture (losing resolution) to return to a somewhat wider shape (1.375:1).






In acquisition technology, for example, LED lighting was near ubiquitous, with focusable instruments, such as the Litepanels Sola, sometimes painfully bright. Panasonic and Sony both showed models of future inexpensive video cameras with large-format imagers, and Aaton joined the range of those offering “digital magazines” for film cameras. In small formats, GoPro’s Hero is a complete HD camcorder weighing just three ounces.
In presentation, there was a reference picture monitor from Dolby (seen in almost its final form at the HPA Tech Retreat). Several booths had OLED monitors, from 7-inch at Sony to 15-inch at TVLogic. Wohler’s Presto router has an LCD video display on each button. And Ostendo’s CDM43 is a curved monitor with a 30:9 aspect ratio.
That barely scratches the surface of the non-3D news from NAB. And then there was 3D.
Atop a tower of Fujinon’s NAB booth, Pace showed something that recognizes the current economics of 3D. With virtually no 3DTV audience, it’s hard to justify separate 3D productions, but, with such major players as ESPN, DirecTV, Discovery, and Sky involved in 3D, the elephant cannot be ignored, either. So the Pace Shadow system places a 3D rig atop the long lens of a typical 2D sports camera. Furthermore, it interconnects the controls (in a variety of selectable ways) so that the operator of the 2D camera need not be concerned about shooting 3D: one camera position, one operator, different 2D and 3D outputs.
There was much more 3D at the show, in every field of video technology (and, perhaps even audio). In acquisition, for example, aside from integrated cameras, 3D mounts, and even individual cameras designed specifically for 3D (like Sony’s HDC-P1), there were also 3D lens adaptors, precision-matched lenses, precision lens controls, and even relay optics intended to allow wider cameras to be placed closer together, as in this picture shot by Eric Cheng of WetPixel.com:
At the other end of the 3D chain, there were both plasma and LCD autostereoscopic (no-glasses) displays using both lenticular and parallax-barrier technology, small OLED displays with active-shutter glasses and giant LED screens with passive circularly polarized glasses. There were LCD and plasma screens (up to 152-inch at Panasonic) and DLP rear-projectors using active-shutter glasses, and both LCD and laser projection using passive polarized glasses.
There were dual-panel displays with beam splitters, and displays intended to be viewed through long strips of fixed polarized materials (to accommodate all viewers’ heights). There were many anaglyph displays in the three-different primary-and-complement color combinations. There were 3D viewfinders using glasses and others with displays for each eye.
Japan’s Burton showed a laser-plasma display that creates 3D images in mid-air. Normally, they’ve viewed through laser-protection goggles, as in the image at the right at the top of this post. But as a safety measure, they showed them instead inside an amber tube at NAB.
storage, it seems that everyone who had anything that could record images had a version that could do so in 3D. Even Convergent Design’s tiny Nano was available in a 3D version. The Abekas Mira is an eight-channel digital production server — or it’s a four-channel 3D digital production server. Want an uncompressed 3D field recorder? Keisoku Giken’s UDR-D100 was just one such product at the show.
There was 3D coax (Belden 1694D, complete with anaglyph color coding). Ryerson University is doing eye-tracking research on what viewers look at in 3D and whether it’s different from HD and 4K.
At least HDTV did eventually penetrate U.S. households. Visitors to NAB conventions in the early 1980s could see aisle after aisle of exhibits claiming compatibility with one or both competing standards for teletext. One standard was being broadcast on CBS and NBC; the other on TBS. There were professional and consumer equipment manufacturers and services offering support. Based on the quantity and diversity of promotion at NAB, it was hard to imagine that teletext would not take off in the U.S.