Three-dimensional display technologies

Three-dimensional display technologies

Jason GengIEEE Intelligent Transportation Systems Society, 11001 Sugarbush Terrace, Rockville, Maryland20852, USA

Jason Geng: [email protected]

AbstractThe physical world around us is three-dimensional (3D), yet traditional display devices can showonly two-dimensional (2D) flat images that lack depth (i.e., the third dimension) information. Thisfundamental restriction greatly limits our ability to perceive and to understand the complexity ofreal-world objects. Nearly 50% of the capability of the human brain is devoted to processingvisual information [Human Anatomy & Physiology (Pearson, 2012)]. Flat images and 2D displaysdo not harness the brains power effectively. With rapid advances in the electronics, optics, laser,and photonics fields, true 3D display technologies are making their way into the marketplace. 3Dmovies, 3D TV, 3D mobile devices, and 3D games have increasingly demanded true 3D displaywith no eyeglasses (autostereoscopic). Therefore, it would be very beneficial to readers of thisjournal to have a systematic review of state-of-the-art 3D display technologies.

1. Fundamentals of Three-Dimensional DisplayThe physical world around us is three-dimensional (3D); yet traditional display devices canshow only two-dimensional (2D) flat images that lack depth (the third dimension)information. This fundamental restriction greatly limits our ability to perceive and tounderstand the complexity of real-world objects. Nearly 50% of the capability of the humanbrain is devoted to processing visual information [1]. Flat images and 2D displays do notharness the brains power effectively.

If a 2D picture is worth a thousand words, then a 3D image is worth a million. This articleprovides a systematic overview of the state-of-the-art 3D display technologies. We classifythe autostereoscopic 3D display technologies into three broad categories: (1) multiview 3Ddisplay, (2) volumetric 3D display, and (3) digital hologram display. A detailed descriptionof the 3D display mechanism in each category is provided. For completeness, we alsobriefly review the binocular stereoscopic 3D displays that require wearing specialeyeglasses.

For multiview 3D display technologies, we will review occlusion-based technologies(parallax barrier, time-sequential aperture, moving slit, and cylindrical parallax barrier),refraction-based (lenticular sheet, multiprojector, prism, and integral imaging), reflection-

2013 Optical Society of AmericaOCIS codes: (090.2870) Holographic display; (110.6880) Three-dimensional image acquisition; (120.2040) Displays

NIH Public AccessAuthor ManuscriptAdv Opt Photonics. Author manuscript.

NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

based, diffraction-based, illumination-based, and projection-based 3D display mechanisms.We also briefly discuss recent developments in super-multiview and multiview with eye-tracking technologies.

For volumetric 3D display technologies, we will review static screen (solid-stateupconversion, gas medium, voxel array, layered LCD stack, and crystal cube) and sweptscreen (rotating LED array, cathode ray sphere, varifocal mirror, rotating helix, and rotatingflat screen). Both passive screens (no emitter) and active screens (with emitters on thescreen) are discussed.

For digital hologram 3D displays, we will review the latest progress in holographic displaysystems developed by MIT, Zebra Imaging, QinetiQ, SeeReal, IMEC, and the University ofArizona.

We also provide a section to discuss a few very popular pseudo 3D display technologiesthat are often mistakenly called holographic or true 3D displays and include on-stagetelepresence, fog screens, graphic waterfalls, and virtual reality techniques, such as Vermeerfrom Microsoft.

Concluding remarks are given with a comparison table, a 3D imaging industry overview,and future trends in technology development. The overview provided in this article shouldbe useful to researchers in the field since it provides a snapshot of the current state of the art,from which subsequent research in meaningful directions is encouraged. This overview alsocontributes to the efficiency of research by preventing unnecessary duplication of alreadyperformed research.

1.1. Why Do We Need 3D Display?

There have been few fundamental breakthroughs in display technology since the advent oftelevision in the 1940s. A clich often used when describing the progress of computertechnology goes like this: If cars had followed the same evolutionary curve that computershave, a contemporary automobile would cost a dollar and could circle the Earth in an hourusing a few cents worth of gasoline. Applying the same metaphor to information displaydevices, however, would likely find us at the wheel of a 1940s vintage Buick.

Conventional 2D display devices, such as cathode ray tubes (CRTs), liquid crystal devices(LCDs), or plasma screens, often lead to ambiguity and confusion in high-dimensional data/graphics presentation due to lack of true depth cues. Even with the help of powerful 3Drendering software, complex data patterns or 3D objects displayed on 2D screens are stillunable to provide spatial relationships or depth information correctly and effectively. Lackof true 3D display often jeopardizes our ability to truthfully visualize high-dimensional datathat are frequently encountered in advanced scientific computing, computer aided design(CAD), medical imaging, and many other disciplines. Essentially, a 2D display apparatusmust rely on humans ability to piece together a 3D representation of images. Despite theimpressive mental capability of the human visual system, its visual perception is not reliableif certain depth cues are missing.

Geng Page 2

Adv Opt Photonics. Author manuscript.

NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

Figure 1 illustrates an example of an optical illusion that demonstrates how easy it is tomislead the human visual system in a 2D flat display. On the left of the figure are some bitsand pieces of an object. They look like corners and sides of some 3D object. After puttingthem together, a drawing of a physically impossible object is formed in a 2D screen (right-hand side of Fig. 1). Notice that, however, there is nothing inherently impossible about thecollection of 2D lines and angles that make up the 2D drawing. The reason for this opticalillusion to occur is lack of proper depth cues in the 2D display system. To effectivelyovercome the illusion or confusion that often occurs in visualizing high-dimensional data/images, true volumetric 3D display systems that preserve most of the depth cues in an imageare necessary.

1.2. What Is a Perfect 3D Display?

True 3D display is the holy grail of visualization technology that can provide efficienttools to visualize and understand complex high-dimensional data and objects. 3D displaytechnologies have been a hot topic of research for over a century [227].

What is a perfect 3D display? A perfect 3D display should function as a window to theworld through which viewers can perceive the same 3D scene as if the 3D display screenwere a transparent window to the real-world objects. Figure 2 illustrates the window tothe world concept. In Fig. 2(a), a viewer looks at 3D objects in the world directly. We nowplace a 3D display screen between the viewer and the 3D scene. The 3D display deviceshould be able to totally duplicate the entire visual sensation received by the viewer. In otherwords, a perfect 3D display should be able to offer all depth cues to its viewers [Fig. 2(b)].

1.3. Depth Cues Provided by 3D Display Devices

Computer graphics enhance our 3D sensation in viewing 3D objects. Although an enhanced3D image appears to have depth or volume, it is still only 2D, due to the nature of the 2Ddisplay on a flat screen. The human visual system needs both physical and psychologicaldepth cues to recognize the third dimension. Physical depth cues can be introduced only bytrue 3D objects; psychological cues can be evoked by 2D images.

There are four major physical depth cues the human brain uses to gain true 3D sensation [2](Fig. 3):

1. Accommodation is the measurement of muscle tension used to adjust the focallength of eyes. In other words, it measures how much the eye muscle forces theeyes lenses to change shape to obtain a focused image of a specific 3D object inthe scene, in order to focus the eyes on the 3D object and to perceive its 3D depth.

2. Convergence is a measurement of the angular difference between the viewingdirections of a viewers two eyes when they look at the same fixation point on a 3Dobject simultaneously. Based on the triangulation principle, the closer the object,the more the eyes must converge.

3. Motion parallax offers depth cues by comparing the relative motion of differentelements in a 3D scene. When a viewers head moves, closer 3D objects appear tomove faster than those far away from the viewer.

Geng Page 3


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

4. Binocular disparity (stereo) refers to differences in images acquired by the lefteye and the right eye. The farther away a 3D object is, the farther apart are the twoimages.

Some 3D display devices can provide all of these physical depth cues, while otherautostereoscopic 3D display techniques may not be able to provide all of these cues. Forexample, 3D movies based on stereo eyeglasses may cause eye fatigue due to the conflict ofaccommodation and convergence, since the displayed images are on the screen, not at theirphysical distance in 3D space [28].

The human brain can also gain a 3D sensation by extracting psychological depth cues from2D monocular images [3]. Examples (Fig. 4) include the following:

1. Linear perspective is the appearance of relative distance among 3D objects, suchas the illusion of railroad tracks converging at a distant point on the horizon.

2. Occlusion is the invisible parts of objects behind an opaque object. The humanbrain interprets partially occluded objects as lying farther away than interposingones.

3. Shading cast by one object upon another gives strong 3D spatial-relationship clues.Variations in intensity help the human brain to infer the surface shape andorientation of an object.

4. Texture refers to the small-scale structures on an objects surface that can be usedto infer the 3D shape of the object as well as its distance from the viewer.

5. Prior knowledge of familiar sizes and the shapes of common structuresthe waylight interacts with their surfaces and how they behave when in motioncan beused to infer their 3D shapes and distance from the viewer.

The human visual system perceives a 3D scene via subconscious analysis with dynamic eyemovements for sampling the various features of 3D objects. All visual cues contribute to thisdynamic and adaptive visual sensing process.

Different depth cues have different effects at different stand-off viewing distances. Figure 5illustrates the general trends of variation of some depth cues as a function of stand-offviewing distance (see Hong et al. [27] and Hoffman et al. [28] for details). In general, theeffects of major physical depth cues decrease with the increase of stand-off viewingdistance, while the effects of psychological depth cues remain the same.

It is often quite difficult for a 3D display device to provide all the physical andpsychological depth cues simultaneously. Some of the volumetric 3D display techniques, forexample, may not be able to provide shading or texture due to the inherently transparentnature of displayed voxels. Some 3D display technologies, such as stereoscopic display,provide conflicting depth cues about the focusing distance and eye converging distance, aphenomenon that is often referred as the accommodation/convergence breakdown (to bediscussed in Section 2.5).

Geng Page 4


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

1.4. Plenoptic Function

In 1991, Adelson and Bergen [29] developed the concept of the plenoptic function (Fig. 6)to describe the kinds of visual stimulation that could be perceived by vision systems. Theplenoptic function is an observer-based description of light in space and time. Adelsonsmost general formulation of the plenoptic function P is dependent on several variables:

the location in space from where light being viewed or analyzed, described by a 3Dcoordinate (x; y; z);

the direction from which the light approaches this viewing location, given by twoangles (, );

the wavelength of the light ; and the time of the observation t.

The plenoptic function can thus be written in the following way:

Note that the plenoptic function and the light field [8] to be discussed in Section 3.1 havesimilarity in describing the visual stimulation that could be perceived by vision systems.

1.5. From 2D Pixel to 3D Voxel (or Hogel)

Most 2D display screens produce pixels that are points emitting light of a particular colorand brightness. They never take on a different brightness or color hue no matter how or fromwhere they are viewed. This omnidirectional emission behavior prevents 2D display screensfrom producing a true 3D sensation.

The profound insight offered by plenoptic function and light field theories reveals thatpicture components that form 3D display images, often called voxels (volumetric pictureelements) or hogels (holographic picture elements) must be directional emitterstheyappear to emit directionally varying light (Fig. 7). Directional emitters include not only self-illuminating directional light sources, but also points on surfaces that reflect, refract, ortransmit light from other sources. The emission of these points is dependent on theirsurrounding environment.

A 3D display mimics the plenoptic function of the light from a physical object (Fig. 7). Theaccuracy to which this mimicry is carried out is a direct result of the technology behind thespatial display device. The greater the amount and accuracy of the view informationpresented to the viewer by the display, the more the display appears like a physical object.On the other hand, greater amounts of information also result in more complicated displaysand higher data transmission and processing costs.

1.6. Classification of 3D Display Technology

There have been a number of books and review articles on the topic related to 3D displaytechnologies in the past [227]. They formed a rich knowledge base in this fascinating field.In this article, we attempt to organize this rich set of domain knowledge bases, plus some of

Geng Page 5


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

the latest state-of-the-art developments, into a unified framework. Figure 8 presents aclassification chart of 3D display technologies. Two fundamentally different categories arethe binocular stereo display technologies that rely upon special eyeglasses worn by viewersfor obtaining 3D sensation and the autostereoscopic 3D display technologies that are glassesfree and in which viewers can gain a 3D sensation via their naked eyes. There are threemajor classes in the autostereoscopic 3D display technologies, namely, multiview 3Ddisplay, volumetric 3D display, and holographic display.

In the following sections, we will provide brief discussions on each technique listed in Fig.8. We try to highlight the key innovative concept(s) in each opto-electro-mechanical designand to provide meaningful graphic illustration, without getting bogged down in too muchtechnical detail. It is our hope that readers with a general background in optics, computergraphics, computer vision, or other various 3D application fields can gain a sense of thelandscape in the 3D display field and benefit from this comprehensive yet concisepresentation when they carry out their tasks in 3D display system design and applications.

2. Stereoscopic Display (Two Views, Eyeglasses Based)We now review a number of binocular stereoscopic display techniques. The binocularstereoscopic display techniques require viewers to wear special eyeglasses in order to seetwo slightly different images (stereo pairs) in two different eyes. Having a description of the3D contents in a scene allows computation of individual perspective projections for each eyeposition. The key issue is to properly separate the stereo image pair displayed on the samescreen to deliver the left image to the left eye and the right image to the right eye. This isalso referred to as the stereo-channel separation problem. Numerous techniques have beendeveloped to separate stereo channels based on the differences in spectral, polarization,temporal, and other characteristics of the left and right images (Fig. 9). We now provide abrief survey of these techniques.

2.1. Color-Interlaced (Anaglyph)

In anaglyph displays, the left- and right-eye images are filtered with near-complementarycolors (red and green, red and cyan, or green and magenta, and the observer wearsrespective color-filter glasses for separation (Fig. 10). Employing tristimulus theory, the eyeis sensitive to three primary colors: red, green, and blue. The red filter admits only red,while the cyan filter blocks red, passing blue and green (the combination of blue and greenis perceived as cyan). Combining the red component of one eyes view with the green andblue components of the other view allows some limited color rendition (binocular colormixture). Color rivalry and unpleasant after-effects (transitory shifts in chromaticadaptation) restrict the use of the anaglyph method.

ColorCode 3D is a newer, patented stereo viewing system deployed in the 2000s bySorensen et al. [30] that uses amber and blue filters. Notably, unlike other anaglyph systems,ColorCode 3D is intended to provide perceived nearly full-color viewing (particularly withinthe RG color space) with existing television and paint media. As shown in Fig. 11, one eye(left, amber filter) receives the cross-spectrum color information and one eye (right, blue

Geng Page 6


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

filter) sees a monochrome image designed to give the depth effect. The human brain tiesboth images together.

2.2. Polarization-Interlaced Stereoscopic Display

Polarization-interlaced stereoscopic display techniques (Fig. 12) are very well suited forvideo projection. When using projectors with separate optical systems for the primarycolors, the left- and right-view color beams should be arranged in identical order to avoidrivalry. The light flux in liquid crystal (LC) projectors is polarized by the light valves.Commercial LC projectors can be fitted for stereo display by twisting the originalpolarization direction via half-wave retardation sheets to achieve, e.g., the prevalent V-formation.

Stereo projection screens must preserve polarization. Optimal results have been reported foraluminized surfaces and for translucent opaque acrylic screens. Typical TV rear-projectionscreens (a sandwiched Fresnel lens and a lenticular raster sheet) depolarize the passing light.LC-based, direct-view displays and overhead panels have recently been marketed [31]. Theirfront sheet consists of pixel-sized micropolarizers, which are tuned in precise register withthe raster of the LCD. The left- and right-eye views are electronically interlaced line-by-lineand separated through a line-by-line change of polarization.

2.3. Time-Multiplexed Stereoscopic Display

The human visual system is capable of merging the constituents of a stereo pair across atime lag of up to 50 ms. This memory effect (or persistence of vision) [32,33] is exploitedby time-multiplexed displays (Fig. 13). The left- and right-eye views are shown in rapidalternation and synchronized with an active LC shutter, which opens in turn for one eyewhile occluding the other eye. The shutter system is usually integrated in a pair of spectaclesand controlled via an infrared link. When the observer turns away from the screen, bothshutters are switched to be transparent. Time-multiplexed displays are fully compatible for2D presentation. Both constituent images are reproduced at full spatial resolution by a singlemonitor or projector, thus avoiding geometrical and color differences. The monitor-typesystems have matured into a standard technique for 3D workstations.

2.4. Head-Mount Display

Figure 14 shows a head-mount display (HMD) with a separate video source displayed infront of each eye to achieve a stereoscopic effect. The user typically wears a helmet orglasses with two small LCD or organic light-emitting device (OLED) displays withmagnifying lenses, one for each eye [34]. Advanced free-form optical design can improvethe performance and reduce the size [35]. The technology can be used to show stereo films,images, or games, and it can also be used to create a virtual display. HMDs may also becoupled with head-tracking devices, allowing the user to look around the virtual world bymoving his or her head, eliminating the need for a separate controller. Performing thisupdate quickly enough to avoid inducing nausea in the user requires a great amount ofcomputer image processing. If six-axis position sensing (direction and position) is used, thewearer may move about within the limitations of the equipment used. Owing to rapid

Geng Page 7


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

advancements in computer graphics and the continuing miniaturization of video and otherequipment, these devices are beginning to become available at more reasonable cost.

Head-mounted or wearable glasses may be used to view a see-through image imposed uponthe real-world view, creating what is called augmented reality. This is done by reflecting thevideo images through partially reflective mirrors. The real-world view is seen through themirrors reflective surfaces. Experimental systems have been used for gaming, where virtualopponents may peek from real windows as a player moves about. This type of system isexpected to have wide application in the maintenance of complex systems, as it can give atechnician what is effectively x-ray vision by combining computer graphics rendering ofhidden elements with the technicians natural vision. Additionally, technical data andschematic diagrams may be delivered to this same equipment, eliminating the need to obtainand carry bulky paper documents. Augmented stereoscopic vision is also expected to haveapplications in surgery, as it allows the combination of radiographic data (computed axialtomography scans and magnetic resonance imaging) with the surgeons vision.

2.5. AccommodationConvergence Conflict

One of the major complaints from users of stereoscopic displays is the inconsistency ofdepth cues, a phenomenon called accommodationconvergence conflict.

Figure 15 provides an illustration of this phenomenon. When observers view stereoscopicimages displayed on a screen, the eyes muscles focus the eyes at the distance of the displayscreen (i.e., the focal distance) in order to clearly see images displayed on the screen. This isdue to the accommodation function of human eyes. On the other hand, the perception of 3Dobjects provided by the 3D display give the human brain the information that the 3D objectsare at their real distance, such that the convergence of the viewers eyes are on theconvergence distance, As shown in Fig. 15, in stereoscopic displays, the focal distanceis not necessarily equal to the convergence distance. This type of visual conflict (i.e., theaccommodationconvergence conflict) may cause visual confusion and visual fatigue ofhuman visual systems (Hoffman et al. [28]). For some viewers, the accommodationconvergence conflict may cause discomfort and headaches after a prolonged time of viewingstereoscopic displays.

The accommodationconvergence conflict can be ameliorated by increasing the number oflight rays that originate from different views and can be perceived simultaneously by theviewers pupil. Super-multiview [36] is one attempt to overcome this restriction. Ultimately,holographic 3D display [4,5,37] technologies can address this issue.

3. Autostereoscopic 3D DisplayMultiview 3D Display Techniques3.1. Approximate the Light Field by Using Multiviews

The plenoptic function discussed in Section 1.4 describes the radiance along all light rays in3D space, and can be used to express the image of a scene from any possible viewingposition at any viewing angle at any point in time. Equivalently, one can use a light field torepresent the radiance at a point in a given direction. The origin of the light field concept canbe traced back to Faraday [38], who first proposed that light should be interpreted as a field,

Geng Page 8


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

much like the magnetic fields on which he had been working for several years. Gershun [39]coined the term light field. In free space (space with no light occluders), radiance alonglight rays can be described as a four-dimensional (4D) light field, as shown in Fig. 16 [8], orlumigraph [40]. Note that the definition of light field is equivalent to the definition ofplenoptic function.

Formally, the light field as proposed by Levoy and Hanrahan [8] completely characterizesthe radiance flowing through all the points in all possible directions. For a given wavelengthand time, one can represent a static light field as a five-dimensional (5D) scalar function L(x,y, z, , ) that gives radiance as a function of location (x, y, z) in 3D space and the direction(, ) the light is traveling. In 3D space free of light occluders, the radiance values of lightrays do not change along their pathways. Hence, the 5D light field function containsredundant information. This redundancy allows us to reduce the dimension of light fieldfunction from 5D to 4D for completely describing radiance along rays in free space.

The ultimate goal of 3D display systems is to reproduce truthfully the light field generatedby real-world physical objects. This proves to be a very difficult task due to fact that lightfield function is continuously distributed. A viewer of a 3D object is exposed to a potentiallyinfinite number of different views of the scene (i.e., continuously distributed light field).Trying to duplicate a complete light field is practically impossible. A practicalimplementation strategy of light field 3D display is to take a subsample of continuouslydistributed light field function and use a finite number of views to approximate thecontinuous light field function.

Figure 17 illustrates the concept of using a finite number of views to approximate thecontinuously distributed light field that in theory has an infinite number of views in bothdirections. This approximation is viable and practical if the finite number of views issufficiently high that it exceeds the angular resolution of a humans visual acuity.Furthermore, a horizontal parallax-only (HPO) multiview display device can beimplemented that generates the parallax effect on the horizontal direction only, allowing theviewers left and right eyes to see different images, and different sets of images can be seenwhen the viewers head position moves horizontally. Even with many fewer views, and withthe HPO restriction, an autostereoscopic 3D display system can still generate multi-views toevoke its viewers stereo parallax and motion parallax depth cues, thus delivering to theviewer a certain level of 3D sensation.

3.2. Implementation Strategies of Multiview 3D Displays

As shown in Fig. 18, a multiview autostereoscopic 3D display system is able to producedifferent images in multiple (different) angular positions, thus evoking both stereo parallaxand motion parallax depth cues to its viewers. No special eyewear is needed.

There are numerous implementation strategies for multiview autostereoscopic 3D displays.Following a general classification approach proposed by Pastoor and Wpking [18], theycan be categorized into the following broad classes (Fig. 19):

Occlusion based,

Geng Page 9


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

Refraction based,

Diffraction based,

Reflection based,

Illumination based,

Projection based, and

Super-multiview.

Each of these strategies has its own advantages and disadvantages. We discuss briefly eachof them.

3.3. Occlusion-Based Multiview 3D Display Techniques

The occlusion-based multiview 3D display approaches have one thing in common: they allhave blockage(s) in the optical path. Due to parallax effects, parts of the image are hiddenfrom one eye but are visible to the other eye. The technical solutions differ in the number ofviewing slits (ranging from a dense grid to a single vertical slit), in presentation mode (timesequential versus stationary), and in whether the opaque barriers are placed in front of orbehind the image screen (parallax barrier versus parallax illumination techniques).

3.3a. Parallax BarrierThe parallax stereogram was first introduced by Ives in 1902[41]. Parallax barrier display uses an aperture mask in front of a raster display, to mask outindividual screen sections that should not be visible from one particular viewing zone. Sincethe barrier is placed at a well-chosen distance pb in front of the display screen, a parallax-dependent masking effect is enforced. In the HPO design, all masks are vertical grids. Thehorizontally aligned eyes perceive different vertical screen columns. Every other column ismasked out and is visible only to the other eye (Fig. 20).

The design parameters of the parallax barrier must be selected appropriately. Usually, thestand-off distance pe and the pixel pitch p of the screen are predetermined by hardware andviewing conditions. e is half of the eye separation distance. Based on similar trianglegeometry, the appropriate barrier pitch b and the mask distance pb can be computed with

The barrier mask can be dynamically adjusted to suit the viewers position, with atransparent LCD panel rendering the mask slits dynamically [42].

Figure 21 shows a multiview parallax barrier display principle. It is important to produce asmooth transition of motion parallax between adjacent views. Some degree of overlap of theluminance profiles of adjacent views may be necessary to ensure smoothness. Too muchoverlap, however, may increase the blurriness of the displayed images.

The parallax-barrier-based displays have several drawbacks:

Geng Page 10


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

Reduced brightness. Only a small amount of light emitted from pixels passesthrough the parallel barriers. The brightness of a display is thus significantlyreduced.

Limited resolution. For a display with N views, the resolution of any individualview is essentially 1/N of the original display resolution.

Picket fence effect in the monocular image. Since each view sees only one pixelcolumn out of N columns associated with one barrier window, vertical dark linesappear in each view, creating a picket fence effect in the monocular image.

Image flipping artifact when crossing a viewing zone. When the viewersposition is not aligned properly with the intended viewing angle, the viewers lefteye may see the image intended for the right eye and vice versa. This causes aflip artifact and confuses the viewer, so depth is not perceived correctly.

Limited number of viewing zones. When trying to increase the number of views,the width of the dark slit of the barrier increases, while the white slit width remainsthe same, causing display brightness decrease and a picket fence effect.

Diffraction effect caused by a small window. With increase of resolution, theaperture of the parallel barrier becomes smaller, which may introduce diffractioneffects that could spread light rays and degrade image quality.

Some popular consumer electronics products, such as Nintendo 3DS [43], are using parallaxbarrier techniques to produce autostereoscopic 3D display.

3.3b. Time-Sequential Aperture DisplaysOne of the time-sequential aperturedisplays was developed by Cambridge University [10]. This time-sequential 3D display usesa high-speed display screen and a time-multiplex scheme to split N image sequences into Nsynthetic images at different virtual positions, as if there were N image sources in differentpositions. The system shown in Fig. 22 uses a high-speed CRT (~1000 Hz) and an array offerroelectric LC shutters to serve as a fast-moving aperture in the optical system, such thatthe image generated by each shutter is directed toward a corresponding view zone (theimages are labeled in the figure as views) (see [10] for details). At any given time, onlyone shutter is transparent and the rest are opaque. Rapidly changing images shown on thehigh-speed display screen are synchronized with the ON/OFF status of each ferroelectric LCshutter to ensure that the correct image is projected to its view zone.

This approach has been extended to a more elaborate design, as shown in Fig. 23, byKanebako and Takaki [44]. An array of LEDs is used as high-speed switching light sources.Each LED in the array can be controlled and synchronized to switch ON/OFF at a highframe rate. At any given time, only one LED is ON. Due to the directional illuminationnature of the LEDs and the illumination optics associated with them, the illumination beamis directed toward a high-speed spatial light modulator (SLM), such as a digital lightprocessor, a LCD, or liquid-crystal-on-silicon (LCOS). The light beam modulated by theSLM becomes the image and is projected toward a set of imaging lenses. Due to thedirectionality of the illumination beam, there is a one-to-one correspondence relationshipbetween the LEDs and the aperture slots in the aperture array. The light coming out from the

Geng Page 11


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

aperture is projected onto a vertical diffuser screen. Multiview 3D display images can thenbe seen from viewers in wide viewing angles.

The time-sequential display scheme has several advantages: (1) it preserves the full imageresolution of the high-speed display screen to each view of the 3D display, (2) it has simplesystem design architecture and there is no need for multiple screens or projectors, (3)calibration of the system is relatively easy, and (4) the cost of the overall system should below if mass produced.

The potential drawbacks of the time-sequential display scheme include the following: (1) thenumber of views is limited by the speed of the SLM. With a 1000 Hz SLM and 25 Hzrefresh rate of the 3D display, the maximum number of views is N = 1000/25 = 40. (2) Theseparation distance of the apertures is limited by the optical design. Wide viewing angle[say, 100 field of view (FOV)] is difficult to achieve. (3) The brightness of the Cambridgedisplay is limited by the CRT technology and aperture size. If a small aperture is selected inorder to achieve dense distribution of virtual image sources, the brightness of each view willbe greatly reduced.

3.3c. Moving Slit in Front of Display ScreenThe so-called Holotron (St. John [45])is a moving-slit display without additional lenses in front of the CRT. Correspondingcolumns of different views are displayed side-by-side behind the slit aperture (Fig. 24). Asthe slit moves laterally, a new set of multiplexed image columns is displayed on the CRT.This way, a set of N views is displayed as a sequence of partial images, composed of Ncolumns, during a single pass of the slit. The horizontal deflection of the electron beam isrestricted to the momentary position of the partial-image area. To reduce the CRTrequirements (sampling frequency, phosphor persistence), a multiple electron gun CRT inconjunction with a multiple moving-slit panel has been proposed.

3.3d. Cylindrical Parallax Barrier DisplayThe FOV of an autostereoscopic displaysmay be extended to 360 with some clever designs. Figure 25 shows an example of amultiview parallax barrier display, dubbed SeeLinder (Endo et al. [46]), using a rotatingcylindrical parallax barrier and LED arrays for covering a 360 HPO viewing area. Thephysically limited number of LED arrays is overcome by rotating the LED array, while themultiview in the horizontal direction is created by the rotating cylindrical parallax barriers.In each viewing location (zone), a viewer can see an appropriate view of a 3D imagecorresponding to the viewers location. The prototype has a 200 mm diameter and a 256 mmheight. Displayed images have a resolution of 1254 circumferential pixels and 256 verticalpixels. The refresh rate is 30 Hz. Each pixel has a viewing angle of 60, which is dividedinto over 70 views so that the angular parallax interval of each pixel is less than 1. The keydesign parameters include the barrier interval, the aperture width, the width of the LEDs,and the distance between the LEDs and the barriers.

This outward viewing design shown in Fig. 25 is for viewers surrounding the outside ofthe display device. A similar design but for inward viewing is described by Yendo et al.[47], where the viewer is located inside a large-size cylinder. The barrier masks are inside

Geng Page 12


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

and the LEDs are outside. Both barriers and LEDs are stationary and aligned with respect toeach other and rotate together.

3.3e. Fully Addressable LCD Panel as a Parallax BarrierA recent advance inextending the parallel barrier techniques discussed in Section 3.3a is to use a fullyaddressable LCD panel with gray-scale transmittance values in each pixel to modulate thelight field generated by the display system (Lanman et al. [48]). The MIT group alsointroduced the tensor display technique that utilizes compressive light field optimization tosynthesize the transmittance values on each of multiple layers of LCD screens (Wetzstein etal. [49]). This unified optimization framework, based on nonnegative tensor factorization,allows joint multilayer, multiframe light field decompositions, significantly reducingartifacts. It is also the first optimization method for designs combining multiple layers withdirectional backlighting. Since the target light field can be defined to have both horizontaland vertical parallax, this advanced barrier technique is able to produce 3D displays withboth horizontal and vertical parallax. This is a promising direction for barrier-based 3Ddisplay technology development.

3.4. Refraction-Based Multiview 3D Display Techniques

3.4a. Lenticular SheetA lenticular lens sheet consists of a linear array of thick planoconvex cylindrical lenses called lenticules. The function of a lens sheet is opticallyanalogous to that of a parallax barrier screen. However, it is transparent, and therefore theoptical efficiency is much higher than its parallax barrier counterpart. Hess [50] patented astereoscopic display using a one-dimensional (1D) lenticular array in 1912. In the spatialmultiplex design of a multiview 3D display, the resolution of the display screen is splitamong the multiple views. Figure 26 illustrates a HPO spatial multiplex display with fiveviews using a lenticular lens sheet in front of a 2D LCD screen. The lenticular lenses arealigned with the vertical pixel columns on the 2D LCD screen. A pixel column is assigned toa single view for every five columns. If a viewer positions his or her eyes in the correct viewzone, he can actually see stereo images from the spatial multiplex screen. When his headmoves, motion parallax can also be experienced.

The lenticular display has the advantages of utilizing existing 2D screen fabricationinfrastructure. Its implementation is relatively simple and low cost. Although lenticular-based displays offer better brightness and higher possible resolution than parallel-barrier-based displays, lenticular-based displays present their own set of challenges:

Limited resolution. For a display with N views, the resolution of an individualview is essentially 1/N of the original display resolution.

Alignment. Aligning a lenticular sheet with a screen requires significant effort.

Cross talk between views and image flips. This may result in one eye seeing theimage intended for the other eye, causing the human brain to perceive the stereoeffect incorrectly.

Geng Page 13


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

Lenticular-based displays also suffer from problems that plague parallel-barrier-based displays, such as the picket fence problem, limited resolution, and limitednumber of viewing windows.

3.4b. Slanted Lenticular Layer on a LCDSince it is obvious that the resolution of a3D image is reduced in lenticular lens systems, a number of advanced techniques have beendeveloped to compensate for it. One of them, developed by Berkel and co-workers [51,52] atPhilips Research Labs, is a slanted lenticular system to distribute the loss of resolution intoboth the horizontal and vertical directions by slanting the structure of the lenticular lens orrearranging the color filter of the pixels.

Figure 27 shows the relationship between the pixels and the slanted lenticular sheet for aseven-view display. As the LCD is located in the focal plane of the lenticular sheet, thehorizontal position on the LCD corresponds to the viewing angle. Therefore all points on theline AA direct view 3 in a given direction, and all points on line BB direct view 4 inanother direction. The way in which the effect of flipping is reduced is evident by examiningline AA, where view 3 predominates, but with some contribution from view 2. Similarly,for the angle corresponding to line BB, view 4 predominates with some contribution fromview 3.

A number of 3D TV products based on the slanted lenticular screen on LCD techniques arecurrently available in the commercial market, including manufacturers such as Vizio, StreamTV Networks, Alioscopy, RealD, Philips, Sharp, Toshiba, and TLC. Some of them areworking on 4 K UHD and even 8 K versions of 3D TVs.

3.4c. LCD Design for Achieving High-Resolution Multiview 3D Displays Usinga Lenticular or Parallax BarrierOne of the major drawbacks of conventionallenticular-lens-based multiview 3D displays is the loss of full resolution of the SLM in eachview. In fact, for HPO display systems, the horizontal resolution for each view is only 1/N ofthat of SLMs native resolution (N is the number of views).

A number of LCD manufacturers are developing new technologies to address this issue. Forexample, a high-density LCD module has been developed by NLT [53] to split theconventional square pixel into N portions, as shown in Fig. 28. Special lenticular arrays aredesigned to cover each portion with different optics, so that each portion of RGB is directedtoward a different view direction. With this design, each view in the multiview 3D displaywill have the pixel resolution of the original LCD. As of May 2013, NLT has made two-view and six-view prototypes of autostereoscopic 3D display panels [up to 7.2 in. (18.29cm)]for industrial applications.

3.4d. Multiview 3D Display Using Multiple Projectors and a Lenticular SheetFigure 29 shows a method for creating a multiview 3D display using multiple projectors, asdemonstrated by Matusik and Pfister [9]. Each of these projectors creates images for a singlecorresponding view. The projectors form images on a special lenticular reflective screen. Inthe vertical direction, the light is diffused in all directions. In the horizontal direction, thelight is focused onto the reflective diffuser screen and then projected back to the direction of

Geng Page 14


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

the projector. Light actually passes through the vertically oriented lenticular screen twice. Inthe first pass, the lenticular lenses focus the projector pixels onto the diffuse screen. In thesecond pass, the same lenticular lenses redistribute the pixels back to the same angulardirection. A transmissive screen can also be used, together with a double lenticular sheet.

Compared with the spatial multiplex methods, the multiprojector approach can preserve theresolution of projected images for each view. There is no need to split resolution by thenumber of views. Also, the size of the display screen is scalable without increasing cost andcomplexity significantly, due to the use of projectors.

The drawbacks of multiprojector-based 3D display include the following:

Expense. Using either frontal or rear projection methods, such displays areexpensive. The cost of having one projector per view becomes exorbitant for even areasonable number of views.

Difficulty of calibration. These displays also require that the projected imagesmust be aligned precisely with one another. In practical application, maintainingoptical calibration for a large number of projectors is a challenging task.

Despite these problems, experimental systems have been produced with more than 100views.

3.4e. Prism MaskA different solution developed by Schwerdtner and Heidrich [54] usesa single panel and light source in connection with a prism mask (Fig. 30). Alternating pixelcolumns (RGB triples) are composed of corresponding columns of the left and right images.The prisms serve to deflect the rays of light to separated viewing zones.

3.4f. Liquid Crystal LensesIf the shape of LC material can be controlled, it can serveas an optical lens in front of a SLM to direct the light beams to desirable directions in realtime. This basic idea is illustrated in Fig. 31. When the voltage is applied between ITOlayers, the shape of the LC cells forms an array of optical lenses equivalent to the opticalfunction of a lenticular lens array. The LC lens can be used to produce a multiview 3Ddisplay.

The unique advantage of using a LC lens for 3D display is that it is electronically switchablebetween 2D and 3D display modes. This feature solves a major problem faced by existinglenticular-based multiview 3D displays: when displaying 2D contents, the resolution islower than its 2D counterpart. The LC-lens-based switchable display can preserve the nativeresolution of the SLM in 2D display mode.

One of the major issues in current LC lens technology is the irregularity of LC alignment inthe boundary region. This causes serious cross talk among views and deterioration of imagequality. Considerable effort is underway to solve this problem. Huang et al. [55]implemented a multielectrode-driven LC lens (MeD LC Lens). By using multiple electrodes,the shape of the LC lens can be better controlled than that of conventional LC lenses that useonly two electrodes. The shape of the LC lens can be dynamically changed at the imagerefreshing rate, allowing the active scanning of light beams for generating multiview 3D

Geng Page 15


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

display functions. The high frame rate images displayed on a SLM are projected in time-sequential fashion to different viewing directions. The LC lens technique facilitates a high-resolution multiview 3D display, since the resolution of each view image has the fullresolution of the SLM.

3.4g. Integral 3D DisplayLenticular-based autostereoscopic 3D displays provide onlyhorizontal parallax. They generally lack vertical parallax. To achieve parallax in bothdirections, the integral display technique uses spherical lenses instead of cylindrical ones topresent horizontally and vertically varying directional information, thus producing a fullparallax image. The integral photography concept was proposed in 1908 by Lippmann [56].Creating 3D integral imagery by digitally interlacing a set of computer generated 2D viewswas first demonstrated in 1978 at the Tokyo Institute of Technology in Japan [57].Inexpensive lens arrays were produced by extrusion embossing, but these lenses have severemoir effects and are not suitable for integral displays. More sophisticated holographic lensarrays have been demonstrated to enhance the viewing angle and depth of field [58].

Figure 32 shows a typical spherical lens array used by integral 3D display devices. Integral3D displays are less common than their lenticular-based counterparts mostly because evenmore of their spatial resolution is sacrificed to gain directional information.

In the image acquisition stage (pickup step) of an integral imaging system, each individuallens or pinhole will record its own microimage of an object, which is called the elementalimage, and a large number of small and juxtaposed elemental images are produced behindthe lens array onto the recording device. In the 3D display stage, the display device with theelemental image is aligned with the lens array and a spatial reconstruction of the object iscreated in front of the lens array, which can be observed with arbitrary perspective within alimited viewing angle. Therefore, integral imaging suffers from inherent drawbacks in termsof viewing parameters, such as viewing angle, resolution, and depth range due to limitedresolution of the 2D SLM and the lens array itself.

Recent progress in autostereoscopic displays is focused on the enhancement of 3Dresolution as well as smooth parallax [59]. Another example of excellent integralphotography display has an image depth of 5.7 m, as demonstrated by Liao [60]. Althoughintegral imaging provides both vertical and horizontal parallax within a limited viewingangle, low resolution resulting from full parallax is still a problem for practical uses.

3.4h. Moving Lenticular SheetMultiview display can be produced via moving parts,as proposed by Cossairt et al. [61], Goulanian et al. [62], and Bogaert et al. [63]. Figure 33illustrates a concept for using a moving lenticular sheet module at high speed (>30 Hz) tosteer the viewing direction of images displayed on a high-speed display screen. The imagecan be generated by a fast LCD or by projection of a high-speed image sequence from aDLP. At each position of the lenticular sheet module, the high-speed display screenproduces an image corresponding to a particular viewing direction. With the back and forthmotion of the lenticular sheet module, the multiview images are scanned through a widerange of viewing angles.

Geng Page 16


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

The drawbacks for this design are as follows:

1. Accurate micro-motion of a large size lenticular sheet module is difficult toimplement, depending on screen size, weight, and design. Therefore, for differentsizes of displays, different types of motion controllers are needed, leading to highcost of scale-up production.

2. Since the motion of the lenticular sheet module is back and forth, the speed is notconstant. There is a significant variation (from zero to maximum speed) duringevery cycle of motion. The scanning speed of the viewing direction is therefore notconstant. This property may affect viewing performance.

3. For large size screens [e.g., 70 in. (177.80 cm)], such a moving screen moduledesign may be very difficult and costly to implement.

3.5. Reflection-Based Multiview 3D Display

3.5a. Beam Splitter (Half Mirror)A field lens is placed at the focus of a real (aerial)image in order to collimate the rays of light passing through that image without affecting itsgeometrical properties. Various 3D display concepts use a field lens to project the exit pupilsof the left- and right-image illumination systems into the appropriate eyes of the observer.The effect is that the right-view image appears dark to the left eye, and vice versa. Thisapproach generally avoids all the difficulties resulting from small registration tolerances ofpixel-sized optical elements.

Figure 34 shows the basic principle of the beam-splitter-based 3D display proposed byWoodgate et al. [64]. Two LCD panels, the images of which are superimposed by a beamcombiner, are used to display the left and right views. Field lenses placed close to the LCDserve to direct the illumination beams into the appropriate eye. For head tracking, theposition of the light sources must be movable. The head-tracking illumination system can beimplemented, e.g., by monochrome CRTs displaying high-contrast camera images of the leftand right halves of the observers face. Multiple-user access is possible by using multipleindependent illuminators.

3.6. Diffraction-Based Multiview 3D Display

With the diffractive-optical-element (DOE) approach, corresponding pixels of adjacentperspective views are grouped into jumbo pixels (ICVision Display [65] and 3D GratingImage Display [66,18].) Small diffraction gratings placed in front of each partial pixel directthe incident light to the respective images viewing area (first-order diffraction; see Fig. 35).Current prototypes yield images of less than 1.5 in. (3.81 cm) in diameter. Advancedconcepts provide for the integration of image modulation and diffraction of light within asingle, high-resolution SLM [67].

3.6a. Directional Backlight Based on Diffractive OpticsMultiview 3D displaycan also be generated by using a SLM (such as a LCD) screen with a directional backlightmechanism. Fattal et al. [68] from HP Labs introduced an interesting directional backlightdesign to produce wide-angle full parallax views with low-profile volume that is suited formobile devices. Figure 36 illustrates this design concept, as proposed in [68]. Special grating

Geng Page 17


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

patterns are etched or deposited on the surface of a glass or plastic light-guide substrate. Thesubstrate is illuminated by collimated light, and the diffractive patterns scatter the light outof the backlight light-guide substrate. The directions of light scattered are determined by thediffractive patterns, which can be carefully designed to implement the multiview directions.Experimental results presented by [68] shows a 64-view backlight that produces 3D imageswith a spatial resolution of 88 pixels per inch with full parallax and a 90 viewing angle.

3.7. Projection-Based Multiview 3D Display

3.7a. USC: Rotating Screen Light Field Display with a High-Speed ProjectorThe light field display developed by Jones et al. [13] consists of a high-speed imageprojector, a spinning mirror covered by a holographic diffuser, and electronics to decodespecially encoded Digital Visual Interface (DVI) video signals. As shown in Fig. 37, thehigh-speed image projector and a spinning mirror covered by a holographic diffusergenerate a 360 HPO view.

Cossairt et al. [69] developed a 198-view HPO 3D display, based on the Perspectra spatial3D display platform [70]. It is able to produce 3D images with viewer-position-dependenteffects. This was achieved by replacing the rotating screen with a vertical diffuser andaltering the 3D rendering software on the Perspectra.

The light field 3D displays can produce impressive 3D visual effects. There are, however,several inherent problems with existing light field 3D display technologies. For example,some approaches rely upon rotating parts and/or a scanning mechanism to produce lightfield distributions. There is a physical limitation on the display volume due to the existenceof moving parts. The trade-offs between displayed image quality and complexity/costs ofdisplay systems sometimes force designers to sacrifice image quality; thus 3D imageresolution and quality still are not comparable with that of high-end 2D displays.

3.7b. Holografika: Multiple Projectors + Vertical Diffuser ScreenA recentdevelopment in multiview 3D display technology utilizes multiple projectors and aholographic screen to generate a sufficient number of views to produce a 3D display effect.These displays use a specially arranged array of micro-displays and a holographic screen.One of the elements used for making the holographic screen is the vertical diffuser (Fig. 38).Each point of the holographic screen emits light beams of different color and intensity to thevarious directions in a controlled manner. The light beams are generated through a lightmodulation system arranged in a specific geometry, and the holographic screen makes thenecessary optical transformation to compose these beams into a continuous 3D view. Withproper software control, the light beams leaving the various pixels can be made to propagatein multiple directions, as if they were emitted from physical objects at fixed spatiallocations.

The HoloVizo display developed by Holografika [71,72], as shown in Fig. 39, takesadvantage of the 1D diffusion property and uses a number of projectors that illuminate aholographic screen. In the horizontal cross-section view, a viewer can see only one very thinslit of images from each projector, assuming that the screen diffuses light in the verticaldirection only. To generate one viewing perspective, these thin slits from different projectors

Geng Page 18


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

have to be mosaicked together. Therefore, the display requires many projectors to worktogether. For the HoloVizo display system, as many as 80 projectors are used for theprototype. With the mirrors on both sides, these projectors are able to generate as many as200 views due to mirror effect. In one prototype, the system is able to produce ~80 millionvoxels [80 views, each with 720p (1280 720) resolution)].

3.7c. Theta-Parallax-Only DisplayFavalora and Cossairt [73] proposed an interestingdesign of a multiview 3D display that exploits the directional light steering property of aspecial screen. As shown in Fig. 40, the rotating flat screen consists of three layers. The topand bottom layers are diffusers. The middle layer is a directional light steering layer,meaning that all light projected from the bottom of the screen is reflected toward one side ofthe view. With the rotation of the flat screen, the displayed images are scanned 360 aroundthe display device, forming a 360 viewable 3D image display.

Similar displays have since been demonstrated in various laboratories; see, e.g., Uchida andTakaki [74].

3.7d. Projector with a Lenticular Mirror SheetAnother design concept ofprojection-based multiview 3D displays, proposed by Krah at Apple [75], is to use aprojector and a reflective lenticular mirror sheet as the reflective screen. As shown in Fig.41, a projector generates multipixel image projection on each strip of the lenticular sheet.Due to the curvature of the lenticular strip, the reflected beams are directed toward differentdirections. By carefully calibrating the projection image and the location of the lenticularscreen, one can produce a multiview 3D display system that has different images fordifferent viewing perspectives.

3.7e. Projector with a Double-Layered Parallax BarrierThe scheme shown in Fig.20 using a single-layer parallax barrier can also be implemented by using a double layeredparallax barrier and a multiprojector array. Figure 42 shows this concept, as proposed byTao et al. [76]. There are multiple projectors; each generates an image for one view. Allimages are projected on the first parallax barrier layer, which controls the light rays fromeach projector to a specific position on the diffuser screen. The second barrier layer, with thesame pitch and position as the first one, controls the viewing directions of the multiviewimages formed on the diffuser screen. Viewers in different viewing locations would be ableto see different views of the images. Therefore, the system setup becomes the multiview 3Ddisplay.

3.7f. Frontal Projection with Parallax BarrierKim et al. [77] proposed a frontalprojection autosterescopic 3D display using a parallax barrier and some passive polarizingcomponents in front of a reflective screen. The advantages claimed by the authors are thatthe display is both space saving and cost effective in comparison with conventional rearprojection counterparts. Figure 43 illustrates its basic configuration detailed in [77]. Thelight coming out from the projector first passes a polarizer, then passes through a parallaxbarrier to form pixelized images on a polarization-preserving screen. There is a quarter-waveretarding film placed in between the barrier and the screen so that the reflected light from

Geng Page 19


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

the screen has the correct polarization direction. Viewers are able to see multiview 3Dimages coming out from the parallax barrier.

In this design, the projected light actually has to go through the parallax barrier twice. As aresult, the optical efficiency of this design is still quite low.

3.8. Super-Multiview 3D Displays

Due to the physical upper limit on how many views a multiview 3D display can generate,there is always a discontinuity in view switching with respect to the viewing direction. Themotion parallax is often provided in a stepwise fashion. This fact degrades the effectivenessof 3D displays.

Another problem in conventional multiview 3D displays is the accommodationconvergenceconflict. While the human eyes convergence capability perceives the correct depth of 3Dobjects, the accommodation function makes the eyes focus on the display screen. Since thereis a close interaction between convergence and accommodation, this accommodationconvergence conflict problem often causes visual fatigue in 3D viewers.

In attempts to solve these two problems, researchers developed super-multiview (SMV)techniques that try to use an extremely large number of views (e.g., 256 or 512 views) togenerate more natural 3D display visual effects, as proposed by Honda et al. [36], Kajiki etal. [78], and Takaki and Nago [17]. The horizontal interval between views is reduced to alevel smaller than the diameter of the human pupil (Fig. 44). Light from at least two imagesfrom slightly different viewpoints enters the pupil simultaneously [36]. This is called theSMV display condition. In bright light, the human pupil diameter is ~1.5 mm, while in dimlight the diameter is enlarged to ~8 mm. The importance of the SMV condition is that theincrease in number of views to provide multiple views to pupils simultaneously may help toevoke the natural accommodation responses to reduce the accommodationconvergenceconflict, and to provide smooth motion parallax. Regarding the motion parallax, the SMVcondition may improve the realism of 3D images perceived by viewers, since the brainunconsciously predicts the image change due to motion.

Since 1995, multiple SMV prototypes have been built and tested. In 2010, a prototypedisplay with 256 views was constructed using 16 LCD panels and 16 projection lenses [17].The display screen size was 10.3 in. (26.16 cm), and the horizontal pitch of the viewingzones was 1.31 mm. 3D images produced by the prototype display had smooth motionparallax. Moreover, it was possible to focus on the 3D images, which means that theaccommodation function might work properly on the 3D images produced by the prototypedisplay, so that the accommodationconvergence conflict might not occur.

3.9. Eye-Tracking (Position Adaptive) Autostereoscopic 3D Displays

It is well known [10] that autostereoscopic 3D display systems have preferred viewerposition(s), or sweet spots, where the viewers can gain the best 3D sensation. Thelocations of the sweet spots are usually fixed and determined by optical and electronicdesign of the 3D display systems. Many researchers, such as Woodgate et al. [64],Henstchke [79], Wang et al. [80], and Si [81] have attempted to make dynamic changes in

Geng Page 20


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

the optical and electronic design of 3D display systems to adaptively optimize them basedon the current position of the viewers eyes.

For example, in Wang et al. [80], a parallax-barrier-type autostereoscopic display devicewas designed to have multiple parallel strip backlight sources. An eye-tracking technique isused to determine the position of the viewers eyes. An adaptive controller then turns on theproperly selected light source for forming images to the viewers left eye position and righteye position.

A recent development in this direction is shown in the Free3C 3D Display [82], with anautostereoscopic 3D display with UXGA (1200 1600) resolution. The Free2C Desktop 3DDisplay is based on a special head-tracking lenticular-screen 3D display principle, allowingfree 3D head movements with reasonable freedom for single viewers. A lenticular screen,placed in front of an LCD, is moved in both the x and y directions by voice-coil actuators,aligning the exit pupils with the viewers eye locations. No stereo viewing glasses areneeded. The cross talk is controlled below 2%.

An interesting eye-tracking 3D display design was developed by Surman et al. [83] in theEuropean Union funded Multi-User Three-Dimensional Display (MUTED) project. Thelatest version of the MUTED design (Fig. 45) uses an array of lenses (similar to lenticularbut with the width of the lenslet being greater than the pitch), a LCOS projector, and a LCDpanel screen. The lens array is used to steer the direction of light illumination toward thelocation of the viewers eyes, which is determined by an eye-tracking system. The imageprojected by the LCOS projector consists of a series of dots, whose locations are designedfor the lens array to focus onto the viewers eye locations. The optical efficiency is low inthis design since only a small portion of the light produced by the LCOS projector is usedfor generating the directional backlight for the LCD panel.

Eye-tracking techniques have found a great fit in mobile 3D display applications. Forexample, Ju et al. [84] proposed an eye-tracking method using a single camera for mobiledevices. Wu et al. [85] proposed an adaptive parallax barrier scheme that used multiplesubbarriers to adjust the viewing zone location based on the viewers detected eye positions.For single-user application scenarios, eye-tracking-based 3D display technologies showgreat promise to fulfill the needs of providing autostereoscopic 3D display functionality.

3.10. Directional Backlight Designs for Full-Resolution Autostereoscopic 3D Displays

Full-resolution autostereoscopic 3D display can be achieved by using clever directionalbacklight mechanisms, together with high-speed LCD panels. The directional backlightmechanisms generate optically distinct viewing regions for multiple views in a time-multiplexed fashion. In the case of two views, a stereoscopic display is generated. Ingeneral, the directional backlight designs are well suited for providing 3D display capabilityfor mobile devices, providing full LCD resolution for each view with a compact packagesize. Additional advantages of using the directional backlight techniques for 3D displaysinclude avoidance of the perception of flicker and elimination of view reversal, a commoncause of viewer fatigue [86].

Geng Page 21


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

3.10a. Directional Backlight Design by 3MSchultz and St. John at 3M [87]proposed a compact directional backlight design for full-resolution autostereoscopic 3Ddisplay using two LED sources, a reflective film, a specially designed light guide, a 3D filmsheet, and a fast-switching LCD panel (Fig. 46). When the LED light source for the left eyeis on and the LED light source for the right eye is off, the light guide with a simple shallowprism structure (thickness 0.5 mm, made of polymethyl methacrylate) manages to directmost of the light rays toward the predetermined location of the left eye. By synchronizingthe image displayed on the LCD panel with the left-view image, the viewer is able to see theimage corresponding to the left view. Switching the LED light sources facilitates thealternative display of images of the left and right views, and these images are directed, dueto the predetermined directions, toward the corresponding eyes. The 3D film featuresnanometer scale (~130 nm) structures of lenticular and prism. There is no requirement toalign the 3D film with the LCD panel pixel structure. To implement two-view display, aLCD panel with a refresh rate of 120 Hz is used. The refresh rate for the full-resolution 3Ddisplay is thus 60 Hz. The cross talk between the left and right views can be reduced to alevel of less than 10%, with the optimization of overall design parameters. A 9 in. (22.86cm) WVGA (800 480) 3D display was demonstrated in [87].

3.10b. Directional Backlight Design by SONY for a 2D/3D Switchable DisplayMinami and co-workers at SONY [88,89] proposed a 2D/3D switchable back-light design,as shown in Fig. 47. The unique light-guide design allows it to function as a parallax barrieras well as a backlight. The 3D illumination light rays from the light sources placed along theside of the light-guide bounce between the reflective surfaces inside the light guide. Whenthese rays hit the scattering patterns in locations corresponding to the slits in the parallaxbarrier, they are reflected toward the LCD panel, evoking multiview 3D display whensynchronized with the image displayed on the LCD (3D resolution 960 360 pixels for eachview). When the 3D backlights are OFF while the 2D backlights are ON, the system acts asa normal full-resolution 2D display, with a resolution of 1080p (1920 1080 pixels).

3.10c. Four-Direction Backlight with a 12-View Parallax Barrier for a 48-view3D DisplayWei and Huang [90] proposed a four-direction backlight together with a 12-view parallax barrier for a 48-view 3D display. This design combines four majorcomponents: a sequentially switchable LED matrix plate, a dual-directional prism array, a240 Hz LCD panel, and a multiview parallax barrier. As shown in Fig. 48, the LED matrix issequentially switched at 240 Hz among four groups of LEDs (indicated by red, green, blue,and yellow in Fig. 48), synchronized with the 240 Hz LCD panel. The directional prismarrays are placed in alignment with the LED matrix grids. Due to the direction of prisms andthe displacements among the locations of LED groups, the light rays originating from eachLED group are directed toward a viewing direction. There are four light directions in thedesign shown in Fig. 48. The parallax barrier in front of the LCD panel is designed to have12 views. Therefore, the total number of views of the autosterescopic 3D display becomes48 (= 4 12). The viewing angle is 40.

3.10d. Multidirectional Backlighting Using Lenslet ArraysKwon and Choi [91]implemented a multidirectional backlight using a LCD panel, a lenticular lens arrays, and a

Geng Page 22


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

uniformed backlight unit. In a structural design similar to that of the lenticular-basedmultiview 3D display (Section 3.4a), the columns in the LCD panel correspond to theirspecific directions of light projection, due to the optical property of the lenticular lens array.A five-direction backlight design is shown in Fig. 48, where there are five columns of pixelsunder each lenslet. Each of these columns produces a collimated light beam in itspredetermined direction, shown as different colors in Figure 49. Time-multiplexing thesecolumns on the LCD panel can generate multidirectional backlight whose direction ischanged sequentially. Synchronizing the backlight direction with another LCD panel withthe displayed image of the respective directional view can produce an autostereoscopic 3Ddisplay.

The main advantages of this design include its ability to make a thin backlight unit and itscompatibility with the lenticular-based 3D display design. Drawbacks of this design includethe requirement of a high-speed LCD panel (in an N-directional backlight unit, the framerate of the LCD panel is N times that of the 3D image to be displayed) and low brightness oflight output in each direction (1/N of the total light output of the uniformed light source).

4. Volumetric 3D DisplayIn contrast to multiview 3D displays that present the proper view of a 3D image to viewersin corresponding viewing locations, volumetric 3D display techniques to be discussed in thissection can display volumetric 3D images in true 3D space. Each voxel on a 3D image islocated physically at the spatial position where it is supposed to be and reflects light fromthat position toward omni-directions to form a real image in the eyes of viewers. Suchvolumetric 3D displays provide both physiological and psychological depth cues to thehuman visual system to perceive 3D objects, and they are considered more powerful anddesirable devices for human/computer visual interface than existing display devices.

We provide a brief overview of a number of representative 3D volumetric displaytechnologies (Fig. 50). This list is by no means comprehensive or inclusive of all possibletechniques.

4.1. Static Screen Volumetric 3D Displays: Static and Passive Screens

4.1a. Solid-State UpconversionOne of the fundamental requirements for avolumetric 3D display system is to have the entire display volume filled with voxels that canbe selectively excited at any desired location. To achieve this goal, one can have twoindependently controlled radiation beams that activate a voxel only when they intersect.While electron beams cannot be used for such a purpose, laser beams can, provided that asuitable material for the display medium can be found. Figure 51 shows a process known astwo-photon upconversion that can achieve this objective. Briefly, this process uses theenergy of two infrared photons to pump a material into an excited level, from which it canmake a visible fluorescence transition to a lower level. For this process to be useful as adisplay medium, it must exhibit two photon absorptions from two different wavelengths, sothat a voxel is turned on only at the intersection of two independently scanned laser sources.The materials of choice at the present time are the rare-earth particles doped into a glass hostknown as ZBLAN. ZBLAN is a fluorozirconate glass with the chemical name ZrF4-BaF2-

Geng Page 23


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

LaF3-AlF3-NaF. The two-photon upconversion concept for 3D volumetric displays was usedby Downing [92], Lewis et al. [93], and Langhans et al. [94] in building their prototypes.These volumetric 3D displays show promising features, such as that there are no movingparts within the display volume. Major difficulties in producing a practically useful 3Ddisplay using this approach are its scale-up capability (existing prototypes had sizes of a fewinches) and the ability to display multiple colors (different colors usually require differentlasers and screen materials).

4.1b. Gas Medium UpconversionAnother 3D display based on the upconversionconcept employs the intersection of two laser beams in an atomic vapor and subsequentomnidirectional fluorescence from the intersection point [95]. Two lasers are directed viamirrors and xy scanners toward an enclosure containing an appropriate gaseous species(rubidium vapor, for example), where they intersect at 90. Either laser by itself causes novisible fluorescence. However, where both lasers are incident on the same gas atoms, two-step excitation results in fluorescence at the intersecting point. By scanning the intersectionpoint fast enough, a 3D image can be drawn in the vapor. The eye cannot see changes fasterthan about 15 Hz. Therefore, if the image to be displayed is repeatedly drawn faster than thisrate, the image will appear to be steady, even though light may be originating from any onepoint in the volume only a small fraction of the time.

The advantage of this 3D display concept is its scalability: It can be built in almost anydesirable size without significantly increasing the complexity of the system. The technicaldifficulties in implementing this concept include the requirement for a vacuum chamber, theneed to maintain a certain temperature, a limitation on the number of voxels by the speed ofthe scanners, and the eye-safe problem presented by laser beams.

4.1c. Crystal Cube Static Screen Volumetric 3D DisplayA static 3D crystal cube(or any other shape)was developed as a 3D display volume by Nayar and Anand [96] andGeng [97]. Within a block of glass material, a large number of tiny dots (i.e., voxels) arecreated by using a recently available machining technique called Laser SubsurfaceEngraving (LSE). LSE can produce a large number of tiny physical crack points (as small as~0.02 mm in diameter) at desirable (x, y, z) locations precisely within a crystal cube. Thesecracks, when illuminated by a properly designed light source, scatter light in omnidirectionsand form visible voxels within the glass volume, thus providing a true volumetric 3Ddisplay. Locations of the voxels are strategically determined such that each can beilluminated by a light ray from a high-resolution SLM. The collection of these voxelsoccupies the full display volume of the static 3D crystal cube. By controlling the SLMengine to vary illumination patterns, different volumetric 3D images can be displayed insidethe crystal cube. A solid screen with dimensions of 320 mm 320 mm 90 mm was builtand tested [97] (Fig. 52).

Unique advantages of the 3D static screen display technology include

no moving screen;

inherent parallel mechanism for 3D voxel addressing;

Geng Page 24


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

high spatial resolution;

easy-to-implement full color display;

fine voxel size (at a submillimeter level);

no blind spot in the display volume;

display volume that can be of arbitrary shape;

no need for special viewing glasses or any special eyewear to view the 3D images;and

no image jitter that is associated with a moving screen.

4.1d. Laser Scanning to Produce Plasma 3D Display in the AirKimura et al.[98] developed a laser scanning 3D display technique that can produced visible voxels in theair from the plasma generated by the laser. They noticed the phenomenon that, when laserbeams are strongly focused, air plasma emission can be induced only near the focal point.They first succeeded in experiments to display 2D images in the air. The images wereconstructed from dot arrays produced using a combination of a laser and galvanometricscanning mirrors. Later, they extended the scanning and focusing mechanism in 3D space,and produced 3D images in the air [99].

4.2. Static Screen Volumetric 3D Displays: Static and Active Screen

4.2a. Voxel Array: 3D Transparent Cube with Optical-Fiber BundlesMacFarlane [21] at the University of Texas, Dallas, developed an optical-fiber-addressedtransparent 3D glass cube for displaying true 3D volumetric images (Fig. 53). As a fairlystraightforward extension of a 2D LCD screen, this type of 3D display is a 3D arrayconsisting of a stack of 2D pixel elements. The voxels are made from optical resin and aretransparent in their quiescent state. Optical fibers embedded in the glass cube are used toaddress the 3D voxel arrays. The image signal is controlled by a SLM. The collection of allactivated 3D voxels thus forms 3D images in true 3D space.

Obvious advantages of implementing a 3D volumetric display using a static voxel arrayembedded in a 3D transparent glass cube are its conceptual simplicity and that it has nomoving parts. However, without a sweeping screen that translates a 2D element array into a3D voxel array, a huge number of voxels have to be built into the 3D cube, which requires atremendous amount of effort in fabrication. A 3D monitor with 1000 voxels on each side of(X, Y, Z) 3D space would need 1 billion voxels to be built into a glass cube. The task ofaccurately laying out such a huge number of voxels and their associated fibers is beyondcurrent manufacturing capability. A 3D voxel array prototype of 48 48 12 wasconstructed at the University of Texas, Dallas. Many issues related to the manufacturingprocess for achieving high resolution and on matching the diffraction index of the fiber withthe glass fill-in material still remain unsolved.

4.2b. Voxel Array: LED MatrixMany groups have attempted to build volumetric 3Ddisplay devices using 3D matrices of LEDs. For example, an 8 8 8 (= 512) voxel displayprototype was developed by Wyatt at MIT [100]. The concept of these LED array displays is

Geng Page 25


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

straightforward, but implementation is quite challenging if the goal is to develop high-resolution 3D volumetric 3D display systems. In addition to many problems faced by theoptical fiber bundle display, there is an issue of voxel occlusion due to the opaque nature ofLEDs themselves. Cross illumination among LEDs is also a concern.

4.2c. Static Screen Volumetric 3D Display Techniques Based on MultipleLayer LCDsFigure 54 shows a static screen multilayer LCD 3D volume visualizationdisplay proposed by Sadovnik and Rizkin [101]. The volumetric multilayer screen includesmultiple electronically switchable polymer dispersed liquid crystal (PDLC) layers that arestacked. An image projector is used to project sequential sections of a 3D image onto thePDLC screens. The timing of projection is controlled such that activation of the PDLCscreen and the projection of an image section are synchronized. A volumetric 3D image isformed by multiple sections of projected images at different z heights. Note that this 3Ddisplay scheme requires the switching speed of the PDLC to be faster than 0.1 ms.

Sullivan [20,102] proposed a similar design of a multiplanar volumetric 3D display system.The display volume consists of a stack of switchable LC sheets whose optical transmissionrates are switchable by the voltage applied on them that is controlled by a controller andsynchronizer electronics. The LC sheets stay optically clear when there is no voltage appliedon them but become scattering when there is a voltage applied. By synchronizing the timingof image projection from a high-speed image projector and the ON/OFF state of each LCsheet, 2D sections of a 3D image can be displayed in proper 3D locations, thus forming atrue 3D display. A volumetric 3D display system based on this concept was built byLightSpace Technologies [103]. As of 2013, EuroLCDs [104] has received the exclusivelicense for commercializing this volumetric 3D display technology.

Gold and Freeman [105] proposed a layered volumetric 3D display concept that forms ahemispheric-shaped screen. Instead of using a planar stack of switchable LC sheets, thissystem customized the production of each layer with different sizes and integrated themtogether to form a hemispherical volume for displaying 3D images.

Leung et al. [106] proposed a 3D real-image volumetric display that employs a successivestack of transparent 2D LCD planar panels. Each layer of LCD planar panels is controlled todisplay a section of a 3D image, and the viewable image combines images displayed on eachof the multiple layers of LCDs, thus forming a true 3D volumetric display system.

Koo and Kim [107] proposed a multilayered volumetric 3D display system that uses a stackof flexible transparent display elements, such as OLEDs, to form a volumetric 3D displayscreen. The flexibility of the OLED element allows for various conformal shapes of screendesigns, such as cylindrical, spherical, and/or cone-shaped volumes.

Advantages of these multi-planar LC-based static screen volumetric 3D display schemesinclude

1. Static screen. No moving parts in the 3D display system.

2. Simple design. Systems using LC sheets need only one-pixel LC sheets.

Geng Page 26


NIH

-PA

Author M

anuscriptN

IH-P

A A

uthor Manuscript

NIH

-PA

Author M

anuscript

Drawbacks of these schemes include

1. Requirement of high-speed switching LC sheets. Assume that a reasonablerefresh rate of a high-quality 3D image is 100 Hz, and there are 100 layers of LCsheets. The switching time for each LC sheet is thus 1/(100 100) = 100 s = 0.1ms. This is about 1 to 2 orders of magnitude faster than any commercially availableLC material can achieve, and it is very difficult for existing technology to handlesuch a high switching speed.

2. Low image brightness due to short exposure time. Brightness perceived byhuman eyes depends not only on the intensity of the light source but also on theexposure time of the light source. The shorter the exposure time, the dimmer thelight appears. With only 50 s maximum exposure time (considering the ON/OFFtransit time of LC sheets), it is very difficult to achieve reasonable image brightnessfor 3D display.

3. Low brightness due to optical transmission loss of projected light goingthrough multiple LC sheets. Even with high-quality LC sheets that have optimallight transmission efficiency of 97% each, after 35 layers of LC sheets, the lightintensity is reduced to 50% of its original strength. It drops further to 25% after 40layers of passage.

4.3. Swept Screen Volumetric 3D Displays: Passive Sweeping Screen

4.3a. Volumetric 3D Display Using a Sweeping Screen and a CRTHirsch [108]proposed a rotating screen 3D display design in 1958. Aviation Week reported a volumetric3D display system developed by ITT Labs in 1960 [109]

Documents

Three-dimensional display technologies