7
90 January/February 2007 Published by the IEEE Computer Society 0272-1716/07/$20.00 © 2007 IEEE 1 A glimpse at tomorrow’s invisibility using today’s digital forgery tools. Graphically Speaking Editor: Miguel Encarnação H ow many of us have had a dream of being invisi- ble? But is that really that far-fetched, impossible, or improbable? As a child I used to daydream of magi- cal powers that let me pursue exciting adventures in magical wonderlands. These great magical powers exhibited technology far beyond the current state of the art. I still clearly remember today those fantastic dreams. Indeed, dreaming that kind of fantasy let me feel good and relaxed, and they certainly colored my childhood. I personally think that daydreaming is also a kind of healing therapy of our day-to-day busy life to satisfy our curiosity. However, nowadays I also feel somewhat more conservative (and do not dream unfor- tunately anymore of Harry Potter worlds), partly because of my science background gained over the last 20 years or so of school education. School taught me the way to reasonably and rationally think without leaving much space for extravagance. That phenome- non is what Sir Arthur C. Clarke called the failure of the nerves in his book Profiles of the Future: An Inquiry into the Limits of the Possible. 1 A good example cited by Clarke are excerpts of scholarly articles written by experts claiming the then impossibility of heavier-than- air flying machines. Nowadays, we all take airplanes, so why do children still dream of magical worlds? Maybe, a simple answer is that children use and look at technology without fully understanding their under- lying mechanisms making it possible. It’s easier for them to naturally extend the scope of those technolo- gies according to their will. In this article, I will exam- ine the concept of invisibility in detail with a good flavor of computer graphics. But first, let me quickly review its historical background. The Invisible Man and the quest for cloaking devices The quest for invisibility was certainly spurred by the science fiction novel by H.G. Wells, The Invisible Man (1897), that was later turned into a movie directed by James Whale (1933). The myth and quest of the invisi- ble man had then entered households. Eventually, the seminal movie spawned a number of sequels such as the recent Hollow Man (2000). Although the original 1933 movie was poor in visual effects (remember the “invis- ible” Griffin shown with his face completely hidden by bandages, wearing goggles and a hat?), the Hollow Man movie was, in contrast, so rich in computer graphics effects, that it was nominated for an Academy Award— not for the best actor performance but rather for its visu- al effects. In Hollow Man, the invisibility power is reached by a team of scientists injecting some substance into Kevin Bacon’s body. Flesh would become quickly but progressively transparent. As H.G. Wells asked, what is the purpose of being invisible? After all, what motivates us that much? Beyond the obvious goals of peeping, robbing, or fight- ing, it’s not so clear. Ideally we would like to have the invisibility power reversible. But this turns out to be dif- ficult, at least in the movies, where film directors fol- lowing H.G. Wells emphasize the dangers of invisibility rather than its merits. Becoming irreversibly invisible turns out to yield an unstable mental state that reaches various paroxysms beyond the point of no return. This explains why most of the science fiction movies such as the Star Trek, Predator, or Harry Potter ones consider reversible invisibility as a magical gift or better as a tech- nological tool. In the Star Trek television series’ “Balance of Terror” (1966) episode, Romulans hide their space- ships simply by pushing a button that activates the invis- ible shield. That is, invisibility is obtained after activating an advanced stealth gadget-cloaking device, which causes someone to be extremely difficult to detect with conventional sensors. Can we legitimately ask Frank Nielsen Sony Computer Science Laboratories The Digital Chameleon Principle: Computing Invisibility by Rendering Transparency __________________________ Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.

Graphically Speaking - LIX

  • Upload
    others

  • View
    12

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Graphically Speaking - LIX

90 January/February 2007 Published by the IEEE Computer Society 0272-1716/07/$20.00 © 2007 IEEE

1 A glimpse attomorrow’sinvisibility usingtoday’s digitalforgery tools.

Graphically Speaking Editor: Miguel Encarnação

H ow many of us have had a dream of being invisi-ble? But is that really that far-fetched, impossible,

or improbable? As a child I used to daydream of magi-cal powers that let me pursue exciting adventures inmagical wonderlands. These great magical powersexhibited technology far beyond the current state ofthe art. I still clearly remember today those fantasticdreams. Indeed, dreaming that kind of fantasy let mefeel good and relaxed, and they certainly colored mychildhood. I personally think that daydreaming is alsoa kind of healing therapy of our day-to-day busy life tosatisfy our curiosity. However, nowadays I also feelsomewhat more conservative (and do not dream unfor-tunately anymore of Harry Potter worlds), partlybecause of my science background gained over the last20 years or so of school education. School taught methe way to reasonably and rationally think withoutleaving much space for extravagance. That phenome-non is what Sir Arthur C. Clarke called the failure of thenerves in his book Profiles of the Future: An Inquiry intothe Limits of the Possible.1 A good example cited byClarke are excerpts of scholarly articles written byexperts claiming the then impossibility of heavier-than-air flying machines. Nowadays, we all take airplanes,so why do children still dream of magical worlds?Maybe, a simple answer is that children use and lookat technology without fully understanding their under-

lying mechanisms making it possible. It’s easier forthem to naturally extend the scope of those technolo-gies according to their will. In this article, I will exam-ine the concept of invisibility in detail with a good flavorof computer graphics. But first, let me quickly reviewits historical background.

The Invisible Man and the quest forcloaking devices

The quest for invisibility was certainly spurred by thescience fiction novel by H.G. Wells, The Invisible Man(1897), that was later turned into a movie directed byJames Whale (1933). The myth and quest of the invisi-ble man had then entered households. Eventually, theseminal movie spawned a number of sequels such as therecent Hollow Man (2000). Although the original 1933movie was poor in visual effects (remember the “invis-ible” Griffin shown with his face completely hidden bybandages, wearing goggles and a hat?), the Hollow Manmovie was, in contrast, so rich in computer graphicseffects, that it was nominated for an Academy Award—not for the best actor performance but rather for its visu-al effects. In Hollow Man, the invisibility power isreached by a team of scientists injecting some substanceinto Kevin Bacon’s body. Flesh would become quicklybut progressively transparent.

As H.G. Wells asked, what is the purpose of beinginvisible? After all, what motivates us that much?Beyond the obvious goals of peeping, robbing, or fight-ing, it’s not so clear. Ideally we would like to have theinvisibility power reversible. But this turns out to be dif-ficult, at least in the movies, where film directors fol-lowing H.G. Wells emphasize the dangers of invisibilityrather than its merits. Becoming irreversibly invisibleturns out to yield an unstable mental state that reachesvarious paroxysms beyond the point of no return. Thisexplains why most of the science fiction movies such asthe Star Trek, Predator, or Harry Potter ones considerreversible invisibility as a magical gift or better as a tech-nological tool. In the Star Trek television series’ “Balanceof Terror” (1966) episode, Romulans hide their space-ships simply by pushing a button that activates the invis-ible shield. That is, invisibility is obtained afteractivating an advanced stealth gadget-cloaking device,which causes someone to be extremely difficult to detectwith conventional sensors. Can we legitimately ask

Frank NielsenSony ComputerScienceLaboratories

The Digital Chameleon Principle: Computing Invisibilityby Rendering Transparency __________________________

Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.

Page 2: Graphically Speaking - LIX

IEEE Computer Graphics and Applications 91

about the physical rationale of invisibility? A trivial butnaive way would be to consider changing the bodyrefractive index to that of the surrounding ambient air,ensuring that our body absorbs no light. We would thenbecome almost as air is—that is, invisible or scientifi-cally speaking, transparent (see Figure 1 for a fake pre-view created by image compositing). But is this thatsimple?

Invisibility: science facts or urbanlegends?

Several times in the 20th century, different sourcesreported that invisibility was allegedly successfullyobtained. Perhaps the most famous but arguable storyis the Philadelphia experiment. The Philadelphia exper-iment myth appeared in 1955 with the publication ofThe Case for UFO’s (Bantam Books) by Morris K. Jessup.After the publication of his book, Jessup received a let-ter from C.M. Allende that detailed a claimed secretexperiment carried out in 1943 by the US Navy inPhiladelphia. Allende said that the USS Eldridgeship was rendered invisible and transported fromPhiladelphia to Norfolk, Virginia, and back in a merefew minutes. This was achieved by applying the princi-ples of Einstein’s unified field theory. This is all the morepuzzling as Einstein indeed worked for the navy duringWorld War II. The experiment supposedly had some ter-rible aftereffects for all the ship crew members whobecame badly ill and developed schizophrenia, amongother ailments. Although Allende claimed in his corre-spondence to have witnessed this experiment fromanother ship, he could not be later identified, nor couldauthorities trace alleged reports of this experiment inPhiladelphia newspapers. The Philadelphia experimentis nowadays considered a hoax. But we cannot rule outfor sure the technological possibilities of Einstein’sspace–time theory.

Invisibility and camouflage in natureComing back to the scientific rationale of invisibility,

we can start by having a look at Mother Nature. We findin nature a beautiful source of inspiration for reachinginvisibility. The first example that comes to mind is jel-lyfish. Jellyfish are well-known sea creatures that aretranslucent and look transparent so that they are almostinvisible in seawater. But their functionalities are unfor-tunately limited. Let us look at other species. Considerflat fishes such as winter flounders that hide themselvesfrom predators by adapting their skin pigments accord-ing to the background environment. These fish canmimic the color of their background environment bycontrolling pigment cells, or chromatophores, locatedin the dermis and epidermis. Chromatophores are irreg-ularly shaped cells containing chemical pigments thatare also sensitive to temperature changes and stress con-dition. Chromatophores are also found in crustaceans,lizards, and amphibians. Color changes take place inchromatophores by hormonal control (a slow processdepending on the body’s emotional state) that yieldseither dispersion or gathering of granules within thecells. This is precisely that density change that lets thesecells control either the darkening or lightening effect of

their appearance. Camouflage is a must value to survivein a world populated by predators. Unfortunately, preda-tors also use camouflage tricks themselves to lure preyfor lazy but efficient foraging—think of a chameleon ora stonefish, for example. In the long run, chemical ormolecular switching might offer the best method for aphotochemical cloaking device. For now, they exhibitfar too slow a latency. Let us look now at the feasibilityof this transparency principle by rendering backgroundimageries, using off-the-shelf, low-latency cameras andprojectors commonly found in computer graphics andvision laboratories.

Plenoptic functions and light fieldsLet me start the principle of rendering transparency

by introducing the notion of a plenoptic function formodeling a scene’s visual appearance. The word plenop-tic stems from Latin roots plenus (meaning full or com-plete) and optics (meaning see). The plenoptic functionis a 7D function that records for every spatial position (X,Y, Z) the intensity of the light of the ray anchored at (X,Y, Z) for every orientation (�, �) at every time t (six para-meters). Because light such as visible light is in generalnonspectral, it’s best described by its spectrum decom-position into pure (ideal) monochromatic lights, thusadding the extra wavelength dimension � to the plenop-tic function. Such a decomposition and analysis of lightis done using a spectrogram. The visible spectrum is asubband of the electromagnetic field with wavelengthsranging in the 400 to 700 nanometers. In summary, the7D plenoptic function P(X, Y, Z, �, �,�, t) describes thefull environment light intensity contribution of themonochromatic lights of wavelength � at any location inspace at any time. For a given position at a given time,the plenoptic function is simply called a (panoramic)light probe. Light probes are nowadays often capturedfor photorealistic image-based lighting of 3D computergraphics. Figure 2 shows a synthetic rendering of a 3Dscene using a high dynamic range light probe.

2 A photorealistic computer graphics rendering obtained by lighting the3D scene geometry with a light probe (shown in the inset).

© P

aul D

ebev

ec, U

niv.

of S

outh

ern

Cal

iforn

ia

Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.

Page 3: Graphically Speaking - LIX

Graphically Speaking

92 January/February 2007

Researchers have extensively studied plenoptic mod-eling, which has evolved into a subfield of computergraphics called image-based rendering. One such keypioneer image-based rendering work is the light fieldproject carried out in 1996 by Levoy and Hanrahan (seehttp://graphics.stanford.edu/projects/lightfield/ formore information).2 Levoy and Hanrahan captured andrendered particular cases of plenoptic functions: 4Dlight fields. A 4D light field simplifies the plenoptic func-tion to practical cases. A light field is only described byfour parameters (s, t, u, v) that parameterizes 3D linesusing two pairs of 2D coordinates (s, t) (the focal plane)and (u, v) (the nodal plane) of two disjoint parallelplanes (see Figure 3). A light field L(s, t, u, v) considersstatic scenes and a given color channel (red, green, orblue). Light fields are also called radiance fields becausethey are related to the space distribution of light ener-gy. To capture a static light field using a pinhole camera,we can select a few (u, v) positions for the camera’snodal point and acquire the corresponding bundle of (s, t) rays (see Figure 3). To capture time-varyingdynamic light fields, we need a 2D camera array thatacquires synchronously these bundles of rays, temporalframe by temporal frame. Once we acquire a light field,we can generate new synthetic pinhole camera views onthe fly using pixel rebinning of source images.

Light field rendering is a brute-force approach in com-puter graphics; however, it yields spectacular photore-alistic results where traditional scene modeling hasfailed. Now that I have finished describing the maintechnical concepts, it’s time to see how to implement thelow-latency optical active camouflage following thechameleon principle.

The digital chameleon principleThe main difference with the approach I take and, for

example, a simple snapshot of a translucent jellyfish isthat I am going to shade real objects (that is, physicalobjects) transparent. These dynamically renderedobjects will appear almost invisible. For simplicity, con-sider a 3D box as depicted in Figure 4 with an observerstanding in front of the box. My goal is to deliver in theviewer’s exploring area a plenoptic field that matchesas closely as possible the plenoptic field that would existif the box were not there. The result is an exact replicaof the image appearing behind the box from any view-point in front of the box. In doing so, I can conceal insidethe chameleon box arbitrary objects (a small world) tohide from the outer world. Let us proceed step by step byconsidering several cases.

First, the simplest case—a fixed viewpoint (see Figure4a). Assume that a pinhole camera correctly approxi-mates the geometric vision of our observer. In that case,the set of rays emanating from the observer’s pinholeeye has a fixed geometry and can be acquired by lightsensors located on the other side of the chameleon box.If we further assume that the scene is located far awaywith respect to the viewer’s distance to the box, then wecan acquire with a good approximation this set of raysusing another similar pinhole camera located on thechameleon box’s back side. Evaluating the transparen-cy resolution obtained is a delicate matter as many fac-tors intervene in the final transparency rendering.However, we might easily guess (by extrapolation) theresult of the digital chameleon box experiment by refer-ring to a similar project: a team of researchers led byTachi and Inami of the University of Tokyo uses a retrore-

4D light field sensor

s

t

u

v

v

s

t

u

u s

t v

u s

t

v

us

tL(u,v,s,t)

Two-plane parameterization

v

3 A 4D lightfield reducesthe 7D plenop-tic function byconsidering astatic scenefrom its exteri-or for a givencolor channel.

Cou

rtes

y St

anfo

rd C

omp

uter

Gra

phi

cs L

abor

ator

y

Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.

Page 4: Graphically Speaking - LIX

IEEE Computer Graphics and Applications 93

flector material called X’tal vision to project backgroundimages onto the masked object that appears transpar-ent (see Figure 5).3 The method is similar in spirit to thatof Figure 4a. The retroreflector cloak consists of a coat-ing of small beads that reflects the light exactly in theopposite incoming direction without being too scatteredor absorbed. Retroreflector materials are already mas-sively used for traffic road signs, allowing drivers to readfaraway road signs with their vehicle lights. Suchretroreflector cloak systems, however, provide only agood illusion of invisibility for a viewer’s gaze direction,matching the surface normal of the object needing tobe concealed. To acquire the optical camouflage back-ground, the University of Tokyo researchers used a videocamera located on the back of the standing human orobject. This limited the freedom of motion of the quasi-invisible actor. The chameleon box experiment, howev-er, gives a reasonable glimpse of the box’s quality, andincites us to expand venues for an active optical cam-ouflage technique.

Next, consider a fixed dot on the front side of thechameleon box (see Figure 4b). The geometric positionof the dot and the observer’s viewpoint defines a geo-metric ray that a projector can render as the same colorof the ray passing entirely through the box. As the view-er moves freely around, the ray’s geometry changes, butanother closely matching projector ray can still renderit with the correct color.

Finally, let us consider a general case (see Figure 4c).Namely, viewing the front side of the chameleon boxfrom an arbitrary viewpoint. It’s easy to extend the twoprevious cases to this general case. By using a cameraarray on the backside of the chameleon box, you canacquire the 4D light field incident onto the wall andreproduce it on the box’s front side using a projectorarray. This process guarantees that the viewer willreceive the same set of rays as if the box were not there.That is, the box is rendered transparent (shaded trans-parent), and I can compute the invisibility illusion bydigitally remapping 3D light rays acquired on the back-side camera panel to rays projected on the front sideprojector panel. The chameleon box in its simplest formis a naive, brute-force, digital-ray-remapping engine.Cameras play the role of light detectors while projec-tors play the role of light emitters. In other fancy words,the two opposite box sides are light portals of theplenoptic function. Of course, in practice, we wouldrather like a free-form cloak (the invisible suit) that per-fectly fits the body or object to conceal: a cloaking deviceas shown in Predator or Harry Potter movies.

How far are we from building a real prototype of sucha digital chameleon box? Actually, not as far as youmight think. Indeed, by noticing that the light field cap-tured by the 2D camera array on one side of the box canalso be entirely computed by simulation using comput-er graphics, you can remove at first the camera array(and reduce messy synchronization engineeringdetails). In that case, observers see a 3D virtual worldprojected in front of the box. This is the basic principleof some families of 3D displays that work by reproduc-ing virtual synthetic scene rays. Several industrial sys-tems are currently built and are shown all over the

(a)

(b)

Camera Television

(c)

Camera array Projector

Camera array Projector array

Chameleon box

Chameleon box

Chameleon box

4 The digital chameleon principle relies on a camera array to acquire alight field on one side of the box and uses a projector array to reproducethat light field on the other side, thus visually concealing the box to anobserver located in front of the ray display. (a) One camera and one screengive a correct image from a fixed viewpoint. (b) Multiple cameras and oneprojector give a single dot view, correct from any direction. (c) Multiplecameras and multiple projectors give a correct image from anywhere.

5 The invisible cloak experiment carried out by researchers at the Tachilaboratory.

Cou

rtes

y Ta

chi L

abor

ator

y, U

niv.

of T

okyo

Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.

Page 5: Graphically Speaking - LIX

world. See, for example, the Holografika display shownat the 2006 Siggraph Emerging Technologies exhibit(see http://www.siggraph.org/s2006/main.php?fcon-ference&p=etech&s=holographic and http://www.holografika.com/).

Thus to build a digital chameleon box, you wouldneed to combine such a 3D ray display with a 2D arrayof cameras. This is, technologically speaking, feasiblesince nowadays camera arrays are mainstream in com-puter graphics and vision laboratories. This tight cou-pling of camera arrays and projector arrays will likelysoon strengthen as 3D cinematography and next-gen-eration videophones challenge these technologies.

Further, to reduce the complexity of using 2D cameraarrays and/or 2D projector arrays, we can consider usingmonolithic, high-resolution acquisition and renderingdevices—that is, a single, high-resolution camera/pro-jector. For a century now, the integral photography fieldhas investigated this architecture. Instead of using cam-era/projector block arrays, integral photography uses asheet of tiny hemispherical lenslets, usually arranged ina honeycomb pattern, manufactured using well-mas-tered photolithographic processes. Each tiny lens cap-tures the light rays coming into the center of the lens witha given incidence angle. A high-resolution area sensorcaptures the sheet, yielding a sheer amount of ray infor-mation. Ideally, the smaller the lenslet the better the res-olution. There are, however, some physical constraints,and you will typically face the geometric optics diffrac-tion problems encountered at small scales.

Let us envision a bit more of the technology of theoptically invisible man, beyond the digital chameleonbox. In practice, we would prefer wearing a suitequipped with interspaced, dense, tiny button-size cam-era/projectors that dynamically autocalibrate them-selves on the fly rather than using the massiveequipment mentioned previously. This is just a matterof time as miniaturization of these key camera/projec-tor light-sensing and emitting devices progress drasti-cally, and novel manufacturing techniques—such assilicon on glass, which hides the large-scale-integrationcircuitry in translucent material—become possible.

To wrap-up, perfect invisibility seems quite far awayfrom today’s practicalities. Yet, computational invisibil-ity, an emerging topic of computational photography,has numerous applications before reaching the con-sumer level. Indeed, how useful would it be to have thefloor of an airplane cockpit rendered transparent! Nomore visual dead zones. Also think of how easy it wouldbe to park buses or simply your car with the back ren-dered transparent, replacing our current loud warningbuzzers used for safety.

Invisibility and shadingInvisibility is the task of masking physical objects so

that they are no longer visible. In contrast, shading isthe task of making up visual appearances of objects.Visual properties of object appearances includereflectance characteristics and numerous phenomenasuch as subscattering or diffusion effects. The appear-ance of an object is simply expressed as a function of theillumination (given by the surrounding environment),

surface reflectance properties (depending on materi-als), and viewer location. Looking back at this column’stitle, I chose to write “The Digital Chameleon Principle:Computing Invisibility by Rendering Transparency,” butI could also have replaced the subtitle using more tech-nical jargon like “Computing Invisibility by ShadingTransparency,” since invisibility can also be interpretedas the process of creating a transparent shader. Shadingis nowadays an essential technology for achieving pho-torealism in computer graphics. Let us address a com-puter graphics–related project called the shader lampsthat also deals with physical objects.4

The main idea of a shader lamp is to replace our usuallight bulbs by tiny computer-controlled calibrated pro-jectors that emit per-pixel controlled light color onobjects to produce realistic object appearances, asdepicted in Figure 6. This method first molds physicalobjects from white clay or covers them with a materialsheet that has similar diffuse matte finish characteris-tics. It then interactively shades the objects by renderingcomputer graphics from projector viewpoints and pro-jects these synthetic images onto the diffuse matteobjects that play the role of geometric proxies. Onceagain, let me emphasize that these special visual effectstake place in the real world and not on the computerscreen as usual, thus making it easy to design tangibleuser interfaces for interactive shading. The number ofshader lamps (projectors) depends on the intrinsic com-plexity of the geometric object since the system needsto account for potential self-shadowing of objects andmust fully cover the scene surfaces. Shader lamps notonly visually rewrite the object’s texture (the diffusecomponent) but also reproduces any bidirectionalreflectance appearance (whether view-dependent,using a bidirectional reflectance distribution function,BRDF, or not). Moreover, not only do shader lampsenable us to project virtual reflectance on registeredreal-world geometry, they also animate physical staticobjects as well by projecting dynamic image-based illu-mination movies. The visual appearances of projector-shaded objects are thus lifted from the real to themixed-reality world of computer-controlled projectors.The shader lamp approach is currently limited to phys-ical, diffuse material objects, and requires tracking thesingle viewer’s location if view-dependent effects suchas specular highlights are required.

Modalities of invisibilityEven if the active optical camouflage looks good, mean-

ing human eyes can’t detect it, an infrared or thermalcamera might easily show someone hiding inside thechameleon box. That is, we have to take into account themodality of invisibility. Indeed, to give yet another exam-ple, consider a silent, black-painted airplane travelinghigh enough at night in a cloudless sky: no one on theground would be able to notice it. Yet, this would be triv-ial for radars to detect that airplane using the reflectionsof emitted radar signals. Aircrafts thus become vulnera-ble. To overcome that issue and hence bypass radar tech-nologies, the American SR-71 Blackbird airplane is usinga special radar-absorbing paint to trap those signals andlimit as much as possible their reflections, making them

Graphically Speaking

94 January/February 2007

Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.

Page 6: Graphically Speaking - LIX

radar invisible. However, we could possibly come up witha new sensitive detection scheme using another modali-ty. Modality emphasizes that invisibility is not ultimatebut rather limited to various sensing methods.

Stealth and antistealth technologiesFor an observer, we might reverse the challenge of

invisibility: can we see invisible things? The questionmight sound absurd at first since we have concentratedso far on making objects optically invisible. But remem-bering that invisibility and visibility is simply a matter ofimagery techniques, we might create novel modalitiesfor seeing so-far invisible objects. Let us take a typicalexample encountered in daily life at the airport securitycheck screening. Guards try to detect efficiently invisi-ble objects hidden in clothes and bags. The usual bagscreening procedure is the x-ray conveyor belt. Yet,nowadays with the high level of terrorism alerts, moreand more substances and objects need to be detected forpreventing their access on board. For that crucial pur-pose, there have been novel imaging modalities that havebeen quickly commercialized to automatically checkwhether someone is wearing undesirable substances,such as suspicious liquids or traces of explosives. The T-ray camera is one such amazing technique. It’s sensitiveto terahertz frequencies and literally lets us see throughclothes or other packaging (see Figure 7). T-rays alsoallow cameras located in space to see through clouds. T-ray cameras are certainly one of the first great techno-logical breakthroughs of the 21st century.

My approach to invisibility based on the plenoptic-modeling paradigm is scalable and should yield a cer-tain amount of invisibility. It’s clearly not a pureapproach such as physics-based ones, but rather anengineering illusion. One drawback is that the digitalchameleon principle does not prevent the box (or ren-dered objects) to have reflected or scattered light dueto its physical properties. Thus, ultimately we need adirect, physics-based, self-contained structure methodthat does not violate the basic laws of physics.

Microscopic-level invisibilityRecently, researchers in physics pointed out a new

family of materials, metamaterials (also called super-lens), exhibiting the property of negative refractiveindices. By using metamaterial, light can theoreticallybend around objects. This property yields a potentiallypure physics-based invisibility technique. Actually, his-

torically, warping light was the first proposed approachto invisibility, reportedly studied in the 1930s using anelectromagnetic field. How does this metamaterialwork? Alu and Engheta of the University of Pennsylvaniause plasmon waves to reduce the light scattering effectof incoming light, thus making the object appear less vis-ible.5 Indeed, we see objects because light bounces offthem. The plasmonic cover prevents light scattering byresonating at the same frequency of the incoming light.That is, plasmons are electrons of a metallic surface mov-ing in rhythm defining a shell. That shell reduces lightscattering providing that the light frequency is close tothe resonant frequency of the plasmons. Gold and silverare two suitable materials for building plasmonic visi-ble-light shields. Although the idea of a plasmonic shieldis theoretically sound, its current practicalities are limit-ed to microscopic size objects under controlled lighting.To be practical, the method needs to take into accountreal sunlight (rich blend of wavelengths) and scaleobjects to a reasonable human size. That is, this does notyet offer a self-contained magic cloak. But with more

IEEE Computer Graphics and Applications 95

6 Virtual reflectance of a white diffuse physical vase. Shader lamps provide a virtual, bidirectional reflectance distribution functionappearance to diffuse material, physical objects.

Cou

rtes

y Ra

mes

h Ra

skar

, Mits

ubis

hiEl

ectr

ic R

esea

rch

Labo

rato

ry

7 Example of a T-ray image.

Cou

rtes

y Q

inet

iQ

Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.

Page 7: Graphically Speaking - LIX

research, plasmonic screening could theoretically hidelarge objects from telescopes whose imageries are basedon long wavelengths instead of short visible waves.

Conclusion Today, invisibility does not seem out of reach to the

realm of science. At least, it’s more probable than it wasa century ago. There are yet many technology gaps tobridge to reach true invisibility. I hope that I’ve con-vinced the reader that transparency could be in princi-ple computed and implemented by a special shaderusing both camera and projector arrays, such as withthe digital chameleon box. This basic idea has alreadyyielded various patent applications, and my guess is thatthis active optical camouflage technique will soon besuccessfully demonstrated.

I should emphasize that we are seeing with our brain.Our eyes are only the raw photo sensors that deliverbasic electrochemical signals to our brain, which thenprocesses these low-level cues into higher cognitionnotions. Thus, it might be possible to think of invisibili-ty at the human brain level. What does being invisiblemean then? In cognitive science, this invisibility phe-nomenon is called cognitive blindness. A typical exam-ple is the case of somebody leaving a meeting roomwithout being noticed by an audience that is deeplyengrossed in a conversation. Such a cognitive invisibil-ity could be individually selective compared with real-world, physics-based absolute invisibility. Yet another

potential investigation topic linking physics-based andhuman-eye invisibility is vibration. Indeed, as stated pre-viously, we see using the persistence of vision property.That is, light is first accumulated in retinal photosensors(cones and rods) before propagating the impulses intoelectrochemical reactions. Thus, we average light, andthis causes various scene aliasing effects such as the carwheels that look like they are spinning in reverse.Looking at an air fan, for example, we literally seethrough it by virtue of the persistence of vision. Thusvibration and light averaging might also be a futuredirection for finding other invisibility tricks.

The approaches to invisibility presented here arewithin the probable reaches of today’s science. Yet thedoor is widely open to our imagination, and we can findother ingenious schemes. As the pace of technologykeeps ineluctably increasing, recent achievements andprogress suggest that the active camouflage paradigmof the digital chameleon box is not just a utopian dream,but is rather becoming closer to reality. For now, let meconclude by citing Alfred A. Montapert: “Nature’s lawsare the invisible government of the Earth,” and theseare the hard ones to spot! ■

AcknowledgmentsI thank Paul Debevec (University of Southern

California), Marc Levoy (Stanford University), andRamesh Raskar (Mitsubishi Electric ResearchLaboratory) for their kind image contributions. I amindebted to Shigeru Owada (Sony Computer ScienceLaboratories) for his valuable advice in drawing Figure4. This article was reformatted, edited, and inspired bythe original article appearing in the December 2006edition of Janus magazine (see http://www.janusonline.net/, “The Digital Chameleon Principle:Computing Invisibility by Rendering Transparency”)that runs a full-length issue on the many fascinatingfacets of invisibility.

References1. A.C. Clarke, Profiles of the Future: An Inquiry into the Lim-

its of the Possible, Indigo, 2000.2. M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc.

Siggraph, ACM Press, 1996, pp. 31-42.3. S. Tachi, “Telexistence and Retro-Reflective Projection

Technology (RPT)”, Proc. 5th Virtual Reality Int’l Conf. (VRIC), Istia Innovation, 2003, pp. 69/1-69/9;http://projects.star.t.u-tokyo.ac.jp/projects/MEDIA/xv/VRIC2003.pdf.

4. R. Raskar et al., “Shader Lamps: Animating Real Objectswith Image-Based Illumination,” Proc. 12th EurographicsWorkshop Rendering Techniques, Springer, 2001, pp. 89-102; http://www.cs.unc.edu/~raskar/Shaderlamps/.

5. A. Alu and N. Engheta, “Achieving Transparency with Plas-monic Coatings,” Physical Rev. E, vol. 72, no. 1, 2005.

Contact author Frank Nielsen at [email protected].

Contact department editor Miguel Encarnação [email protected].

Graphically Speaking

96 January/February 2007

www.computer.org/internet/

Stay on TrackIEEE Internet Computing reports emerging tools,technologies, and applications implemented through theInternet to support a worldwide computing environment.

Authorized licensed use limited to: Ecole Polytechnique. Downloaded on October 1, 2009 at 05:25 from IEEE Xplore. Restrictions apply.