Exploring Programmable Light Spaces€¦ · bronze mirror (Figure 2) [1]. The non-uniform distribution of the mirror thickness by the molded image behind the mirror creates small

Exploring Programmable Light Spaces using Actively Deformable Mirrors

Abstract In this study, we propose a new approach for displaying images and controlling light spaces using actively deformable mirrors. This approach enables programming the mirror-reflected light from sunlight or other parallel/point light sources to create arbitrary light spaces in various scenarios.

Author Keywords Video displays; programmable light; ubiquitous displays; programmable space; programmable architecture; architectural space; interaction design

ACM Classification Keywords H.5.2. [Information interfaces and presentation]: User Interfaces: Interaction styles, Prototyping

General Terms Design; Human Factors

Introduction Visual displays are widely used in indoor/outdoor scenarios because of the recent developments in display technologies with video projectors [9], handheld laser projectors [17], and wearable displays [5]. At present, these ubiquitous display surfaces surround our lives and enrich our real-world experiences. Outdoor and/or large-scale video projections create persuasive

Copyright is held by the author/owner(s).

CHI 2013 Extended Abstracts, April 27–May 2, 2013, Paris, France. ACM 978-1-4503-1952-2/13/04.

Munehiko Sato 1 [email protected] Mehdy Chaillou 2, 3 [email protected] Tomohiro Tanikawa 2 [email protected] Michitaka Hirose 2 [email protected]

1 Department of Advanced Interdisciplinary Studies, Graduate School of Engineering, The University of Tokyo 7-3-1 Hongo, Bunkyo, Tokyo, Japan 113-8656 2 Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo 7-3-1 Hongo, Bunkyo, Tokyo, Japan 113-8656 3 Department of Applied Physics, Electrical and Information Engineering, Graduate School of Engineering, ENS Cachan 94230, France

Work-in-Progress: Ubiquitous Computing CHI 2013: Changing Perspectives, Paris, France

1659

experiences, such as projection mapping [15]. Some new buildings have built-in full-color façade displays that can be dynamically controlled to display videos and other visual media [2][12].

This study reports initial results of a new scalable visual display technique used in a Magic Mirror Projector for both indoor and outdoor applications, which is inspired by the traditional Chinese/Japanese Magic Mirror [1]. The system uses a light reflecting surface (e.g., a glass mirror, metal mirror, glass window, or glass wall) and a parallel light source (e.g., sunlight) or a point light source (e.g., an LED). Fractional undulations of the mirror surface slightly change the reflection direction and form a pattern in the projected light [3]. These undulations can be produced and actively controlled by

installing linear actuators behind the mirror. This technique enables to project visual images wherever the reflected light is being projected.

Displaying Moving Images using an Actively Deformable Mirror Principle Figure 1 illustrates the principle of a Magic Mirror Projector: a dynamic light projector using an actively deformable mirror. Parallel (or point) light sources project light onto a mirror, and the reflected rays leave the surface of the mirror at the same angle of reflection as that of incidence. If the mirror surface is flat and smooth, the projected light is uniform. However, if the mirror surface has undulations, the projected light has brighter or darker areas as a consequence. Therefore, by controlling the undulations of the mirror surface, we can display arbitrary patterns in the projected light. This phenomenon works best when a first-surface mirror, e.g., a polished metallic surface, is used. Undulations of such mirrors are very small and unnoticeable when we directly look at the mirror surface. Various materials can be used as mirrors, such as metallic first-surface mirrors, mirror-finished glass walls/windows, liquid mirrors, interfaces of two liquids with different refractive indices, and normal glass/plastic mirrors. Appropriate actuators for the applications and sizes must be used, such as magnetic solenoids, electromagnets, piezo-actuators, voice coils, and air/oil cylinders.

The idea of interfering with the light reflection direction by modifying the mirror surface is not new; in fact, it has thousands of years of history. It is known as the Chinese/Japanese/Oriental Magic Mirror (called Makyoh in Japanese) that projects a drawing molded behind a

Figure 1. Principle of image projection using a deformable mirror. 1: light source, 2; mirror, 3: array of linear actuators, 4: projection surface, 5: incoming light (parallel light or point light), 6: reflected light, 7: projected light. A: A flat mirror surface with actuators switched off, B: Undulations of mirror surface with actuators switched on.


1660

bronze mirror (Figure 2) [1]. The non-uniform distribution of the mirror thickness by the molded image behind the mirror creates small differences between the normal force and friction force created when the mirror surface is being polished. This irregular distribution of the frictional force results in a pattern on the mirror surface with minute undulations after the polishing is completed. This pattern slightly changes the direction of the reflected light and projects the drawing in the projected light. Recent research indicates that these original Magic Mirror images are Laplasian images by pre-focal ray concentration [3], but details are beyond the scope of this work-in-progress paper.

Key characteristics

§ Ubiquitous display: The Magic Mirror Projector can project light images at arbitrary locations in the environment by controlling the pan and tilt of the mirror. It can be used indoors with an artificial light source or outdoors with direct sunlight. One of the major advantages of the Magic Mirror Projector is that it can project an image under direct sunlight. While general high-power video projectors do not have enough luminance, the Magic Mirror Projector can generate a sufficient difference in contrast between the mirror-projected image and environmental direct sunlight.

§ Scalability: The size of the mirror can range from ~1 mm to ~10 m. The number, density, arrangement of actuators, and the type of actuators may vary as well. Therefore, this display is scalable both in size and resolution.

§ Power efficiency: This is a space- and power- efficient display. Even with recent power-efficient LED projectors, light bulbs consume most of the

power. As the Magic Mirror Projector uses a light source that already exists in an environment, the power consumption can be minimal, even in an outdoor setup. The projector unit can be placed far from the surface where the images are projected. No hardware is required for the projection surface. Furthermore, a convex mirror can project an image larger than the physical size of the mirror.

§ Non-matrix-based image representation: While all actuators are separately controlled, they are connected to a single mirror. When some of the actuators are being driven, the resulting projected image is a smooth “anti-aliasing” output. This is a display that can project smoother images with a limited number of “pixels” (actuators) than traditional display devices.

Implementation We implemented a proof-of-concept Magic Mirror Projector with a 210 mm × 297 mm stainless-steel mirror (1 mm in thickness) and seven pull solenoids (Shinmei Electric CO LTD., SS-103-501, Figure 3, right). Each solenoid plunger is mounted behind the stainless-steel mirror with 35 mm spacing in the arrangement, as shown in Figure 3. A PC and Arduino microcontroller boards control the solenoids with 8 bit pulse width modulation (PWM) control. The computer software can drive the individual solenoid’s PWM value to output still images and animations. An example output of an animation is shown in Figure 4.

Other Systems Using Deformable Mirrors There are other applications of actively deformable mirrors: Active Optics and Adaptive Optics.

Active Optics is a technology to correct deformations of a large mirror in a telescope [4]. The size of mirrors in

Figure 2. Ancient Japanese Magic Mirror: A line drawing of a figure appears in the reflected parallel light (top), the mirror surface appears to be completely flat to the naked eye (bottom).


1661

reflecting astrometric telescopes can be more than 10 m and their shapes are influenced by mechanical stress, temperature, and wind. An array of actuators installed behind the large mirror corrects the shape of the mirror.

Adaptive Optics is a technology to improve the optical system performance (e.g., laser communication system [11]) or imaging human retina (Adaptive Optics Retinal Imaging [7]) by aligning the wavefronts of incoming light using a microelectromechanical system (MEMS) deformable mirror or liquid crystal array. Wavefronts of light may be interfered by atmospheric turbulence and other factors. The Adaptive Optics technique is used to correct this error in the wavefront for better communication or imaging.

Although deforming the mirror surface has been used in several applications, as described above, the idea and technique of dynamically controlling the mirror shape to display animations for visual representations have not yet been applied yet, to the best knowledge of authors.

Related Work Several studies have attempted to control light spaces both in and out of the HCI field.

Transparency controllable liquid crystal films [8] are widely used in office partitions, airplane windows, and architecture. The Leafy Light Display is a research project and artwork by Yasuhiro Suzuki and Atsushi Hiyama that creates artificial sunbeams streaming through tree leaves with hundreds of three-layered leaf-shaped liquid crystal films [13][14]. Rekimoto proposed a programmable physical window and wall, Squama, using a grid of the same small liquid crystal films [10]. He proposed several applications such as real-world pixelization, programmable shadows, and ambient displays. An artist group made “light pixel projectors,” Heliomatrix [6], using a set of mirrors equipped with computer-controlled pan-tilt platforms.

Application Spaces In this section, we discuss a variety of possible applications for the Magic Mirror Projector from large buildings to palm tops.

Figure 3. Proof-of-Concept implementation of a Magic Mirror Projector displaying an image in the reflected parallel light.

Figure 4. Animation projected using our Magic Mirror Projector prototype. (From top to bottom: clockwise motion displayed. See an auxiliary video figure.)


1662

Programmable Light Space with Architecture and Urban Space A very large-scale Magic Mirror Projector can be installed in skyscrapers, or even in a city. At present, external walls/windows of numerous buildings are made of glass mirrors. These glass mirrors reflect sunlight and project the reflected light on the ground, street, and people. The undulations of the glass wall/window segments can be controlled by installing actuators (e.g., voice coil transducers) behind the glass mirrors. It can work as an urban-sized ambient display, for instance, to warn people walking at a specific location, or to direct a huge traffic of pedestrians by projecting gentle visual cues that induce a sense of self-movement (Vection [16]) as in Figure 5, left.

Indoor/Outdoor Staging Representations Using the Magic Mirror Projector, various artistic and entertainment installations can be made both indoors and outdoors. It is easy to add a video projection feature to an LED spotlight by reflecting a light using a convex mirror (Figure 5, middle).

On-the-go Portable Projector Information displays, particularly video projectors that are viewable under direct sunlight (~100,000 lux) are still under development. The Magic Mirror Projector can serve as a portable video projector for augmented reality applications such as route guidance and real-world annotations. In this setup, a small mirror and an array of miniature linear piezo-actuators can be used. A larger picture can be projected efficiently using a convex mirror (Figure 5, right).

Conclusion and Future Work In this study, we have presented the concept of creating programmable light spaces using a deformable mirror, the Magic Mirror Projector. We have proposed a new approach to project arbitrary still images and animations at various locations using a deformable mirror. We have described the application spaces using this method. Although our current prototype can display only a simple animation with a small number of actuators, it can straightforwardly to enhance the expressiveness of the projector by increasing the number and density of the actuators, which we are working on. However, there are some points to be

Figure 5. Example application spaces: Programmable architecture and urban space (left), indoor/outdoor installations (middle), and an on-the-go portable projector (right).


1663

improved, such as joints between the mirror and solenoids creating circular artifacts in the projected image. These artifacts have occurred at the edge of the joints (20mm in diameter, 5mm in thickness), where the stiffness of the mirror changes discontinuously. In the next implementation, we will investigate the use of conical joints, very thin joints, integrated joints, and other shapes with finite element analysis for an improved result. A more precise and efficient control algorithm by Zernike polynomials expansion is under implementation, and multicolor support is another area of future work. We believe that it will be a versatile

display technology that will make various exciting applications possible.

Acknowledgements This research project is partially supported by the publicly offered project ”Mixed Realty Digital Museum” of the Ministry of Education, Culture, Sports, Science and Technology (MEXT) of Japan. The authors would like to thank Atsushi Hiyama for reading over a draft of this paper.

References [1] Ayrton, W.E. and Perry, J. 1878. The Magic Mirror of Japan. Part I. In Proc. of Royal Society of London, Vol. 28, 127–148.

[2] Ars Electronica Center. 2009. Ars electronica center. http://www.aec.at/center_building_en.php

[3] Berry, M. V. 2006. Oriental magic mirrors and the Laplacian image. European Journal of Physics 27, 1, 109–118.

[4] Hardy, J. W. 1978. Active optics: a new technology for the control of light. In Proc. of the IEEE, 66(6), 651-697.

[5] Harrison, C., Benko, B., and Wilson, A.D. 2011. OmniTouch: wearable multitouch interaction everywhere. In Proc. UIST '11. ACM, 441-450.

[6] Heliomatrix. 2009. http://www.heliomatrix.org/

[7] Merino, D., et al. 2011. Observation of cone and rod photoreceptors in normal subjects and patients using a new generation adaptive optics scanning laser ophthalmoscope. Biomedical optics express, 2(8), 2189-2201. [8] NSG UMU Products. www.umupro.com

[9] Raskar, R., et al. 1998. The office of the future: a unified approach to image-based modeling and spatially

immersive displays. In Proc. SIGGRAPH '98. ACM, 179-188.

[10] Rekimoto, J. 2012. Squama: modular visibility control of walls and windows for programmable physical architectures. In Proc. AVI '12, ACM, 168-171.

[11] Roorda, A., et al. 2002. Adaptive optics scanning laser ophthalmoscopy. Opt. Express, 10(9), 405-412.

[12] Sato, M., et al. 2011. Large-scale displays for public spaces. ACM Comput. Entertain. 9, 1, Article 4.

[13] Suzuki, Y., and Hiyama, A. 2008. Projection and staging equipment. Japanese PAT No. 2008-5103682, Filed Aug. 31st, 2006, Issued March. 13th, 2008.

[14] Suzuki, Y., Hiyama, and A., Hirose, M., 2007, Display of Light Filtering down through the Trees, Trans. of VRSJ, 12, 3, 397-400. (In Japanese)

[15] Valbuena, P. 2007. Augmented sculpture series. http://www.pablovalbuena.com.

[16] William, H., Warren, Jr., et al. 2001. Optic flow is used to control human walking, Nature Neuroscience 4, 213–216.

[17] Willis, K.D.D., Poupyrev, I., and Shiratori, T. 2011. Motionbeam: a metaphor for character interaction with handheld projectors. In Proc. CHI '11. ACM, 1031-1040.


1664

Documents

Exploring Programmable Light Spaces€¦ · bronze mirror (Figure 2) [1]. The non-uniform distribution of the mirror thickness by the molded image behind the mirror creates small