Taiwan Imaginary: Puppet Show in Hand, the Creative Apparatus

Using Multimedia Augmented Reality

Su-hui LIN Assistant Professor of Department Graphic Arts Communication,

National Taiwan Normal University Taiwan

linhuinne@gmail.com

Abstract – An "augmented reality" multimedia project is realized by Su-hui LIN, investigator coming from Taiwan, Docteur en Histoire de l’art de l'Université Paris-Sorbonne(Paris IV). The goal is to transform the traditional hand puppet show of Taiwan into a new stage. In first phase, contemporary Taiwanese artists will be invited to collaborate with this project. Enabling to present the historical story through out the VR environment, some 3D hand puppet models will be created through VR4MAX software. The audiences are allowed to interact with the digital content by waving their hand or moving their body. 3D, AR (augmented reality) and VR technology are utilized in this project, the investigator hopes to bring up audience’s attention and experiencing traditional folk art in a new way. Finally, this project will be presented by projecting the content onto the inner or outer wall of one Taiwan historical building. Under the augmented reality system, the traditional folk art of Taiwan will be illustrated in a more interesting way. This kind of demonstration will give Taiwanese audience some new thought, not only skinning in the dream of being "the computer kingdom" but also cherishing the root of the culture of Taiwan.

Keywords: Taiwan image, multimedia, augmented reality, hand puppet show.

I. INTRODUCTION With European countries' preserving and promoting their

heritage nowadays, Su-hui LIN, the reporter of this article, starts to ponder the heritage of her own country: Why do youngster in Taiwan feel so indifferent to “Puppet Show in Hand”, which indoctrinate virtue and was ever well-known in the past?

Coming from the background of art and being an educator of Department Graphic Arts Communication, the reporter here thought herself has the responsibility to resuscitate her heritage when lots of new technology being updated nowadays. This doesn't mean looking back ancient culture rigidly, rather, but means making youngsters face tradition culture, feel its value with “Augmented Reality”. This report introduces research background and the significance of problems first, and after document discussion, decide to present "Puppet Show in Hand" by means of " Augmented Reality". The reporter here not only exposes the manipulating principle of “transparent screen”, but offers concrete proposal of how to put it into practice.

A. Research background and motivation When in France, reporter frequently saw French workers

trying to preserve their heritage uninterruptedly. They even transfer the story of buildings into digital contents and project the image to the outwall of them, which lets simulation and reality reflect each other. That kind of shocking effect impresses tourist a lot1. She usually meditates then: If there are any possibilities to set up a similar apparatus for the heritage of her country?

Last year, Department for Education held a poll of "Taiwan Impression", it is "Univers in Hand-Puppet Show" that ranked first among all the choices2. It had ever been the most well-known and popular recreative activity in Taiwan when television first imported to here during 1960s" However, as time went by, it almost sequestered from Taiwanese's daily life, and significant part of young students now even don't know what "Puppet Show in Hand" is. The kind of demotic recreation has functions of Virtue being able to purge the minds, has the value of passing down cultures, preserving and promoting document reservation. It is a pity that "Puppet Show in Hand" has fallen to such kind of situation. Therefore, reporter proposed "Taiwan Impression: Creative apparatus research program of Augmented Reality by "Puppet Show in Hand", aiming at designing the mechanism of "multimedia Augmented Reality" to present this heritage of Taiwan, and letting Taiwan people, immerged in the dream of "Computer Kingdom", be able to "remember where one's happiness comes from" and "envisage the future".

B. The purpose and importance Nowadays Taiwan traditional "Puppet Show in Hand" has

fallen the situation of being not able to interest young students and arouse their will to participate in it. Facing this transition, if nobody makes an efficient move, perhaps Taiwan will lose the valuable cultural property. Although there are some report dealing with the research of traditional Taiwan "Puppet Show in Hand", the quality and quantity is still not adequate. Endangered cultural properties are needed to be particularly

1http://www.nytimes.com/2007/04/02/theater/02eyel.html, 2007/12/25. 2http://info.gio.gov.tw/ct.asp?xItem=28281&ctNode=3802&mp=1。2007/12/1.

2008 International Conference on Computer Science and Software Engineering

978-0-7695-3336-0/08 $25.00 © 2008 IEEE

DOI 10.1109/CSSE.2008.558

349

2008 International Conference on Computer Science and Software Engineering

978-0-7695-3336-0/08 $25.00 © 2008 IEEE

DOI 10.1109/CSSE.2008.558

349

respected and preserved, and people in the field of education ought to bring it into curriculum and promote it. In this way, not only students' artistic appreciation can be enhanced, but also foreign countries can see the traditional folk art in Taiwan and the visibility of this country can be improved. With new technique, such as 3D, VR, and "AR", the state of "half real, half fictitious" would reach the effect that couldn't be observed on stage before. Adopting this technique to the interpretation of traditional "Puppet Show in Hand", the audience can observe its characteristics-such as "soaring to sky, hiding into ground" and "stand overhead", more efficiently and make it show more vitality. Hence, it is necessary and exigent to put this research into practice.

II. LITERATURE ON THE SUBJECT After confirming the direction of research, the literature

search focuses on items about this project such as artistic multimedia, Augmented Reality and "Puppet Show in Hand" for making further discussions to decide research methods and executive procedure.

A. Contemporary multimedia art: Since 20th century, information transmission and

knowledge collection being updated unceasingly with the development of technology, new media based on digital technology have been the indispensable factor in modern life. Digital technology has certainly become important material for the artistic presentation nowadays. Multimedia digital elements are adopted unprecedentedly, and ready-mades, such as communicating net, magnetic waves, artificial satellites, radio and television, used for disseminating information, has become artists' cynosure and being mythologically visualized.

When writing the essay with the topic of "Illustration et présentation des machineries intellectuelles à partir de Marcel Duchamp" in France, the reporter here particularly noticed Marcel Duchamp's (1887~1968, French artist) whole life important concept "inframince" closely related to the formation of multimedia art.

During 1960's, influenced by the idea of Duchamp, Nam June Paik, Shigeko Kubota used electronic media technology as new method of artistic presentation, and their work predominated at once. Besides, artists realized their work with digital media technology. For example, Jenny Holzer holding Duchamp's spirit transferred cultural documents to digital images and sounds, and projected them onto streets and building, presenting them in people's living environment and interacting with pedestrians3. Her manner told us that the best way to make people accept traditional hairloom is interpreting tradition vocabulary with contemporary manners. Her manner of integrating simulation images and real scenes is actually Augmented Reality.

B. Augmented Reality ：

3 LIN, Su-hui (2003)：Illustrations et présentations des machineries

intellectuelles à partir de Marcel Duchamp), Paris：Thèse de Paris IV.

The system of Augmented System could be traced back to 1960s when Ivan Sutherland, precursor of computer drafting, developed first set of computer-drafting system, which bring the field of computer drafting real chance of developing. In early 1990s, scientists from Boeing Company created the term of "augmented reality". The most important concept of this technique is the integration and interaction between simulation images and real scene. It emphasizes not only the interaction between audience and screens, but also mutual integration between simulation and reality, spaces and materials4.

"Skill of Augmented Reality" has been applied broadly in the field such as recreation, medicine, education, military or commerce. With usage of "Augmented Reality" in industrial products designing, designers are able to do multiple aspects of simulation and planning for their products with engineering design. Educationally, "Skill of Augmented Reality" could make teaching materials more flexible, giving learners a more energetic and efficient environment.

The performance on stage of dramas could fully realize the interaction between simulation and reality by using apparatus of "Augmented Reality". Relying on this presenting environment, not merely the interaction between performers and screens could be presented, but also the audience would experience the feeling of virtually being there and the creative space formed by the integration between simulation and reality, spaces and materials. Recently, everybody can even touch with fingers the virtual articles in performing arts of foreign countries. Performers and virtual figures communicate, interact each other through "translucent screen", thus audience are impressed a lot. Referring to this principle, the reporter here plans to apply "apparatus of augment reality" to "Puppet Show in Hand", and offer this method for performers in Taiwan now to collaborate creative thinking. Effects presented by this apparatus would differ from that by traditional stage. "Model of Puppet Show" could make more difficult moves, and enlarge the space of imaging when creating.

Figure 1 shows that audience won't see directly real

performers lying on the floor. What they actually see are the projected images of performance on the stage. By means of this projecting way, real performers don't need too difficult or too dangerous motion to let audience see amazing acrobatic performances on the stage. This effect of this program would consider in stage II.

4 Azuma, R., A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, v6, n.4: pp. 355-385, 1997.

Figure 1 http://www.nytimes.com/2007/04/02/theater/02eyel.html?_r=1&oref=slogin

350350

In France, arts performers usually transfer arts

performances into digital video contents, projecting them onto ancient buildings, and let real scene and virtual images reflect each other. The unpredictable effects on the stage animate these old buildings again (figure2). During stage III, while the reporter here refer to this foreign technique "Augmented Reality", the contents, however, would be the endangered "Univers in Hand - Puppet Theater". This will let the people of Taiwan, even the world be able to see "the beauty of Taiwan Impression".

C. Puppet Show in Hand ： Puppet Show in Hand, originated from Chuen Zhou, Zhang

Zhou of Fukien, and spread in Chao Zhou of Guanton and Taiwan. The heads of "Puppet" are hollow woody head cases, the trunks and limbs are apparels made from clothes. During presentations, the operators stretch their fingers into the apparels of "Puppet" and the performances proceed. Because this kind of "Puppet" early days resembles "bags made form clothes" a lot, here comes the name "Peppet Theater (which has the very meaning).

"Puppet Show in Hand" has long history, Chinese ancient books ("Ancient Histories of Wu-ling[武林舊事]" and "Record of Dreaming China at Tokyo[東京夢華錄]" have recorded that there had been "Cloth Puppet in Hand" on the lists of banquets in Sung dynasty palace. This kind of play originated from a tale from Chuen Zhou: During Sung Dynasty, a scholar named Liang Ping-ling(梁炳麟), after failing to persue prestige in big examination many times, had ever learned tread-hanging-Puppet, developing it to figures operated directly by hands. With manner of literature, brilliant oratorical ability, being able to cite lots of anecdotes, he attracted huge number of people to watch his show immediately, and "Puppet Show in Hand" began prevailing from then on. By middle 17th century, it had been very popular in the southern Fukien. By 18th century, according to the difference of performing places, not only did professional "Puppet Show in Hand" organization appeared, but also they divided into outdoor traditional performance, performed outside, and indoor traditional performance, performed in theaters.

After 1960s, indoor traditional performance evolved into a tributary of television "Puppet Show in Hand". Modern "Puppet Show in Hand" impressed Taiwan most, even caused a great sensation: Not only did it reach the amazing audience rating of 97%, being a popular social subject at once, but also many figures in the plays nowadays still influence people life in Taiwan a lot. In 1988, Wen-Tze Huang and Chiang-Hua Huang brothers even transferred their business to video market and published a series of "Pili Puppet Shows". In 1993, they established "Pili Satellite TV", the first TV channel focusing on "Puppet Show in Hand"， was broadcasted, and entered the field of movie screen further. Although having tried to raise the

number of audience with some media such as video tape, VCD, DVD and cable television, or game software, the ability of interest created by Puppet Theater to young people is still not able to match that by Japanese animation and comics.

Just as the stunning effect created by contemporary artists that they interpret the ancient tradition by means of brand new art creating manners such as recording images or digital skills, the reporter here maintains that traditional "Puppet Show in Hand" could also be newly interpreted with "Augmented Reality" and the ingenuity of artists in modern Taiwan and performed everywhere people live. The effect will be direct and profound.

III. The setting of the "multimedia augmented reality" of "Puppet Show in Hand"

The creative work of this project still don't have precedent example in Taiwan. Thus, there are the principle of setting and the schedule of administration go as followed:

A. Technique Principle Arrange a space that isn't too bright (such as classroom,

rostrum or stage) and set up projector bright enough, short-focused and changed angle and almost translucent "transparent screen". First, transfer the presenting process of artistic performer into documental video, and utilize projector to project the video on the transparent screen, the angle of which inclines to the audience. Because of this inclination, the picture projected will be transformed. At this point, the angle of the projection must be inclined as well, and after adjusted by inclination, the images perceived can be corrected and become positive images finally. As long as the angle of projection is accurate and projected on the screen, the audience would see the vivid images suspend on the air. Besides, artistic performers can introduce aside or interact with simulating images.

B. Setting procedure From small space to high environment, interaction of

audience from indoors to outdoors, to demotic population, even to whole world, this project acquires three stages for ideal realizing.

Stage I: Set up a mechanism of "multimedia augmented reality". By molding of "Puppet Show in Hand" with VR4MAX and combining the skill of artistic multimedia design, a set of digital video can be created and be able to interact with audience.

First, mold of "Puppet Show in Hand" by using VR4MAX, and project them on transparent screen, so that the audience would see the images just as real "Mold of Puppet" acting in real spaces.

Figure 2 77e channel of MOD SODA_Place du Palais

351351

Second, install the mold "Puppet Show in Hand" produced

by VR4MAX into the creations(Figure 3) of artists such as Liu Pochun, professor of National Taiwan Artistic University, and interact with audience by means of mutual interaction: Besides the feeling "the shadow of trees" and the space of interaction between swaying and sounds/lights when audience go through the media of repetition, overlapping and being penetrated, they would even see "Puppet Show in Hand" appearing suddenly. The latter can be translucent, flying on air, appearing or vanishing suddenly and being incorporated with motion of passing audience and the space of creations, shuttled mutual, and become a part of works, making people experience this creative thinking "sculptures of voyage" besides multi-dimensional spaces.

Stage II: Utilize the skill of "multimedia augmented reality" in this project with "transparent screen", and supply it to modern artistic performers. The effect of combining "Puppet in Hand" simulated figure and performers will be totally different from traditional stage. "Mold of Puppet" can help performers present some difficult motion couldn't be realize before. "He" can be translucent, can appear or vanish suddenly, or make some motion of "soaring to sky, hiding into ground", enhancing the space of imagination in creating.

Stage III: Choose one building representing Taiwan impression as the background of presentation, and set up mechanism of "multimedia augmented reality" on its outwall or inwall, projecting the Taiwan folk culture this project aiming to perform. For one thing, use it as the finale of the project; for another, this is newest concrete presentation of modern trends of artistic technology and technical art. The goal is to present to demotic population the Taiwan imaginary folk art, which this project wants to promote, and enhance people's cultural manners by means of artistic presentation.

IV. Conclusion: The goal to achieve and its significance If this project is carried out, it will bring positive effects to

cultural education, national development and other applications.

First, use this set of mechanism to touch audience, especially the youngsters. The presentation of new technology can make them experience the beauty of endangered folk art "Univers in Hand ". In new games, have them accept the story

of Virtue in Puppet Theater, achieving the aim of purging people's heart.

Second, set up stages and project the digital video of Stage I on them with "transparent screen", and let factitious and real figures appear on the stages together, enhancing the space of presenting creating. Enjoying the plays, audience will be more vivacious and creative.

Third, project the contents of digital video in first two stages on representative buildings of Taiwan Impression, make the "Taiwan Impression", which this project wants to promote, presenting in front of demotic population. Depending on the power of art, enhance cultural manners of people, and let whole world see how folk arts in Taiwan would connect with the times in whole new appearance through media broadcasting

This project can be "new Renaissance in Graphic Arts Communication". Depending on this project, young people in Taiwan can wake up from the dream of "computer kingdom" and "remember where one's happiness comes from", "envisage the future" further. Efforts of the reporter here must be able to draw attention of the field of "Graphic Arts Communication", and inspire their thoughts of "new Renaissance", responding to the "Taiwan Imaginary" technical art movement. The people of Taiwan can make traditional cultures acquiring "new revival" and promote them to international stage. Furthermore, they will transmit the achievements to everywhere in the word through Internet and let people on earth see Taiwan. The reporter here not only wants to creative economical miracle, but also will resuscitate Taiwanese folk

References

[1] Azuma, R.(1997), A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, v6, n.4: pp. 355-385.

[2] LIN, Su-hui (2003) ： Illustrations et présentations des machineries intellectuelles à partir de Marcel Duchamp), Paris：Thèse de Paris IV.

[3] RUSH, Michael, Les Nouveaux medias dans l’art, Thames & Hudson,2000.

[4] SCHNECKENBURG, Manfred; RUHRBERG-Karl(2000), L’art au XXe siècle, Taschen.

[5] Fast Forward: Media Art : Sammlung Goetz (Hardcover) by Sammlung Goetz, Ingvild Goetz, Stephan Urbaschek http://www.amazon.com/gp/product/3980806391/103-3691167-7875824?n=283155 2007 / 12/ 01.

[6] http://info.gio.gov.tw/ct.asp?xItem=28281&ctNode=3802&mp=1. 2007 / 12/ 01.

[7] http://www.taiwan-image.tw/index.htm [8] http://www.nytimes.com/2007/04/02/theater/02eyel.html [9] http://zh.wikipedia.org/wiki/ , 2007/12/20. [10] VIDEO DATA BANK NEW LISTINGS 2004-2005VIDEO DATA

BANKNEW LISTINGS 2004-2005 http://www.vdb.org/resources/catalog2004.pdf

Figure 3 Liu Pochun( 劉 柏 村 ), Steel structure, (human beings vs.

352352