網路連線協同學習與表演之布袋戲網路劇院

(1)

網路連線協同學習與表演之布袋戲網路劇院

許志宇柯龍嶺陳建霖

朝陽科技大學資訊與通訊系

王豪鋒王輝清

中興大學應用數學系

摘要

本文提出網路學習與數位內容典藏的一種創新觀念，將網路互動技術平台應用在布袋戲的學習、表演與欣賞，因此產生出由資訊與網路技術所建構出的網路劇院。透過這個網路劇院可使一般愛好民眾或布袋戲的藝術家都可以利用網路劇院進行即興創作或是合作表演，並且也可以將作品放在網路劇院上提供大家觀賞。這種布袋戲網路劇院，它也是屬於一種數位藝術表演的平台，能促進網路藝術 (net art) 的發展。本文內容包含網路劇院之觀念、系統架構與實作，並仔細說明如何利用網路與資訊技術來實現布袋戲網路劇院的目標。

關鍵詞：布袋戲，學習，表演，網路劇院，網路藝術。

HAND PUPPET THEATER ON INTERNET FOR DISTANCE LEARNING AND ART PERFORMING

Chih-Yu Hsu Long-Ling Ke Jian-Lin Chen

Department of Information and Communication Engineering ChaoYang University of Technology

Taichung County, Taiwan 41349, R.O.C.

Hao-Feng Wang Hui-Ching Wang

Department of Applied Mathematics National Chung-Hsing University

Taichung, Taiwan 402, R.O.C.

Key Words: hand puppetry, learning, performance, hand puppet internet theater, net art.

ABSTRACT

This paper proposes an innovative idea for an Internet application for collaborative learning, about performing with and appreciating Taiwanese hand puppets. The Internet hand puppet theater is a platform for distance learning and it is constructed by the computer vision and networking technology. Performing of an old art with new technologies produces a new cultural style to make traditional culture more accessible worldwide.

There are three technologies, Human-Computer Interaction, Collabo- rative and Distributed Visualization and User Interface, integrated together

(2)

for realization of the Internet Hand Puppet Theater. The original concept, system framework and implementation of the Internet hand puppet theater is presented in the paper.

I. INTRODUCTION

Glove or hand puppets resemble cloth sacks, and are also called bag puppets [1, 2]. Wood is used to figure a hollow puppet head sewed onto a cloth costume which is the puppet’s body. Besides, the puppet’s hands and feet are also sewed on to the cloth costume. The master puppeteer’s hand enters the puppet’s costume and makes it performe. Puppetry needs intellectuals to write scripts for stage performance [3]. The factors of puppetry’s at- traction for audiences are the skill of the operators and the stories in the scripts. The traditional hand puppetry needs a space at the temple to build a stage, in which place the master puppeteer performs for the deities. The audiences of traditional hand puppetry are getting fewer because the temples do not like to hold such kinds of activities. The business model of hand puppetry has been taken over by cable television and video tapes. A lot of factors have made traditional hand puppetry gradually fall from favor, but the most important one is less participation by intellectuals [4]. In order to encourage the intellectuals to join into the culture of hand puppetry, puppetry has been pre- served in a digital way [5], and offers a website for users to view puppet movies on the internet [6]. In the website, there are some interactive learning games and e-cards [7]

to interest people. Although the website offers puppetry movies to people, it can’t offer people a chance to control a hand puppet for learning and performing purposes.

Since the hand puppets, stages and lights and special effects are expensive for different families, most people find it impossible to play with puppets by themselves. To solve this problem, we apply information and networking technology to build a Hand Puppet Internet Theater and let people have their own virtual puppets and stages for learning, performing and appreciating puppetry. There are three technologies, Human-Computer Interaction, Col- laborative and Distributed Visualization and User Interface, integrated together for realization of the Internet Hand Puppet Theater. Hand Motion Capture and Shape Rec- ognitions are the main technologies used to realize Hu- man-Computer Interaction. To accomplish Collaborative and Distributed Visualization, Networking and 3D virtual reality technology are introduced. The User Interface has to consider voice communication on the Internet and the

technology of Voice over Internet was adopted. In the following sections, the backgrounds of these technologies are introduced for understanding.

To build the Hand Puppet Theater, programming software must be installed in a personal computer so the puppeteers can control the virtual puppet on the personal computer. The puppeteer’s hand images captured by a webcam can be used to detect the hand’s movement.

Different from technology using a data glove, a webcam with image processing algorithm has advantages without expensive equipment. The software can transfer parameters of hands to the server and then transmit them to some other client’s computer. These parameters can control the movement of the specific virtual puppet.

The goal of visualization is to display the 3D models of puppets or stages. The 3D models include lots of point data and computers rapidly calculate and show the render- ing result of these 3D data [8]. As the clock speed of CPUs gets more rapid, 3D scientific visualization pro- gresses with the efficiency of graphic chips. The goal of Computer-Supported Cooperative Work (CSCW) [9-11]

can be realized by visualization as well as by the Internet high bandwidth. The development of cooperative software has grown and there are many different 2D and 3D groupwares available on the market. There are two rea- sons for developing cooperative software. The first reason is to share good things with friends and share the suc- cess with others. Second, work done by ten must be faster than work done by one, besides this, ten people together can multiply their creation and thought, so creative software creates space for developing. However, the software development of CSCW give us no direct help for the development of Hand Puppet Theater on Internet, but the concept of net cooperative learning and the technology is used to construct the puppet theater.

Internet Puppet Theater must transfer the voice of the player to other client computers. The technology of Voice over Internet Protocol (VoIP) [12] is used to deliver voice. The voice communications on Internet can be achieved on packet-switched networks by setting up an SIP Express Router (SER) server with Session Initiation Protocol (SIP) [13]. An SER server has a database to register and identify the user’s ID and Code and SER also according to the RFC3261 [14] to support the SIP under

(3)

Fig. 1 A puppet controlled by one hand

the TCP and UDP. Hence, the audio streams can be sent by P2P through network.

To design Hand Puppet Internet Theater, the Java language and technology are chosen as the programming language because of platform independency. Java is suitable for cooperative learning, playing and appreciating the puppetry performance on the internet. The concepts and implementation technologies of the Hand Puppet Internet Theater are illustrated in the following chapters.

II. ORIGINAL CONCEPT

Hand Puppet Theater on Internet is a new art form different from the traditional puppetry theater. Hand Puppet Theater on Internet adopts the Java language and technology to build cross-platform software, and succeed at cooperative learning, performing and appreciation of puppetry.

1. Creativity

Traditional puppetry needs hands to control puppets as shown in Fig. 1. Since the traditional puppetry often is performed at a temple, the place and time limit the development of stage performing and the size of audiences is shrinking. In recent years, television has influenced traditional puppetry shows and a new art form has been cre- ated. For example, video shows of Pili and Jin-Guang are performed by puppets with gorgeous shape and effects of light which give audiences a real impression. Al- though the puppetry is attractive, it is difficult for a family operation to have puppets and stages with effects of light.

This is the reason that the people can not learn and perform puppetry. To solve these problems, we use the

technology of networking and virtual reality to generate virtual puppets and stages on the internet. The creativity of Hand Puppet Theater on Internet will help people learn, perform and appreciate puppetry on the Internet via computer. The technologies, hand motion capture, shape recognition, cooperative puppet playing, voice communication, and three dimensional virtual reality used to implement Hand Puppet Theater on the Internet will be described in section 3.

2. The Concept of the Hand Puppet Internet Theater The Hand Puppet Internet Theater is an internet platform for cooperative learning, playing and appreciating puppetry. Puppeteers can learn, perform and appreciate the puppetry on the Internet through a platform which is different from the traditional puppetry stage for performing puppetry. The Hand Puppet Internet Theater in cy- berspace just needs a computer and a webcam to control the virtual puppetry in the computer by the movement of fingers and palms. Hence, people need not be crowded by a narrow stage to control puppetry, and also need not set up equipment.

3. Technologies of the Hand Puppet Internet Theater The goal of the Hand Puppet Internet Theater is trying to realize that the people do not need any real puppet and they can interactively perform, learn and appreciate the puppetry together in different places using virtual puppets and stages via the computer and Internet. These technologies used to implement Hand Puppet Theater on the Internet are generally described in this section and their implementation will be presented in more detail in section 3.

The goal is controlling a virtual puppet in the computer by movement of fingers and palm, and adopting pat- tern recognition technologies of continuous sequence images to recognize the movement of fingers and palm.

This technology is one kind of algorithm to calculate rigid body motion by the optical flow and recognize the shape of a hand. The optical flow algorithm supposes the corresponding pixels have the same brightness for two continuous image frames. Based on the assumption, we can obtain linear differential equations of image pixels for displacement in the horizontal and vertical directions.

After adding a smooth restriction of the optical flow vari- ety, we get a displacement solution of image pixel by It- erative Method. The benefit of adopting optical flow algorithm is that we don’t need to understand the sur- roundings of the tracked object.

(4)

While performing the puppetry, performers need to communicate with the other performers. The technology of voice communication adopts an on-line multimedia protocol, SIP, to establish, modify and stop a session. SIP must work with other protocols to send audio and video packets because the SIP is only in charge of multimedia connections, because SIP possesses a Client-Server framework and it can use HTTP to send packets. Be- cause the SIP packet is smaller than an H.323 packet, the SIP is more suitable than H.323 for the transmission framework of WAN. Compared with H.323 protocol, the SIP is simpler, extensible and scalable. After network connection sessions build, the TCP protocol and Cli- ent-Server framework are adopted to transfer data packets.

The hand’s postures and palm’s movements are captured and calculated by an optical flow algorithm to pro- duce the parameters to control virtual puppets. The parameters with the information for puppet’s clothes, weapons and stage backgrounds are sent to the Server with the TCP protocol and then transferred to the other performers’

or audiences’ computers. After Client’s software obtains the data packets, it will visualize the shapes and movements of puppets and stage environment of the puppetry in the other performers’ or audiences’ computer. However, puppetry, just like on-line games, can be interactively controlled by yourself and others at the same time on the Internet. For every puppetry artist, he can play by him- self or cooperate with others to perform a show. By re- cording software, the puppetry can be recorded and put on the web server for audiences to appreciate it.

III. IMPLEMENTATION

To achieve the goal of the Hand Puppet Internet Theater, the system framework is extremely important and it relies on the networking and information technology to realize the concept.

1. System Framework

The system as shown in Fig. 2 is a client-server framework. In order to build an interactive platform on the Internet, the system framework adopted the TCP protocol to send packets, including the information of the parameters of hand movements, to control all puppets and clothes and weapons and stage backgrounds to the other client computers of performers or audiences.

The information on all puppets, clothes, weapons and stage backgrounds received by software in the client com-

Client 1 Client 2

Internet Internet

Server

Fig. 2 The framework of Client-Server system

(a)

(b)

Fig. 3 (a), (b) Two computers are connected via Internet Network. Two puppets are shown in each LCD at the same time and they are controlled by two people using their palm movements

puter will visualize the corresponding stage background and continually update puppets’ movements in each computer of performers or audiences as shown in Fig.3.

2. Hand Motion Capture and Shape Recognition The objective of hand motion capture and shape recognition is a transformation from hand motion and shape captured by video camera into the puppet’s movement.

The relative motion between the hand and the video camera will change the brightness of image pixels. An optical flow algorithm [12] is used to compute the brightness gradient of each pixel in two continuous image frames for capturing hand’s rigid body motion as shown in Fig. 4.

Translational motion and rotational motion are rigid body motions which can show the speed of the hands.

After the speed of a hand is detected, the posture of the

(5)

(a)

(b)

(c)

Fig. 4 Webcam in (a) captures hand movements. In (b) and (c) differences of hand movements in two frames generate optical flow to control one virtual puppet’s movements

hand must be recognized. The postures of hands are dif ficult to recognize due to natural variability. Recent de- velopments have overcome this problem; it has been shown that specific patterns of variability in shape and grey-level appearance can be captured by statistical mod-

(a)

(b)

Fig. 5 (a) Black contour of one hand (b) sequence images of hand contours

els that can be used directly in image interpretation [13, 14]. Fig. 5(a) and (b) show the black outline of one hand used to train a shape model and sequence images of hand contours. They were obtained from a set of images of an author’s hand. The contour points of finger ends and junctions are used as landmarks; other landmark points were equally spaced along the boundary between finger ends and junctions.

3. Cooperative Puppet Playing

As shown in Fig. 3, two computers are connected by network. Two puppets in the LCD are controlled by two people using their palms’ movements. Fig. 3(a) and (b) show two changing states before and after those palms’

movements. The most important thing is how to update the synchronous image of each Client. If the monitoring

(6)

images are sent from Client 1 to Client 2 via Internet, such a great amount of data will cost lots of transferring time and destroy any interest in the puppet playing. Because sending great lots of data causes the payload of the server and network bandwidth to go up, synchronous monitoring images of each host don’t achieve real time purposes.

The system is designed by using the parameters of each hand’s movements to control the movements of a virtual puppet in each client computer. Parameters of each client person’s hand movements are sent to the server and broadcast to all the other client computers. The movement of all virtual puppets is synchronous in each client computer. People who perform the puppetry are the same as those who play on-line games, they can control a puppet themselves or perform with others. If you want to use a different platform, you should consider the problem of cross-platform. So, the system’s design adopts the Java language to develop the executable program, and use the cross-platform capability to execute the developed executable program on different platforms.

4. Voice Communications

While performing the puppetry, each participant needs to deliver the voice to the other performers’ or audiences’ computers via the Internet. The Session Initiation Protocol (SIP) is a signaling protocol, widely used for setting up and tearing down multimedia communication sessions such as voice and video calls over the Internet.

Other feasible application examples include video confer- encing, streaming multimedia distribution, instant mes- saging, presence information and online games. The protocol can be used for creating, modifying and termi- nating two-party (unicast) or multiparty (multicast) sessions consisting of one or several media streams. The modification can involve changing addresses or ports, in- viting more participants, adding or deleting media streams, etc. Wikipedia [15] described Session Initiation Protocol (SIP) as a signaling protocol, widely used for setting up and tearing down multimedia communication sessions such as voice and video calls over the Internet. SIP was originally designed by Henning Schulzrinne (Columbia University) and Mark Handley (UCL) starting in 1996.

The latest version of the specification is RFC 3261 [16]

from the IETF SIP Working Group. SIP is an application-layer control protocol that can establish, modify, and terminate multimedia sessions (conferences) such as Internet telephony calls. SIP can also invite participants to already existing sessions, such as multicast conferences.

Invite Invite

Invite

Client A Client B

Client C

SIP server

Fig. 6 Diagram of function of SIP server

Media can be added to (and removed from) an existing session as shown in Fig. 6.

The SIP protocol admits several users to join into a group. Because the SIP possesses Client-Server framework and it can use the HTTP present packet information, the SIP protocol is suitable for an Internet phone communication system. By using SIP protocol and Voice over Internet Protocol (VoIP) [17], participants can not only control puppets, but also talk to each other at the same time in the performance of puppetry.

5. Three Dimensional Virtual Reality

Hand Puppet Internet Theater used the technology of 3D virtual reality to make people have a feeling of a 3D world. There is one virtual puppet on a stage as shown in Fig. 7. The 3D models are constructed by commercial software and implemented with our developed software.

IV. CONCLUSION

Human-Computer Interaction, Cooperative and Dis- tributed Visualization and User Interface are three technologies integrated together for realization of the Internet Hand Puppet Theater. The implementation of Internet Hand Puppet Theater is shown in 3D display and it is easy to transfer the hand’s rigid body motion to the puppet in real-time because of short computing time. The change of hands’ postures is difficult to use to control the postures of the virtual puppets in real-time because it takes a lot of time to run the recognition algorithm. The problem can be solved by an efficient algorithm or powerful personal computers in the future.

Performing of Taiwanese hand puppets on this plat-

(7)

Fig. 7 Pictures of two virtual puppets (a), (b) and (c) a stage

form with new technologies produces a new cultural style to make folkloric culture more creative and worldwide.

With networking technology, the Internet breaks the limits of time and area to make puppetry be accepted by other countries people. In the future, the stories played by Taiwanese hand puppets will cause people worldwide to know Taiwan’s culture. The concept of Hand Puppet Theater on the Internet gives contributions to art and the world while acknowledging those artists and thinkers who have helped us to a greater understanding of our place.

Collection of works by artists produced by this platform whose art and beliefs support the connection between the culture of Taiwan and the world, will be possible in the future.

ACKNOWLEDGEMENT

We thank National Science Council (NSC) supporting the funding of project (NSC 97-2115-M-324-001) which partially helped this study.

REFERENCES

1. Mi Zhou: Wu Lin Jiu Shi. China Book Store, http://

www.amazon.cn/dp/bkbk710276, (2007).

2. Meng Yuan Lao: Dong Jing Meng Hua Lu. China Book Store, http://www.amazon.cn/mn/detailApp?qid=

1227440889&ref=SR&sr=201-1&uid=168-7513730-7 950655&prodid= bkbk845220, (1982).

(a)

(b)

(c)

(8)

3. Lu Li Zheng: Bu Dai Hu Bi Ji. Taiwanese Magazine Store, http://findbook.tw/book/9789579943208/basic, (1991).

4. Qi Nan Zhang: Huang Hai Dai Ji Ji Bu Dai Hu Ju Ben Yan Jiu. Taiwanese Student Book Store, http://findbook.

tw/ book/9789571512051/basic, (2004).

5. Lin, W. P. and Tsai, D. R., “The Metadata Study Is Based on XML in Traditional Puppet Show,” Interna- tional Conference on Business and Information (2006).

6. Website of National Digital Archives Program: http://

www.ndap.org.tw/

7. Website of Handy Puppet Digital Museum: http:

//folkartist.e-lib.nctu.edu.tw/collection/palm_drama/ed ucation/gameIndex.shtml

8. Baldeschwieler, J. E., Blumofe, R. D., and Brewer, E.

A., “ATLAS, Systems Support for Worldwide Applica- tions,” Proceedings, the 7th ACM SIGOPS European Workshop (1996).

9. Rogers, A. S., “An Introduction to Groupware and CSCW,” BT Technology Journal, Vol. 12 No. 3, pp.

7-11 (1994).

10. Sierhuis, M. and Selvin, A. M., “Towards a Framework for Collaborative Modeling and Simulation,” In Pro- ceedings of CSCW96 (1996).

11. Philip, B., Michael, G., and Greg, J., “A Prototype Mo-

lecular Interactive Collaborative Environment (MICE),” In Proceedings of the Pacific Symposium on Biocomputing, pp. 118-129 (1998).

12. Horn, B. K. P. and Schunck, B. G., “Determining Op- tical Flow,” AI Memo 572. Massachusetts Institue of Technology (1980).

13. Cootes, T. F., Cooper, D., Taylor, C. J., and Graham, J.,

“Active Shape Models - Their Training and Applica- tion,” Computer Vision and Image Understanding, Vol.

61, No. 1, pp. 38-59 (1995).

14. Cootes, T. F., Taylor, C. J., Cooper, D. H., and Graham, J., “Training Models of Shape from Sets of Examples,”

Proceedings, the British Machine Vision Conference, Springer-Verlag, BRD, pp.9-18 (1992).

15. Wikipedia: http://en.wikipedia.org/wiki/Voip

16. RFC (3261): http://en.wikipedia.org/wiki/Session_

Initiation_Protocol

17. Wikipedia: http://tools.ietf.org/html/rfc3261

Manuscript Received: Feb. 04, 2009 First Revision Received: Feb. 13, 2009 Second Revision Received: Mar. 18, 2009 and Accepted: Apr. 03, 2009