Interactive Visual Display of Museum Artifacts with
3D Magic-Ball
Yi-Ping Hung1,2, Gwo-Cheng Chao1, Yu-Pao Tsai , 3 Chien-Wei Chen2
1
Graduate Institute of Networking and Multimedia, National Taiwan University, Taiwan.
2
Department of Computer Science and Information Engineering, National Taiwan University, Taiwan
3
Department of Computer and Information Science, National Chiao Tung University, Taiwan.
Abstract. An interactive visual display system for displaying digitized 3D objects, which
provides the users to browse the 3D objects in an intuitive manner, is developed. Our system allows the users to see a virtual object appearing inside a transparent ball, and to observe 3D objects from any direction by rotating the ball. The display system looks like a crystal ball and is very attractive and expressive. In this paper, we describe the implementation of the display system and the application for displaying museum artifacts.
1. Introduction
With the spreads of digital satellite broadcasting and Internet-based electronic-information services, digital content (information in digital form) is expanding rapidly in society and individual households in the form of text, still pictures, motion pictures, sound, etc. In such a digital-multimedia era, the archiving of this huge volume of data has become a major issue. Recently, many countries are promoting the project of digitalizing and preserving the cultural relic in archives. Such an archive would become crucial since these heritages can be injured by natural causes, decay or any other accident. In the aggressive thinking, the people can appreciate these great wisdom heritages through the digitalizing preservation.
A very important research subject is how to demonstrate these digitalizing materials and let the cultural relic reappear its elegant demeanor in processes of digitalizing. An example of the digital archive is the digital Michelangelo project [1]. Since 1992 Professor Marc Levoy and his students have been investigating methods for digitizing the shape of three-dimensional objects using laser scanners. In 1996, they digitized a small Buddha statuette and emailed the computer model from Palo Alto to a fabrication facility in Los Angeles where an accurate replica was made, thus demonstrating at least the feasibility of building such a machine. Their Digital Michelangelo Project officially began in January of 1997 with a two-year planning period , and the production phase of the project, from September of 1998 to June of 1999, consisted of scanning the artworks themselves. At present, they have built a
full-resolution model of one statue – Michelangelo’s St. Matthew and several medium-resolution models of a second - the head of David. They have also built crude models of the other eight statues.
Another example of the digital archive is the great Buddha project [2]. Dr. Ikeuchi and his students aim to develop various methods based on computer vision and graphics technologies to construct digital archives of cultural heritages. They obtain the geometric modeling by using three steps : mesh generation, alignment, and merging. For mesh generation, they use varieties of 3D laser scanners that have different measurement ranges and accuracies. For alignment of range images, they developed a robust simultaneous alignment method, which can be executed in a parallel manner for further efficiency. Also their merging method takes special care of sensor noise, by considering the consensus of multiple pre-aligned range images. So far, they have applied their geometric modeling pipeline to several important statues in Japan, including the Nara and the Kamakura Big Buddha.
There have been rapid advancements in 3D techniques and technologies, by the influence of the World Wide Web and the motion type environment. At present, the computer vision, the virtual-reality research, and development have become the trend. Many well-known schools, research institutions, and big companies are engaging in this field. The history of 3D displays is well summarized in several works. Okoshi [3] and McAllister [4] present histories of the development of 3D technologies. On the other hand, hardware has both improved and become considerably cheaper, making real-time and interactive 3D available to the hobbyists and the researchers as well.
Our team also has the contribution and the progress in research of the digital archive. Previously we have design a 3D stereoscopic kiosk system (Fig.1), which combines the Kiosk and the techniques of digital archives, to demonstrate the virtual exhibition of the 3D historical relics of the museum survey system. These system uses object movies, panoramas, 3D/VR and 3D stereoscopic displays to present the contents of digital archives. This kiosk system combined modern science and technology of the archive to preserve the cultural relics. In this research we first focus on the works of cultural relics for 3D digitization to obtain the suitable source material this plan needs. We have selected 48 pieces of cultural relics for 3D digitization (or more precisely, for shooting stereoscopic object movies), where 24 pieces of the tri-colored glazed pottery of the Tang Dynasty are from the National Historical Museum and 24 pieces of the cultural relics of the Shang Dynasty are from the Institute of History and Philology, Academia Sinica. The total numbers of the acquired images exceed 20 thousands, and the data amount is over 15.5G. We then perform background removal for those images, collect the related information of those cultural relics, design how the content should be presented, plan and build the website, investigate the different stereoscopic display technologies, and perform the system integration.
The goal of this paper is to develop a 3D Magic ball system (Fig.2) for virtual exhibition of cultural relics. A sphere and transparent development, i-ball, which can demonstrate the space image (can observe objects inside the display from any direction)[5]. “i-ball” displays virtual objects within the transparent ball, and by capturing viewer’s behavior, the system can display images interactively. Stage unit is introduced for the purpose of rotating the ball in the “i-ball” system. We take the concept of the "i-ball" as the starting point and start to research developing 3D Magic
ball, which can let the user contact, operate, and interact simultaneously.
This paper will describe the hardware design and installment of the 3D Magic ball and the application of displaying museum artifacts by integrating object movie techniques.
Fig.1: A 3D stereoscopic kiosk system.
Fig.2: A 3D Magic ball system.
2. Object Movie
2.1 Data Acquisition
We choose artifacts of the Tang Dynasty in the National History Museum and bronze ware of the Shang Dynasty in Institute of History and Philology, Academia Sinica to be displayed in our 3D magic ball system. The artifacts chosen are the representatives with complete and correct records of excavation, and are valuable for academic research. Some of the artifacts are shown in Figure 3.
Fig. 3: Some of those artifacts displayed in our 3D Magic ball system.
In order to render high quality and photo-realistic 3D artifacts in our 3D magic ball system, we use image-based technique (Object Movie). An object movie is a set of images taken from different perspectives around a 3D object; when the images are played sequentially, the object seems to be rotated around itself. This technique was first proposed in Apple QuickTime VR and its advantage of being photo-realistic is suitable for delicate artifacts.
In this work, we use Texnai’s autoQTVR standard edition, which provides accurate angular control and automatic camera control, as Figure 4 shows. The Texnai’s autoQTVR standard edition is controllable with traditional PC through RS232 serial port connection, and after setup process, images of different viewing directions will be automatically taken. By using this Object Movie capture device, we know the pan and tilt angles of the viewing direction associated with each capture image, and thus we can choose correct image to render correct views according to pan and tilt angles of viewing direction. In this way, the user can interactively rotate the virtual artifacts at will and enjoy the realistic interaction through this Touchable 3D Display, the 3D magic ball.
2.2. Background Removal
In order to improve the display quality of the object movie, or even to integrate the object into a new background [7], the original background of the object movie should be removed. Although many techniques of video object segmentation have been proposed [6][8], no automatic method is efficient enough for removing the background of object movies. To obtain a better segmentation result, the user intervention is necessary.
In this work, we develop an interactive software tool for removing the background of object movies, which lets the users perform the work of background removal in less time with least intervention. As the first step, our software tool automatically extracts an initial contour of each image in object movies based on the characteristics of the acquired images of object movies. These characteristics are (1) the distribution of background color is Gaussian, and (2) the color difference between foreground and background is distinct. If the initial contours of some frames are not good enough, the users can manually correct some of those frames with some easy-to-use modification tools provided by our system. After user intervention, the verified segmentation results are used to refine other unverified frames with an automatic propagation process.
After all frames of the object movie are verified by the user, we perform the alpha estimation process on boundary pixels of the extracted foreground to obtain the alpha values, because these pixels may be composed of both background and foreground. Here, we adopt the method proposed by Hillman et al. to estimate the alpha values of the boundary pixels. Figure 5 shows comparison of the segmentation result before alpha estimation, 5(b), and after alpha estimation, 5(d). Using the alpha value, we can produce a smooth contour blending when we integrate OM into a new background, as shown in Figure 5(e).
(a)
(a)
(b) (c) (d) (e)
Fig. 5: Comparison of the segmentation before alpha estimation and after alpha estimation. (a) shows an original image. (b) shows the alpha map before alpha estimation. (c) shows the segmentation result with green background using the alpha map shown in (b). (d) shows the alpha map after alpha estimation, and (e) shows the segmentation results with green background using.
3. The Hardware of the 3D Magic Ball
The hardware of the 3D Magic ball consists of two major modules: 1. Optical unit for image formation
2. Interactive control device.
3.1. Optical Unit for Image Formation
Fig.6 shows the element of the Image demonstration .The image displayed from the LCD(15’ ,XGA) is reflected by the mirror, and then penetrates the Fresnel lens. Thus, the user can see the image appearing inside the glass ball (diameter 25cm), as
shown in Fig.7. Notice that the image is reflected by the mirror, we should flip the image in horizontal direction before displayed it in LCD monitor. In addition, when observing image it is better that the light is dim slightly than in doors and indirect lighting in the environment is preferred.
Fig.6: The element of the Image Fig.7: The image appearing inside the transparent. demonstration. Ball.
3.2. Interactive Control Device
The controlling device of the display system is the “transparent ball". Fig.8 shows an example that the user looks a virtual object appearing inside the transparent ball and browses the object by rotating the transparent ball. Under the transparent ball there are 3 wheel mice to support it, as shown in Fig.9. In order to support the transparent ball, we modified the mouse ball to be prominent. In addition, we want to obtain the pan and tile angles of the transparent ball using the three mice when the user rotates the transparent ball to browse the object movies. Because a general PC is unable to receive three mice signals simultaneously, we design a hardware device using 8051 single chips to detect the 2D motions of the three mice and then transmit the motions to the computer using the COM port.
Next we have to calculate the pan and tilt angles using the set of received mouse motions. For easy calculation, we carefully fix the three mice in a rigid half-circle, as show in Fig.9. Notice that the estimated pan and tilt angles are used to response to the user’s intervention so it do not require high accuracy. Here, we adopt a simple algorithm to estimate the pan and tile angles.
1) Pan angle value: We let the three mice be carefully located on an identical horizontal circle, and adjust the mice so that they have only x-direction motion when the transparent ball has rotation about pan direction. Therefore, we can obtain the pan angle of the transparent ball by averaging the x-direction motions of the three mice.
2) Tile angle value: We know that y-direction motions of the three mice are different when the transparent ball has rotation about tilt direction; the mouse2 has the biggest value of y-direction motion, but other two mice have smaller values. Fig 11 illustrate the traces of mice motions (black circles) when the transparent ball has rotation either about pan direction or tilt. We first calculate the perimeter of each circle and then calculate the ratio of the y-direction motion of each mouse and the tile angle value of the transparent ball. Thus, if we can obtain y-direction motion of mouse, we can calculate the tilt angle of the transparent ball.
Fig.8: The viewer sees the image Fig.9: 3 wheel mice to support the transparent ball. appearing in the transparent ball.
(a) (b)
Fig.10: The hardware device used to receive the motions of the three mice. (a) Outside of the coverage box . (b) the inner of (a).
Fig 11. Illustration of traces of mice when the transparent ball is rotated.
4. Summary
In this paper, the artifacts are digitized with the image-based approach (or more precisely, the object movies approach) so that its digital presentation is photo-realistic. In addition, we develop a 3D interactive display system, called the 3D Magic-Ball, with which the user can see a virtual object appearing inside a transparent ball and can rotate the virtual object by rotating the transparent ball. We hope that our system can attract people to appreciate and to learn from the museum artifacts, and can thus create more values for the digital content.
References
1. http://graphics.stanford.edu/projects/mich/
2. http://www.cvl.iis.u-tokyo.ac.jp/research_e/geometry/modeling_from_reality.htm 3. T. Okoshi, Three-dimensional imaging techniques, Academic Press, NY, 1976
4. David F. McAllister, Ed. Stereo Computer Graphics and Other True 3D Technologies, Princeton U. Press, Princeton, NJ, 1993
5. H. Ikeda, T. Naemura, H. Harashima and J. Ishikawa : “i-ball: Interactive Information Display like a Crystal Ball ” , Conference Abstract and Applications of SIGGRAPH 2001, p.122 , 2001
6. I. Patras, E. Hendriks, and I. Lagendijk, “Video Segmentation by MAP Labeling of Watershed Segments,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 3, pp. 326-332, 2001.
7. R Szeliski, H-Y Shum. “Creating full view panoramic image mosaics and texture-mapped models.” In Proceedings of Computer Graphics (SIGGRAPH’97), ACM Press: Los Angeles, CA; pp. 251–258, 1997.
8. D. Wang, “Unsupervised Video Segmentation Based on Watersheds and Temporal Tracking,” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 8, No. 5, pp. 539-546, 1998.
