UNCLE SAND: A SAND DRAWING APPLICATION IN IPAD

(1)

UNCLE SAND: A SAND DRAWING APPLICATION IN IPAD

1 Chi-Fu Lin (林奇賦), ² Chiou-Shann Fuh (傅楸善)

1

Dept. of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan

E-mail: [email protected]

ABSTRACT

Uncle Sand is a convenient application to produce sand drawings with your fingers. With only a few finger movements, you could realize all imaginations and reproduce the greatest times on your iPad.

Keywords: Sand drawing; Creativity; Multi-touch;

iPad;

1. INTRODUCTION

Different from other applications, Uncle Sand exploits the multi-touch feature of iPad to offer you with the most fantastic experiences and fun of sand drawing and produces sand drawings with high-quality light and shadow effects. Moreover, you could save the whole drawing process, integrate background music, replay your drawings, and even share your great work with your friends on social network.

The sand drawing is a special way of painting different from the brush paint, but the sand material, as a brush with both hands free to fiddle with sand to produce different shapes. Today's touch technology is quite mature, the Tablet PC and the growing popularity of handheld devices, prompting the design of a sand drawing art creation software on the Tablet PC, whether adults or children to have the opportunity to experience the fun of sand to create, play unlimited imagination and creativity, and create a unique work.

On the first day of its release in April 2010 in the US, Apple sold over 300,000 iPads and sold 3 million within the first 80 days after release. Initial sales were largely to consumers who are fans of Apple and technology enthusiasts wanting to use the iPad as a personal device, but attention soon turned to how the device could be used in artist settings [6].

2. RELATED WORK

The sand drawing is an art form, combining some elements of traditional animation and performing arts, telling a story, or using sand to create a series of scenes.

This is usually placed into the image projected onto the surface of the sand and shining light through it to the sand on the screen.

SandCanvas [1] is a multi-touch digital art medium inspired by sand animation, developed in NUS-HCI Lab.

2.1. Commercial Drawing Applications for iPad Apple iPad is undoubtedly one of the world’s best Tablets. After Apple has launched its iPad, many companies such as Samsung, Motorola, Sony, and so on, started developing their own Tablets. But they could not compete with the quality and the perfection of iPad in a short time. The touch sensitivity, retina display, extremely fast processors and creative applications are the strength of iPad [8][9][10][11].

2.1.1. Adobe Photoshop Touch

The current version provides 15 kinds of navigation teaching. Users can follow the steps to reach the same effects which are showed on the picture. The basic skills such as remove the background, modify and adjust the color, add or delete the portrait behind. The shift effect is the essential effect of the many photographic software.

Users just use the Fade effect which is in the software, it can be easily customized the scope of the shift effects.

The basic effect, Gaussian Blur, users can adjust the level of ambiguity, and then use the menu bar on the top right hand side “Add Fade”. There are straight and circle Fade effects for users to choose.

(2)

2.1.2. Paper

For functions, the free Paper only provides the users a pen. They have to buy the rest of tools such as pencils, marker pens, pens, and watercolor brushes. Users can decide to pay the additional fee by their needs. You can slide the tool bar from the bottom to the top. There are only nine colors and blank canvases. Without multi-function choice and the picture layers cannot be added, either. About the support of community, Facebook, twitter, E-mail, and so on. The main idea of Paper is doing a quick painting, so it only provides basic functions. However, through these different kinds of pens, it brings a totally different contents and ways. It can be used on meeting records, mood notes, and designing man

2.1.3. Sketch Club

Although it has plenty of functions, it can be operated easily. The buttons from the left hand side on the menu bar are: picture layers, forward, backward, brushes select, brushes setting, color identify tools, save, and share. About the picture layers, Sketch Club provides five layers, each layer has various effects of the stack to each other. Also, it can load pictures, set the level of transparent, and change the location of the pictures to create different effects. Like the upper right side, we load two pictures and use the overlay mode. It will turn out like what we show you on the picture. We can also find out that about the brushes, except the erasers, there are nine kinds of tools. Including the normal brushes, glasses, and pixel effects. Every kind of effect can show the different outcome. Users can explore by themselves to find out the interesting parts of the software. After selecting the brushes, users can also press the button on the right hand side of the menu bar. It can adjust the setting of specific details, for instance, the different types of the brushes. It is very flexible, and users can paint any effects that they want.

2.1.4. Art Rage

Art Rage is a pay raster image editing software which is designed by Ambient Design Ltd. Mainly to allow users to use the advantages of Tablet PC to paint on the screen with the computer pen. The software will provide any kinds of environments and tools that are in reality of painting to let the users paint willfully. Art brushes, watercolor brushes (felt pen), pastels, crayons, pencils, and erasers are all provided in the software. Furthermore, it also provides four different high quality of canvas.

Fig. 1: Adobe Photoshop Touch.

Fig. 2: Paper.

Fig. 3: Sketch Club.

(3)

Fig. 4: Art Rage.

2.2. iPad Features & Touch Technology The basic iPad 1 generation internals are:

 Processor: A custom Apple A4 ARM processor based on a single-core Cortex-A8, running at a 1 GHz.

 Volatile storage: 256 MB DRAM.

 Non-volatile storage: 16, 32, or 64 GB solid state storage drive.

 Wireless connectivity: 802.11 a/b/g/n and Bluetooth 2.1, the same as every iPhone.

 Besides, the 3G model features an A-GPS (Assisted GPS), and hardware for communicating over UMTS/HSDPA (820, 1900, and 2100 MHz) and GSM/EDGE (850, 900, 1800, and 1900 MHz.

The Apple iPad’s touch screen works on a capacitive principle [14]. It senses our body's natural electrical capacitance when our finger makes contact with the screen's surface. A fine grid of capacitive sensors, known as the digitizer sits behind the glass, so when it feels to touch our finger contactors and relays digitizer positioning.

The iPad's screen distinguishes the presence of multiple fingertips at the same time. In the case of the iPad, the screen detects up to 11 separate points of contact. The digitizer tracks the motion of these points, and real-time processor allows users to multi-touch gestures to control the iPad.

The height and the width of the iPad resemble the most popular size of a drawing panel. The iPad is also very thin, with a depth of 1.34 cm, which makes it a very easy device to transport and handle [14]. The iPad has a relatively large 9.7 inch diagonal multi-touch LED display with a 1024 x 768 pixel resolution, which provides viewing of bright, high-quality images.

Compared with iPhone’s 3.5-inch screen size, the iPad is more suitable for drawing. The iPad also has a wide viewing angle display (178°) which allows different people to view content on the iPad within the angle, hence the ability for users to collaboratively share viewing content with one another without distorting images.

3. DESIGN AND IMPLEMENTATION Uncle Sand is designed to run on Apple iPad. Our software was written with xcode, an Integrated Development Environment (IDE) containing a suite of software development tools developed by Apple for developing software for OS X and iOS.

We choose Apple iPad as our operating platform, and there are several special challenges:

1. Design easier and friendlier interface for users.

2. Simulate sand accumulation and pouring.

3. Record and broadcast the process of painting.

4. The original limits of iPad touch panel hardware.

3.1. Uncle Sand User Interface

The different painting brushes and tools occupy much screen space in the normal painting software, and the numerous tools make the screen look messy. Therefore, we analyzed the common ways of making sand drawing, and also the ways of holding Tablet PC (see Figure 5).

According to these two results, we designed two semi-transparent function keys on the both sides of the screen. Users can move the function keys to make the painting process smoother. The tools in common use are hidden in the bottom of the screen. Users can just pull them out when the tools are needed (see Figure 6).

Left and right sides represent hand sweep mode and sand pouring mode. Users can freely transform the mode of the left and right by dragging. Toolbar is usually hidden at the bottom. Pressing it will pop-up menu. Clicking the home icon will switch to the main screen. On the left of the screen are some commonly used functions right before archiving the works (see Figure 7).

(4)

Fig. 5: An initial screen.

Fig. 6: The user interface panel. From left –home page, renew, save drawing, play drawing, album page, and

share to Facebook.

Fig. 7: Home screen.

Clicking the new icon will reset screen (see Figure 5).

We design the style of the album's interface similar to movie theater. Escape door corresponds to exit function;

trash corresponds to deleted function; and palm corresponds to the read function. The bottom of the screen lists all the paintings saved (see Figure 8).

Fig. 8: Album screen.

3.2. Simulation of sand accumulating and pouring The accumulation of sand refers to [2][3]. Due to the limits of iPad processor speed, we need to balance the verisimilitude and the fluency to make the painting look real. On the other hand, we also want to keep the fluency of painting. It needs many tests and adjustments, for example, to find out more efficient algorithms or use some of the graphics engines to accelerate the processing speed.

We have designed several modes to draw: hand sweep mode, sand pouring mode, and sand settling mode.

Fig. 9: When you touch the hand sweep mode button, you can use two fingers to sweep sand.

(5)

Fig. 10: When you press the sand pouring mode button, you can use multi-fingers to pour sands.

Fig. 11: When you press two buttons at the same time, you can settle the sands.

3.3. Record and Play of the Process of Painting About the process of painting, we record the users’

drawing tracks and ways of painting continuously to let the users replay the process of painting at any moment.

Thus the entire process will look like a movie rather than a painting.

First, we label every action with a unique number.

When the user triggers any action, we will act and record the labels, and write to the log file finally.When the user wants to play back, we sequentially read the log file, different labels corresponding to different functions to reproduce the actions done before.

3.4. The Original Limits of iPad Touch Panel Hardware

If iPad touch panel does not work with some special devices, it cannot detect the sense of pressure. Also, the touching area is sensed by point instead of the superficial measure. All these factors will influence the users when they are painting. It also increases the

difficulties of simulation. Regarding these limits, we figure out several alternative ways: First, we use gravity sensors to detect the pressure when the finger touch the screen, the strength of pressure will directly affect the thickness of the lines. We could take this to simulate the function of the sensation of gravity pressure.

Fig. 12: Using multi-fingers to draw.

(6)

3.5. Use Accelerometer to Simulation Pressure The accelerometer in iPhone and iPad measures the acceleration of the device relative to free-fall. The accelerometer measures the acceleration of the device in three different axes: X, Y, and Z. A value of 1 indicates that the device is experiencing 1g of force exerting on it.

We use the UIAccelerometer class to receive acceleration-related data from the onboard hardware.

The raw acceleration values are stored in the UIAcceleration object passed into this message. The values are simply what the hardware is reading - where +1 is approximately equal to 1G of acceleration (see Table 1).

Table 1: The Various Values of the X, Y, and Z Axes.

Position X Y Z

Landscape Left 1.0 0 0

Landscape Right

-1.0 0 0

Upside Down 0 1.0 0

Vertical Upright Position

0 -1.0 0

Flat Down 0 0 1.0

Flat Up 0 0 -1.0

By constantly receiving the current acceleration value, and the analysis of different percussion force, received by the three-axis value changes, the calculations and the different strength of classification, and finally when the light tapping force, draw the lines are also smaller, tap the strength of heavy thick lines (see Figure 13).

Fig. 13: By accelerated to produce different thickness of lines.

Table 2: Classification of drawing modes

Mode Description

Pouring Press the sand pouring mode button, use multi-fingers to

pour sands.

Hand sweep Press the hand sweep mode button, use two fingers to

sweep sand.

Settling Press two buttons at the same time.

Canvas pouring Press the sand pouring mode button and shake device.

Fingertip drawing Use any of the fingers, sliding canvas.

4. USER EVALUATION

After we completed the app, need to find some users to test our user interface is friendly, the painting process is smooth, and verisimilitude, to collect the user's actual experience and feedback, to provide our future work for improvement.

4.1. Test Method

After designing the app, we need some app users to test the app to find out if it is friendly to use, and also the level of verisimilitude. We collect the feedback and the actual experience from the users for our future approval.

We tested ten users, three users under the age of fifteen, five users that are between fifteen to thirty, and two more than the age of thirty.

The way of testing friendly interface, we let the users use Uncle Sand freely. We do not teach the users how to use it, so we can observe if the users can use this app.

We found out that half of the users can completely use all the function. Most of them do not know that sweep sand mode needs to be done by two fingers, and they do not know that the sand can be drained off by pressing both of the two buttons at the same time.

After the testing, we show the manuscript by several simple pictures (see Figure 15, Figure 16, Figure 17 and Figure 18), and we also teach the users how to use the function buttons on the screen. Finally, we found five users to test, and this five users can use all the functions and they can draw smoothly without any teaching.

Most users feel interesting and fresh about making sand drawing on the Tablet PC, and give positive feedback. They also think that without those complicated buttons, they can be more concentrate on making sand drawing.

During the testing, we found out that users are fond of using the function of replay because it is different from the normal way of painting (see Figure 14). But we also

(7)

realized that because of using the log file to record the actions of users, they cannot share the process to other people who do not have this app. Therefore, we add a new function: conversion. It can output the painting process into h.264 video format (.mov) with the music they like and the video editing software, so the users can share it to anyone to watch.

Fig. 14: Play mode.

Fig. 15: Using multi-fingers to draw.

Fig. 16: Hand sweep.

Fig. 17: Pouring.

Fig. 18: Settling.

5. CONCLUSION

We design and implement a sand drawing APP in iPad, so that users do not need a complex environment to experience the fun of the sand drawing and the ability to record all the painting process, and playback.

For different modes of operations, we simplify the complexity of the operation as much as possible through several gestures to smooth painting.

Complete the implementation of APP we have uploaded to the Apple store to provide all users to download. APP is divided into a lite and full in two versions. Lite version [12] cannot output the paintings into a movie format, and the lack of function sharing to social networking sites. Full version [13] is without any limitations.

Users can receive very different creative experience, to let everyone enjoy their unlimited imagination and creativity. Tablet PC and mobile devices are becoming increasingly popular, how to design a good APP. We need to work to research. Everyone can successfully use and have good user experience.

(8)

REFERENCES

[1] R. H. Kazi, K. C. Chua, S. Davis, R. Davis, and K. Lim,

“SandCanvas: A multi-touch art medium inspired by sand animation,” Proceedings of ACM CHI, Vancouver, Canada, pp. 1283-1292, 2011.

[2] K. Onoue and T. Nishita, “An Interactive Deformation System for Granular Material,” Computer Graphics Forum, 24(1), pp. 51-60, 2005.

[3] R. W. Summer, J. F. O'Brien, and J. K. Hodgns,

“Animating Sand, Mud, and Snow,” Proceedings of Graphics Interface, Vancouver, Canada, pp. 125-132, 1998.

[4] D. Myriam, “About Sand Drawing Live Sand Drawing

Performances: Magicalshows,”

http://tramage.com/About-sand-drawing.html, 2012.

[5] Y. Zhu and R. Birdson, “Animating Sand as a Fluid,”

Proceedings of ACM SIGGRAPH, Los Angeles, CA, pp.

965–972, 2005.

[6] P. McDougall, “iPad Is Top Selling Tech Gadget Ever,”

http://www.informationweek.com/news/227700347, 2010.

[7] T. V. Wilson and S. Crawford, “How the iPod touch Works,”

http://electronics.howstuffworks.com/ipod-touch2.htm, 2010.

[8] Adobe Systems Inc., “Adobe Photoshop Touch”, http://itunes.apple.com/app/adobe-photoshop-touch/id4957 16481, 2012

[9] FiftyThree, Inc., “Paper by FiftyThree on the iTunes App Store,”

http://itunes.apple.com/us/app/paper-by-fiftythree/id50600 3812, 2012.

[10] J. Scott, “Sketch Club on the iTunes App Store,”

http://itunes.apple.com/us/app/sketch-club/id404414176, 2011.

[11] Ambient Design Ltd., “ArtRage on the iTunes App Store,”

http://itunes.apple.com/us/app/artrage/id391432693, 2011.

[12] D. P. Studio, “Uncle Sand Lite on the iTunes App Store,”

http://itunes.apple.com/tw/app/uncle-sand-lite/id44778296 5, 2011.

[13] D. P. Studio, “Uncle Sand on the iTunes App Store,”

http://itunes.apple.com/tw/app/uncle-sand/id447985381, 2011.

[14] Apple Inc., “iPad User Guide For iOS 5.1 Software,”

http://manuals.info.apple.com/en_US/iPad_User_Guide.pd f, 2012.