1. INTRODUCTION
In human daily activities, two hands nimbly cooperate with each other when people interact with objects. The hand’s movements and gestures are manifold. In Human-Computer Interaction (HCI), the benefit of bimanual interaction has been studied in past research. Although the interaction in mobile device typically involves both hands and people have ability to use both hands to operate the mobile device, the interface restricts the collaboration of two hands. This thesis is concerned with investigating design guidelines of bimanual interaction in mobile device through exploratory study of human’s daily activities.
1.1 Background
With the development of technology such as multi-touch screen and motion sensing technology, people could interact with interface naturally just like the interaction with real objects in daily life.
People swing the controller to hit the baseball on the screen like swinging real baseball bat while playing games with Wii. KinectTM of Microsoft allows player’s whole body to be the controller.
SixthSense (Mistry & Maes, 2009) lets user use natural hand gestures to interact with the information. In iPhone, people could scroll the contact list directly by finger instead of a scrollbar and the acceleration of the list would depend on how fast the finger moves. In these designs, the manipulation is natural, and the feedback is predictable.
These interaction styles are built on users’ pre-existing knowledge of the physical world. They make people achieve their goals more easily (Jacob et al., 2008). Over the last few decades, researchers have developed many different methods to apply pre-existing knowledge to interface design. The desktop metaphor that consists of folder, file, trash can and etc are derived from the office environment (Agarawala & Balakrishnan, 2006). The gesture interfaces apply the hand’s motion of communication to be the input modalities (Alpern & Minardo, 2003). Hinckley et al. (2010) have applied the bimanual interaction while people painting in physical world to develop a painting system.
When people interact with these natural interfaces, their body involves more than the “window, icon, menu, and pointing device” (WIMP) interface. The physical actions of body could facilitate the cognitive development through repetitive actions as known as motor memory (Klemmer, Hartmann,
& Takayama, 2006). This kind of memory could help people distinguish different functions by differentiations in actions and appearance (Djajadiningrat, Wensveen, Frens, & Overbeeke, 2004).
2
Through the meaningful actions, users have stronger expression of each function. In addition, Klemmer et al. (2006) also showed the errors that can be reduced since physical behaviors are highly related to commitment and risks.
Hands are the nimblest parts of human body. They could cooperate to do complicated tasks such as writing, painting, and sculpting. In Human-Computer Interaction (HCI), many researchers have studied the bimanual interaction in virtual environment. Bier, Stone, Pier, and Buxton (1993) developed Toolglass widget that allows user to use both hands to control the tools in drawing task. It could reduce steps, cursor motions, and errors. Russell et al. (2005) demonstrated that performing a compound task, painting, with two hands is faster than one hand. These researches showed people not only nimbly use two hands in physical world but also collaborate well in virtual environment.
1.2 Motivation
The interaction of mobile devices typically involves two hands. Current smart phones are equipped with many sensors for bimanual interaction such as multi-touch screen and accelerometer.
People have the ability to interact with them with two hands, for example, they use both hands to drive a car, shoot an arrow, or play labyrinth while playing games. But especially in general functions, the interfaces are still only designed for single hand. We believe that the bimanual interaction has potential in mobile device interface design. This thesis could support the designer to built bimanual interaction in mobile device.
The studies of bimanual interaction in mobile devices are seldom. Edge and Blackwell (2009) have investigated bimanual painting system with augmented reality in mobile phone. They allow user painting in 2D and 3D environment. Taylor and Bove (2009) have developed “The Graspables”; it is a box that could sense how user grasp it and provide corresponding function such as phone and camera. The aspects of bimanual interaction in mobile device need to be understood deeply and widely such as the cooperation of two hands, the input modality, and the usage of metaphors.
In order to develop natural interfaces, researchers would study the aspects of physical activity and physical interaction. Kruger, Carpendale, Scott, & Greenberg (2003) observed how people orient puzzle on table and brought up some implications on tabletop. Fitzmaurice, Balakrishnan, Kurtenbach, and Buxton (1999) conducted an exploratory study about supporting artwork orientation. They studied how people draw in physical world to develop a rotating UI in painting software. But in recent mobile device interaction study, this kind of study is seldom. The majority of researchers provided a concept design directly, and then demonstrated evaluation from user testing. (Bartlett,
3
2000; Wigdor, & Balakrishnan, 2003; Crossan, & Murray-Smith, 2004) The aspects of physical world are also an important issue while building mobile device interface.
To sum up, people deal with many tasks with both hands no matter in physical or virtual world.
Although the sensor techniques in mobile devices have developed maturely, present mobile device interfaces seldom allow users use it with two hands. The results of this thesis would help designers develop bimanual interaction in mobile devices. In order to design a suitable and user-center interface, the exploratory study of human’s daily activity could be also contributory in mobile device interaction design. We focus on this method and make this area more complete.
1.3 Objectives
The main purpose of this thesis is to develop guideline for mobile device interface that allow bimanual collaboration. We explored what should be concerned when people use both hands to interact with mobile devices from daily human’s actions first. From the observation of bimanual actions in daily life, we could define the collaboration patterns and roles of each hand. This pre-existing knowledge could help us address some implication of mobile device interaction design in future.
In sum, the goals of this thesis and the related issues are listed as follows
1. Realize how two hands cooperate with each other in human’s daily activity.
To this end, the following issues were posed:
(1) Which kind of activity is appropriate?
(2) The context of the activity.
(3) Patterns of bimanual collaboration.
(4) The roles of each hand in the activity.
2. Develop guideline of bimanual collaboration for mobile device.
To this end, we should concern:
(1) How to apply the knowledge of bimanual collaboration into mobile device interface?
(2) Which kind of task in mobile device is appropriate?
(3) How users feel about these new interaction styles?
4 1.4 Thesis Structure
We have organized the rest of this paper in the following way: Chapter 2, literature review, describe the theoretical foundation of bimanual action, the frame of activity analysis we adopt, the strategy of designing an intuitive interaction and the introduction of design pattern. Chapter 3, methodology, presents the research structure and the methods of each research activities. Chapter 4, the observation data of activity is analyzed and interpreted. Chapter 5, this study develope the design patterns for mobile device and presents the method of development. Chapter 6, the method of user evaluation is described. Results of user evaluation are analyzed and used to modify the guidelines.
We wrap up this thesis in Chapter 7 with conclusions based on our findings and with a discussion and future vision that are based on the implications of our findings.
5