QPalm: A gesture recognition system for remote control with list menu

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 1

Abstract—The coming ubiquity of digital media content is driving the need of a solution for improving the interaction between the people and media. In this work, we proposed a novel interaction technique, QPalm, which allows the user to control the media via a list menu shown on a distant display by drawing circles in the air with one hand. To manipulate a list menu remotely, QPalm includes two basic functions, browse and choosing, realized by recognizing the user’s palm performing circular and push motions in the air. The circular motion provides fluidity in scrolling a menu up and down, while push motion is intuitive when the user decided to choose an item during a circular motion. Based on this design, we develop a vision system based on a stereo camera to track the user’s palm without interfering by intruders behind or next to the operating user. For more specifically, the contribution of the work includes: (1) an intuitive interaction technique, QPalm, for remote control with list menu, and (2) a palm tracking algorithm to support QPalm based on merely depth and motion information of images for a practical consideration.

Index Terms—remote control, selection technique, human-computer interaction, gesture recognition

I. INTRODUCTION

Interaction design generally refers to the discipline of defining the behavior of products and systems that a user can interact with. Many interaction techniques are developed to help the user manage jobs more intuitive, effective, and natural. Some methods use a substantial device, like a pen, to control the media content [4][2],

Manuscript received October 9, 2001. (Write the date on which you submitted your paper for review.) This work was supported in part by the U.S. Department of Commerce under Grant BS123456 (sponsor and financial support acknowledgment goes here). Paper titles should be written in uppercase and lowercase letters, not all uppercase. Avoid writing long formulas with subscripts in the title; short formulas that identify the elements are fine (e.g., "Nd–Fe–B"). Do not write “(Invited)” in the title. Full names of authors are preferred in the author field, but are not required. Put a space between authors’ initials.

F. A. Author is with the National Institute of Standards and Technology, Boulder, CO 80305 USA (corresponding author to provide phone: 303-555-5555; fax: 303-555-5555; e-mail: author@ boulder.nist.gov).

S. B. Author, Jr., was with Rice University, Houston, TX 77005 USA. He is now with the Department of Physics, Colorado State University, Fort Collins, CO 80523 USA (e-mail: author@lamar.colostate.edu).

T. C. Author is with the Electrical Engineering Department, University of Colorado, Boulder, CO 80309 USA, on leave from the National Research Institute for Metals, Tsukuba, Japan (e-mail: author@nrim.go.jp).

while others simply use a gesture to make a command [1] [6]. The EyeToy, devised in 2003, is developed for playing the games on PlayStation 2 by using a camera sensor. The Eyetoy allows the players to interact with games by their hand motions and sound without the need of holding a game controller. Wii, another popular game console was released by Nitendo in September, 2006. A distinguishing feature of the console is its wireless controller, the Wii mote, which can be used as a hand-held pointing device in which the built-in sensor can detect its motions in three dimensions. In addition to game industry, digit home situates another scenario requiring sophisticated interaction design. From air condition to television, most home appliances can be controlled via a control panel built-in or a remote controller. Not only would the user be confused by multiple controllers, but these appliances might share different metaphors in-between designs could also confuse its users. It is apparent that developing new solutions is demanding for next generations.

In this work we propose a new type of interaction technique, QPalm, for remote control with list menu. The interaction technique is based on recognizing the users’ hand motion by using a stereo camera so that the user has no need to take any controller. In the proposed design, all information is gathered into a list menu, while the menu is displayed on a screen distant to the user. QPalm allows the users to browse and to choose an interested item in the list menu by simply drawing circles in the air with their hands. Specifically the browse and the choosing functions for the interaction are described as follows. Browse function allows the users to control the list menu by

QPalm: A Gesture Recognition System for

Remote Control with List Menu

Graduate Institute of Networking and Multimedia, National Taiwan University email: hung@csie.ntu.edu.tw

Fig. 1. This figure indicates the interface of QPalm. A circular motion makes the highlighted item scroll up and down, while a push motion triggers the choosing function.

> m cl ve m ite an so be in or be th m in W m as w di co in m su in w Th pl O pr m vo gr th w up ea pr vi M ex Fi ov REPLACE T making circul lockwise scr ersa (Figure making a pus em (Figure 1 nd a push mo o that the pro

QPalm is d ecause list m nterfaces. Mo rganized into e manipulate he users’ ha motion detect nformation o Without cons makes our sys s the system wearing and ifferent light ontributions nteraction tec menu, and (2 upport QPal nformation of Recent yea which use the he most fam layer which October, 200 roduct is the manner to sc olume. Ther raphics displ he thumb on t wheel and the p and down. asier since a There are a ractical app irtual scroll r Moscovich et xisting gener

ig. 2. (a) The sy verview.

THIS LINE W

ar motions w rolls the hig

1(a)). Choo h motion, on 1(b)). The co otion seems oposed work devised prim menu is one of ost informati o a hierarchic ed remotely v ands with ro tion algorithm of the image sidering col stem practica would not b would work ting conditio of the wo chnique, QPa 2) a hand m lm based o f images for II.RELA

ars there are circular mot mous example h is designe 01. The mo e click wheel croll through re is a clic lay placed on the click whe e highlighted In their desi smooth moti also some cir lications lik ring (VSR) [2 t al., is a sc ral positionin ystem environm ITH YOUR P

with one hand ghlighted ite

osing functio nce the user ombination o drawing the is named as marily worki

f the most co ion and func cal list menu via QPalm. T obustness, w m based on es taken by or informat al to be applie e harmed by k in most e ons. For mor ork include: alm, for remo motion detec on merely d a practical c ATED WORK e some prod tion as their m e is the iPod ed and relea st well-know l, which is u h menu item k wheel an n the device. eel and move d item in the ign, to brow ion is perform rcular-like in ke navigatin 2], which is p crolling techn ng device su

ment from a side

PAPER IDEN d. Drawing a em down an on is activa r reaches the of a circular m letter 'Q' in QPalm. ng with list ommonly-use ctions can be u, and therefo To efficiently we develop a motion and y a stereo c ion of the ed in the real y the users’ c environment re specifical (1) an in ote control w ction algorit depth and m consideration ducts or rese main user int 1, a portable ased by Ap w feature o used as a rot ms and contr nd an anti-a A user simp es along the c e menu then se in a list m med. nterfaces app ng documen proposed by nique that u uch as a tou e view. (b) Our TIFICATION a circle nd vice ated by e target motion the air t menu ed user e easily ore can y track a hand d depth camera. image l world olorful ts with lly, the ntuitive with list thm to motion n. earches terface. media pple in of this tational rol the aliasing ly puts circular moves menu is plied to nt. The Tomer uses an uchpad, sty Cl co VS wh Th cir vie on wh tha wh tha the Ra of co mo qu im sam Fi thr cir of Ra su tra lon ge ke tha In po an Th the ac Ne ste de On giv It sit To dim system N NUMBER (D ylus or stan lockwise m ounterclockw SR, fast and hile small an he VSR a rcumference ew that shou n a touchpad heel while s at the VSR i hen most sc an half a pa e VSR, is pr adial Scroll [ f motions as ontrolling the ore slowly, uickly, they d mproved vers me author, g.2.2(b), the ree points o rcles, and ma ffers the eye adial Scroll. uitable for aditional m ng-distance t III. P In this work, estures desig eeping reactin an notify the this section osition of us nalysis of the he workflow e region of ccording to t ext, useful f ep to further etected palm nce a pre-def ves visual an The environ is like an o tting on the s o detect the mensions, a DOUBLE-CL ndard mouse motions scr wise motions d large move nd slow mo adopts the of a circle uld scroll. In t d and mouse crolling in a is a tenable s crolling actio ge. Another oposed by G [3]. The Rad s VSR but e speed of sc they draw a draw a relate sion of Rad called Cur Curve Dial u of the traject ake response e-free param In the exp navigating methods w targets. PALM DETEC , the circular gned for QP ng while the e user after an n we introdu ser’s palm i e palm motio of the system f interest is the position features are locate the us ms are then fined motion nd sound feed nment of our ordinary scen ofa, a televis trajectories stereo camer LICK HERE T e to naviga roll the v scroll it up. ements prod ovements yie length trav to decide th the experime e is compare a document. scrolling alte ons are expe approach, w G.M. Smith e

ial Scroll use different in crolling. If u a larger circ edly smaller dial Scroll is rve Dial [4 use the angle

tory to judg es immediate meter entry t periment, th short-distan work better CTION AND T and push mo Palm. The s user is draw n entire circl uce the appro

in real-time, ns is describ m is shown i defined in and depth o extracted fo ser’s palms. adopted for n is found, th dbacks to the system is illu nario in a liv sion placing i of the user ra released by TO EDIT) < ate the docu view down

Besides, wi duce fast scro

eld slow scro veled along he distance ent, the VSR ed with the m The results ernative, espe ected to be l which is sim

t al. and call es the same d n the functio users wish to cle; to scroll circle instea s purposed b 4]. As show e formed by th ge the size ely. This tech

that absent eir work is nce targets r for bro

TRACKING

otions are the system is de wing a circle, le is accompl oach to dete , followed b bed in next se in Fig.2(b). F the image of the user’s r particle fil The trajector r motion an he QPalm int e user. ustrated in Fi ving room: in front of the r’s palms in y Videre Des 2 ument. and ith the olling, olling. g the of the R based mouse show ecially longer milar to led the design ons of scroll more ad. An by the wn in he last of the hnique in the more while owsing e basic evised rather lished. ect the by the ection. Firstly, frame s face. ltering ries of alysis. terface ig.2(a). a user e user. n three sign is

> in ca he ob co on co N th oc A us an A lo M lo ef re M fa fa B ar us pe la Si th de G de H us se ef th ar th in of F REPLACE T ntroduced in amera provi ence we can bject in the s ommercial c ne of them to olor and dep Noted that the he depth is u ccurred in di . Preprocessi Firstly, we ser’s face. O nd Meanshif Adaboost is g ocate a face Meanshift wh ocally. Comb fficient fac egion-of-inte Meanshift cou ace, Adaboos ace positions . Identifying R While an us re three parts ser's face r erforming a abeled ru. A ince r1 alway he range of efined as th Gaussian distr The length efined as the However, whe ser's palm ca equence of t ffect, we use he range is dr re sufficient he distance b nstead of usin f a pixel belo

ig. 3. The origi

THIS LINE W the system, ides disparit obtain the de cene in real-ameras to co o provide col pth images c e blue region un-defined du isparity comp ing with Fac

use Adaboo Once a face is ft [9] algorith generally mo e from an in hich simply bining of th e tracking rest for later uld fail in a st algorithm every three Region-of-In ser is sitting s in the side v r2, the regio motion, r1,

side view ill ys attaches t r1 if r2 is kn e depth of t ribution. of r1 wher arm length o en the user p an easily drift the circular e a mixture o rifting in a slo to achieve t between r1 ng a fixed va onging to r1.

inal color and s

WITH YOUR P

laying on t ty computat epth and colo time. One ca ompute dept or informatio captured by n of the dep ue to an un-putation. ce Detection ost algorithm s found, the hm is applied ore computa nput image searches a s he two appro method t computation a local search is applied to seconds. nterest and watchin view, which a on of the u

and the non lustration is o r2, we hav nown. Here the user’s fa re users' han of a normal h performs a c ts in depth du motion. To of Gaussian ow way, two this task. Ad and r2 as th

alue. Eq.1 sho

stereo images at PAPER IDEN the television tion on har or informatio an also use a th informatio on. Fig.3 sho the stereo c th image ind -matched con m [8] to loca system is tri d to track th ation demand in compari similar distri oaches leads to give a n. However, h while trac o verify the t ng television are the region user's hand n-interested r shown in Fi ve a general i the range o face, modele nd may app human in the circular motio uring even th alleviate th to model r1. o or three Gau dditionally, w he Gaussian ows the prob

t frame t. TIFICATION n. This rdware n of an pair of on, and ows the camera. dicated ndition ate the ggered he face. ding to ison to ibution s to an good in case cking a tracked n, there n of the while regions ig.2(a). idea of of r2 is ed as a pear is initial. on, the he same he drift . Since ussians we take n mean bability , w of is mi a be ob is rou str de us rep C. ide reg co po As the inp the us fra co co mo po his the va tw ob Ho mo pe Fi N NUMBER (D , where _, is f . K is the an estimate ixture at time probability elonging to btained by tru extracted to ugh shape of ructure of hu etection resul ed to update presents . Palm Tracki In this secti entify and gion-on-inter ondensation [ ossible interp ssume that th e observatio put image, i e posterior d er’s palm is , 1, , is to e ame . The most i ompute the ombine three otion inform ositions. Ope s/her hand a e operation, alues than the wo ends of

bservations, owever in a ore than one eople who ma

(a)Probab ig. 4. Use the d

DOUBLE-CL , the distance number of G e of the weig e t. After app map tells user's hand uncating pix o define the f a hand (see uman body i lt more reliab the mean of ing Using Pa ion, the part d track th

rest report [7] algorithm pretations is r he state of tar on, the regio s referred as ensity | relying on. , with express the important st posterior pr e observation mation of the erating with nd then mak the palm pa e rest and is u the hand we can loc a living room e people the ay interfere w bility Map depth informatio the regio LICK HERE T , , e of depth be Gaussian dis ght of the it plying Eq.1 to the probab , namely xels with low

region of in e Fig.4). This into consider ble. The final f the mixture article Filter ticle filterin he user’s ted in pre m, the probabi represented b rget object is on-of-interes s , we woul indicating A set of par the weight posterior d tep in partic robability o ns including e user’s hand h QPalm, th kes circles in art of the arm usually posit

shape. Com cate a palm m scenario, t ere. We shou with the oper

(b) Regio on to obtain the on-of-interest. TO EDIT) < , , , etween x and tributions an th Gaussian o all pixels, w bility of a . The imag w probability nterest and g s method tak ration, makin l detecting re of Gaussian ring g is conduc palm in evious step ility distribut by a set of pa s modeled as st reported ld like to co the probabil rticles, denot s of the pa ensity | cle filtering of a particle g depth, pose d to evaluate he user first n the air. Bas m has larger ioned in the mbining the position re there are po uld consider rating user. F on-Of-Interest e probability ma 3 |(1) d mean nd _, in the we get pixel ge , y in , give a kes the ng the esult is which cted to n the p. In tion of articles. s x and in the mpute ity the ted by articles at is to e. We e, and e palm t rises sed on depth one of e two liably. ossibly r other For the ap and

> pe ca th ca al ev pr ha pa ce w vi va pr R co pe on Fi th w fi co Fi sh in Th el pa be th U ro w cl ci th ve Fi w an REPLACE T eople stand o an effectively he user, dept ase, we utili lleviate the valuating a p 1) Depth I robability ma and position alm position , , as entered on t window size iew. Since t alues than t refer the part 2) Shape In ROI are take

onsidering t erforming ci n the top of ig.5(a), a goo he connected with intersecti ltered out si omponent an ig.5(b)). Als hown in F ntersects with his informat liminating pa 3) Motion I article evalu etween the lo he motion d Using optical obust to the i within a wind lusters, , means the it his cluster an ector _, w

ig. 5. Use the i wrist or forearm. nd the leftest on THIS LINE W or walk behi y ignore them h informatio ize the mot

kind of int particle is list Information: ap Pt indicati

ned. For each n, say , , s the average

the position is set as a r the palm par the rest whi

ticle with hig nformation: en part in th the features rcular and p f the connec od particle’s d component ions only at t ince it is in nd cannot b o, if a partic ig.5(c), the h the two o tion offers a article locate Information: uation by ocation of a p irection of t l flow make interferences dow of a pa , , , h cluster, ni i nd is the m which formed (a) information of . The right two ne is a particle a

WITH YOUR P

ind the user, m. But for th on does not h ion informa terference. T ted as follow From last ing the proba h particle re , we comput e probability reported by regular palm rt of the arm ile operating gher depth sc The connect he evaluation s of a han push motions cted compon window has at the botto the upper ed n the bottom be a candida cle is located connected opposite edge an excellent d on wrist or The optical considering palm in the p the current es the detec s in the envir article are fi , , , s the number mean vector o d by lt-1, the p (b) shape to reject images are par at good position PAPER IDEN depth inform he people sit n help much. T ation of a pa The techniq ws. step, we h ability of the eporting a gu te the depth y within a w y the particl m size in the m has larger g with QPal core. ted compone n step as w nd’s shape. s, a palm is l nent. As sho s intersection m edge. A p ge of its win m of the con ate of a palm d on the fore component es of the w rejection fil r forearm. l flow is app the relati previous fram frame (see ction results ronment. The rst split to s , , , r of optical fl of this cluste previous loca (c)

the particle loc rticles on bad lo n. TIFICATION mation next to To this alm to que for have a e user’s uess of score, window e. The image r depth lm, we ents in well by While located own in ns with particle ndow is nnected m (see arm as t then indow. lter for plied to ionship me and Fig.6). s more e flows several where lows in er. The ation of a po ha the in ob dif inf ,w an co ap In pr tra for kin pa us us em A. sy en fir the cir tha y t-An Fi pr cated on ocations N NUMBER (D palm, and th otential direc ave a positive e particle. Th Eq.2. corr By applying bjects near th fferent direc formation, th , , s where M is a nd c2 are the w ombination o pplying Eq.3, next frame, ocess then it In previous acking user’s rmed by the nds of trajec alm and their

ed to detect er is drawin mployed to d Detection of A prediction ystem should ntire circle. F rst estimated e order of the rcle or just a at we have t 1),… , (xt-k, yt n estimated g. 6. Use the o evious location DOUBLE-CL he location ction of this e correlation he correlatio , 1 N g Eq.2 to par e user can be ction with he evaluation · _, , 0 shape‐detect mean filter weights whic of the mean , the goodnes , the best n terates with n IV. MOTI s section, th s palm is dem location of a ctories record r variations t if it expres ng a circle, etermine if a f Circular M n of circular react during For detecting based on th e points on th a causal moti the trajectory t-k)}, and k is circle is th optical flow to n of the palm an LICK HERE T of the parti s particle. T n with the op on estimated · · rticle evaluat e filtered out the user. B n of a particle · ction, using ω as it ch make conf n filter and ss of all parti particles are new observat ION ANALYS he method f monstrated; h a user’s palm ded are the 2 in depth. Th ses a circle, and the traj a push motion Motion r motion is r g a motion, r g the circular e trajectory. he trajectory ion made by y, named as the number o hen compute examine the co nd the particles. TO EDIT) <

cle px,y show

This vector s ptical flows n function is s · tion, some m if they move By combinin e is compute , , , 1 (3 ts window s f(px,y) to be a correlation. cles is determ e selected an tions. IS for detectin hence the traj m is extracted 2D positions he 2D traject which mea ectory in de n is occurred required sin rather than af r motion, a b Then, we ch truly describ y the user. A Tt = {(xt, yt) of recorded p ed by solvin orrelation betwe 4 ws the should nearby shown (2) moving e in the ng all ed as: , , 3) ize. c1 a linear After mined. nd this g and ectory d. Two of the tory is ans the epth is d. ce the fter an best-fit heck if bes the ssume ), (xt-1, points. ng the een the

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 5 over-constrained system [5]. 2x 2y 1 2x 2y 1 2x 2y 1 c c c x y x y x y (4)

Eq.4 is a basic form of Ax=B where x can be solved by the least squares method. It minimizes the errors between observed data and the predictive model. In our system, the predictive model is a circle and the best-fit circle is then assumed to be centered at (c1, c2) with radius

.

When the best-fit circle is estimated, the next step is to determine if the trajectory truly expresses this circle. The first and intuitive information is that the mean square error between an observed point on trajectory and its nearest point on the predicted circle should be small. Moreover, while an user draws a circle, the palm location should be ordered in a clockwise or counterclockwise way. Hence, the angle which is formed by the palm location, the center of best-fit circle and the horizontal line should increase or decrease progressively (see Fig. 7(a)). This method effectively eliminates noises and casual motions. At last, a circle with too large or too small radius is removed since in most time it is formed by a straight line or a motionless gesture.

B. Push Motion Detection

A push motion is quite different from the circular motion since the system only needs to react one time after it detects a legal motion. There are several related works aim at recognizing a pre-defined gesture and most of them adopt the approach which trains target motion at first. In our situation, a more straightforward way can be used to solve this problem because a push motion has clear changes in depth, as in Fig. 7(b). A circular motion is often performed parallel to the image plane of a camera, while a push motion is perpendicular to the image plane. Besides, a push motion must appear after a circular

motion. As a result, we detect a push motion by checking if there is a continuous increase in palm's depth while the user is drawing a circle. If a push motion is detected, there is one second delay for accepting another push gesture. This technique prevents producing multiple choosing events during one choosing motion.

V. EXPERIMENTSANDUSERSTUDY The proposed method is applied to two videos which simulate the environment in a living room. The first video has 476 frames while the second video has 325 frames, and both with a 320 x 240 resolution. It runs on a PC with Intel Pentium 3.40G Hz CPU plus 2.49GB RAM and all tasks were implemented by C++ code. The stereo camera is equipped with two 2.8 mm lens on it. The distance between camera and users is about 2 m and the frame rate is 16 to 18 frames per second (fps). After applying our algorithm, the frame rate descends to 10 to 12 fps. Some results are shown in Fig.8.

In video 1, a person passes by the user at frame 20 and 298. There are also multiple passers appeared at frame 303 and 416. The detecting result is not affected by the passers since the depth information is used in this system. Additionally, in video 2 a person tries to influence the user by making some movements nearby the user at frame 84, 172, 204 and 287. The detecting results show that the system is robust to the interference from other people.

User Study: In order to check the practicability of the

interface QPalm, a series of experiments and comparisons were taken. In these experiments, participants were asked to select a specified item in a list menu with two kinds of selection techniques: the EyeToy-like selection technique and QPalm. In EyeToy's design, a camera is set in front of the users and they can see their own image on the display as a feedback. A previous, a next and a choosing button are placed on the display and users simply touch and wave in the area of these buttons to make a command to the menu. An experiment contains three blocks and each corresponds to those two different techniques. What participants see is a simple list menu and the items are showed vertically (see Fig.1). The target item's number is

(a) Estimated Circle (b) Choosing Fig. 7. (a) A point on the trajectory forms an angle θ with the center of the estimated circle and the horizontal line. It should increase or decrease in a progressive way if the user is drawing a circle. (b) The changes in depth while doing a choosing action

Fig. 8. (a) Some people passed by the user. (b) A person sit by the user and tried to interfere the user.

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 6

generated randomly in the range of menu size. Before one block starts, we teach users the technique allowed in next block and let users scroll on the list menu until they satisfied. Besides, we put three practical trials in the beginning of a block to ensure the participant getting used to the specified technique. In a block, we ask users to repeat five tasks in different menu sizes (20, 50 and 120) hence there are 2 * (3 + 5 * 3) = 36 trials in an experiment. A total of 8 participants took part in the study and comprised of 4 males and 4 females. Half of them had the experience of playing in Sony's EyeToy game while all of them were skilled in using computer. Another, there were no users familiar with the use of QPalm.

Completion Time: We reported the result of

completion time and showed it in Fig. 9(a). Note that we can directly see a significant variation while the total item number is increased. Also, the EyeToy’s technique performs a longer completion time than the other two techniques in all these trials. While the total item number is 20, there are no obvious differences in these two techniques. However, if the total number increased to 120 (35.077s), the EyeToy’s method showed a worse result than QPalm (23.933s). Users responded that they can quickly scroll to their concerned area by QPalm techniques but cannot by an EyeToy-like interface.

Accuracy: In these tests EyeToy's technique shows a

better result since it allows user to choose the desired target more slowly. There is a worse outcome (83.76\%) in QPalm while total item number is 20. It is because that we let users judge if they were ready for the experiment themselves. Users may not actually be proficient in the selection technique but enter a true experiment. The results of QPalm technique after the first session are obviously better when total items number increases.

Questionnaire Responses: Most participants reported

that a gesture-base method helps them jump to their concerned region faster. In order to examine the relationship between total item number and these two techniques, we asked user to subjectively choose which one they preferred under different total menu size (20, 50 and 120). According to Questionnaire responses, users preferred the technique. However, if the number of items

is small, users considered the EyeToy-like technique more easily to perform.

VI. CONCLUSION

The proposed a novel interaction technique, QPalm, which allows the user to control the media via a list menu shown on a distant display by drawing circles in the air with one hand. QPalm provides two basic functions, circular and push motions, for the user by recognizing the user's palm trajectory in three-dimensions. The circular motion provides fluidity in scrolling a menu up and down, while push motion is intuitive when the user decided to choose an item during a circular motion. In the future, we would like to extend QPalm in a multiple-user scenario. While people are watching television, it is usual that more than one person are trying to control the menu. It is impossible that the system gives the control of menu to all users at a time. A possible solution is to set two cameras in the environment, one with wide-angle lens and the other with pan-tilt-zoom function. When the system detects a motion asking for the control in the wide-angle camera, the pan-tilt-zoom camera then focuses on the interest area.

ACKNOWLEDGMENT

This work was supported in part by the National Science Council, Taiwan, under grants NSC 96-2752-E-002-007-PAE, and by the Excellent Research Projects of National Taiwan University, under grant 95R0062-AE00-02.

REFERENCES

[1] William T. F., Craig D. W.: Television Control by Hand Gestures. IEEE Intl. Wkshp. on Automatic Face and Gesture Recognition. (1995) 179-183

[2] Tomer M., John F. H.: Navigating documents with the virtual scroll ring. Symposium on User Interface Software and Technology. (2004) 57-60 [3] Smith G. and Schraefel M. C.: The Radial Scroll Tool: Scrolling Support

for Stylus-or-Touch-Based Document Navigation. Proceedings of the 17th

annual ACM symposium on User interface software and technology (2004) 1:53-56

[4] Grham, S., Schraefel M. C., Patrick B.: Curve dial: eyes-free parameter entry for GUIs. Conference on Human Factors in Computing Systems, CHI. (2005) 1146-1147

[5] James A., Kevin N.: Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes. Proceedings on User interface software and technology. (2000) 73-80

[6] Ivan L., Tony L.: Tracking of multi-state hand models using particle filtering and a hierarchy of multi-scale image features. Proceedings of the Third International Conference on Scale-Space and Morphology in Computer Vision. (2001) 63-74

[7] Michael I., Andrew B.: CONDENSATION conditional density propagation for visual tracking. International Journal of Computer Vision. (1998) 29,1:5-28.

[8] Rainer L. and Jochen M.: An Extended Set of Haar-like Features for Rapid Object Detection. IEEE ICIP. (2002) 1:900-903

[9] Dorin C., Visvanathan R., Peter M.: Real-Time Tracking of Non-Rigid Objects using Mean Shift. Computer Vision and Pattern Recognition. (2000) 2:142-149

(a) Average completion time of different selection techniques. (b) Accuracy rate of different selection techniques