Evaluation

In order to determine whether the dynamic feedforward and feedback guiding system indeed help the users to perform better gaze gestures, we conducted a within-subjects study to compare GazeBeacon with Crib-Sheet guide on completion time, recognition rate, and selection accuracy.

4.2.1 Task and Procedure

Figure 4.6: The interface of GazeBeacon and the gesture-command pairs used in the evaluation study.

We created a simple five-item gesture-command shortcut menu, the interface, and the gesture set is shown in Figure 4.6. Each of the gesture was mapped to a common command of word processing applications including ’Copy’, ’Paste’, ’Undo’, ’Save’ and

’Select All’. The gestures to execute them also covered up possible combinations of straight lines and curved lines.

we instructed the participants to perform gaze gesture selections on the shortcut menu with the guidance of GazeBeacon or with the traditional Crib-Sheet. When the study began, a task command would pop up at the center, the participants should follow the

CHAPTER 4. GAZEBEACON 32

assignment and the results were recorded.

The order of the guiding techniques was counterbalanced, ensuring the training effect for subsequent conditions were avoided. Each gesture was repeated for ten times and ran-domly ordered, overall, each participant received 5 (gestures) × 10 (repeats) × 2 (guidance techniques) = 100 trials in the study.

4.2.2 Participants

Eight participants (all females) with novice-level eye tracking experience were re-cruited, ranging from 22 to 24 (mean age = 22.875, s = 0.83). All of them have normal or correct-to-normal vision.

CHAPTER 4. GAZEBEACON 33

4.2.3 Result and Discussion

Figure 4.7: The mean completion time and mean recognition rate of Crib-Sheet and GazeBeacon.

The result of the experiment is shown in Figure 4.7. The completion time of Gaze-Beacon is significantly longer than Crib-Sheet (p<0.001), while the recognition rate of the former is significantly higher the latter (p<0.001). It might due to the trade-off be-tween the correct rate and performance of menu execution. With the dynamic visual guide opening, the users tended to be more cautious on the gesture path, resulted in the lower performance but the higher recognition rate.

CHAPTER 4. GAZEBEACON 34

Figure 4.8: The mean completion time and error rate of Crib-Sheet and GazeBeacon in block 1 and block 2.

Figure 4.9: The error rate of Crib-Sheet and GazeBeacon.

Table 4.1: Confusion matrix by gesture for different guiding techniques. (a) Crib-Sheet.

(b) GazeBeacon.

CHAPTER 4. GAZEBEACON 35

We used chi-square tests to analyze the result of the error rates of the two techniques.

The error rate of GazeBeacon is significantly lower than the Crib-Sheet guide (Figure 4.9) (p<0.001), inferring that GazeBeacon improves the correct rate of gaze gestures.

Enhancing the Features of Gestures

For deeper understanding, Table 4.1 shows the confusion matrix by gesture. The error rate of ’Select All’ and ’Copy’ has no difference between the two techniques, this might because the two were both single-element gestures, they were more likely to be interpreted.

However, ’Undo’ tended to be misinterpreted as ’Copy’ if the first corner element was not performed well. ’Save’ tended to be misinterpreted as ’Select All’ if the second arc element was not performed well. The error rate of the complex gestures were significantly different between the two techniques (p<0.001), especially the one of ’Paste’, combining corner and arc make it much harder to perform and be correctly recognized.

Figure 4.10: GazeBeacon enhanced and emphasized the essential features of the gestures.

GazeBeacon enhanced and emphasized the essential features of the gestures, decreas-ing the error rate of the recognition. We believe that the results suggest that GazeBeacon takes more advantage on improving the selection accuracy of complex graffiti gesture input.

Chapter 5

Conclusion and Future Work

5.1 Conclusion

In this paper, we proposed GazeBeacon, a gradual visual guiding system designed for gaze gesture interaction. It continuously provides real-time feedback and feedforward graphical cues under gaze points during the progress of the interaction by combining the concept of smooth pursuit and dynamic guide. We found two issues in our pilot study: miscalculation and misestimation. First, we mitigated the miscalculation problem by adding smoothing filters on gaze points and resampling the path before length calcu-lation. Second, we solved the misestimation problem by adding a focus point at the end of the guidance path, prompting the users to put their focus on the continuously tracking object. A user study was conducted to further verify the influence of different guidance techniques on various gesture primitives. After solving the problems, we describe the details of the design and the implementation of GazeBeacon. Then we further evaluated the completion time, the recognition rate and the selection accuracy of GazeBeacon, com-pared with the traditional crib-sheet guide. The result shows that although GazeBeacon

CHAPTER 5. CONCLUSION AND FUTURE WORK 37

makes users spent more time on execution, it significantly improves the accuracy of menu selections on gaze-gesture interfaces.

5.2 Future Work

Possible future works can be organized as follows.

5.2.1 Continuous v.s. Discrete Gaze Gestures

One of the common approaches of gaze gestures is saccadic eye movements. By Sim-plifying the gestures and mapping them onto existing encoding system such as EdgeWrite, the gestures are composed of only straight lines, and take benefits from the nature of hu-man eyes being skilled in saccadic straight direction movements.

However, the straight-line-formed gestures should be predefined by each different sys-tems and lead to inconsistent user experiences for the users switching among applications.

On the other hand, if the pattern was not defined by the system yet, the users might be confused about the actual gesture of the commands.

Although the efficiency of GazeBeacon in this study is lower, the continuous feed-forward and feedback of it can be used on every single-stroke gaze gestures, including graffiti-based ones. Since the users remember the paths not only the dots, GazeBeacon can keep inheriting the benefits of gesture-based interfaces like various semantic mean-ings.

CHAPTER 5. CONCLUSION AND FUTURE WORK 38

5.2.2 Modifications on the Recognizer

In the implementation of GazeBeacon, we used a well-known stroke recognizer called

$1. It was widely applied on many gesture-based interfaces. However, due to the nature of human eyes, the recognizer needs further modifications for gaze-controlled interactions, the shape matching method used in it might be extended based on the features of the gaze gestures.

Figure 5.1: The recognizer can pre-filtering the guidances of the gesture input by its initial movement.

For instance, the rotated angle should be limited in a specific range so that the A shape may not be confused with C shape since their direction of the initial move is totally different. By the modifications of the used recognizer, it might improve the accuracy of the response from the guidance system to the users’ input.

5.2.3 General v.s. Specific Guidance Design

In the guidance design of GazeBeacon, we proposed a method could be directly ap-plied to general usage cases. However, we noticed that the behavior of the users on various gesture elements was quite different from each other in the study. For example, the proper value of the waiting threshold of the focus point on squares and rectangles are not the same as the one on triangles, since acute angles need less emphasize. And

CHAPTER 5. CONCLUSION AND FUTURE WORK 39

for horizontal straight lines, the users tends to move a little bit higher than the intended gestures.

On the contrary, The offsets within one individual user are quite similar, they were in-clined to repeat the offset mistakes previously made by themselves. Therefore, if we want to extend the guiding ability based on the general design, we can further discuss some specifications. On gesture elements, we might adjust the trajectory of the guidance based on the mentioned findings, make it not fully matched to the gesture template. On individ-ual users, we can dynamically tune the guidance by neutralizing the offset to gradindivid-ually redirect the users back to the intended one.

Appendix A

Mean Paths of the Gesture Input

Figure A.1: The mean paths of the gestures input with different guiding techniques in line section.

APPENDIX A. MEAN PATHS OF THE GESTURE INPUT 41

Figure A.2: The mean paths of the gestures input with different guiding techniques in corner section.

APPENDIX A. MEAN PATHS OF THE GESTURE INPUT 42

Figure A.3: The mean paths of the gestures input with different guiding techniques in arc section.

Bibliography

[1] F. Anderson and W. F. Bischof. Learning and performance with gesture guides. In Proceed-ings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1109–1118.

ACM, 2013.

[2] G. Barnes. Rapid learning of pursuit target motion trajectories revealed by responses to randomized transient sinusoids. Journal of Eye Movement Research, 5(3):4, 2012.

[3] O. Bau and W. E. Mackay. Octopocus: a dynamic guide for learning gesture-based command sets. In Proceedings of the 21st annual ACM symposium on User interface software and technology, pages 37–46. ACM, 2008.

[4] M. Bennett, K. McCarthy, S. O’Modhrain, and B. Smyth. Simpleflow: enhancing gestural interaction with gesture prediction, abbreviation and autocompletion. In Human-Computer Interaction–INTERACT 2011, pages 591–608. Springer, 2011.

[5] D. S. Best and A. T. Duchowski. A rotary dial for gaze-based pin entry. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, pages 69–76. ACM, 2016.

[6] A. Bragdon, A. Uguray, D. Wigdor, S. Anagnostopoulos, R. Zeleznik, and R. Feman. Ges-ture play: motivating online gesGes-ture learning with fun, positive reinforcement and physical metaphors. In ACM international conference on interactive tabletops and surfaces, pages 39–48. ACM, 2010.

[7] A. Bragdon, R. Zeleznik, B. Williamson, T. Miller, and J. J. LaViola Jr. Gesturebar: im-proving the approachability of gesture-based interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2269–2278. ACM, 2009.

[8] X. Cao and S. Zhai. Modeling human performance of pen stroke gestures. In Proceedings of

43

BIBLIOGRAPHY 44

the SIGCHI conference on Human factors in computing systems, pages 1495–1504. ACM, 2007.

[9] G. Casiez, N. Roussel, and D. Vogel. 1C filter: a simple speed-based low-pass filter for noisy input in interactive systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2527–2530. ACM, 2012.

[10] W. Delamare, C. Coutrix, and L. Nigay. Designing guiding systems for gesture-based in-teraction. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pages 44–53. ACM, 2015.

[11] H. Drewes and A. Schmidt. Interacting with the computer using gaze gestures. In Human-Computer Interaction–INTERACT 2007, pages 475–488. Springer, 2007.

[12] D. Freeman, H. Benko, M. R. Morris, and D. Wigdor. Shadowguides: visualizations for in-situ learning of multi-touch and whole-hand gestures. In Proceedings of the ACM Inter-national Conference on Interactive Tabletops and Surfaces, pages 165–172. ACM, 2009.

[13] E. Ghomi, S. Huot, O. Bau, M. Beaudouin-Lafon, and W. E. Mackay. Arp`ege: Learning multitouch chord gestures vocabularies. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces, pages 209–218. ACM, 2013.

[14] J. P. Hansen, H. Lund, F. Biermann, E. Møllenbach, S. Sztuk, and J. S. Agustin. Wrist-worn pervasive gaze interaction. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, pages 57–64. ACM, 2016.

[15] H. Heikkil¨a and K.-J. R¨aih¨a. Simple gaze gestures and the closure of the eyes as an interac-tion technique. In Proceedings of the symposium on eye tracking research and applicainterac-tions, pages 147–154. ACM, 2012.

[16] A. Huckauf and M. H. Urbina. Gazing with peyes: towards a universal input for various ap-plications. In Proceedings of the 2008 symposium on Eye tracking research & applications, pages 51–54. ACM, 2008.

[17] P. Isokoski. Text input methods for eye trackers using off-screen targets. In Proceedings of the 2000 symposium on Eye tracking research & applications, pages 15–21. ACM, 2000.

[18] H. Istance, A. Hyrskykari, L. Immonen, S. Mansikkamaa, and S. Vickers. Designing gaze

BIBLIOGRAPHY 45

gestures for gaming: an investigation of performance. In Proceedings of the 2010 Sympo-sium on Eye-Tracking Research & Applications, pages 323–330. ACM, 2010.

[19] R. J. Jacob. What you look at is what you get: eye movement-based interaction techniques.

In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 11–18. ACM, 1990.

[20] A. Kurauchi, W. Feng, A. Joshi, C. Morimoto, and M. Betke. Eyeswipe: Dwell-free text entry using gaze paths. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 1952–1956. ACM, 2016.

[21] E. Mollenbach, J. P. Hansen, M. Lillholm, and A. G. Gale. Single stroke gaze gestures. In CHI’09 Extended Abstracts on Human Factors in Computing Systems, pages 4555–4560.

ACM, 2009.

[22] Q. Roy, S. Malacria, Y. Guiard, ´E. Lecolinet, and J. Eagan. Augmented letters: mnemonic gesture-based shortcuts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2325–2328. ACM, 2013.

[23] C. Sch¨onauer, K. Fukushi, A. Olwal, H. Kaufmann, and R. Raskar. Multimodal motion guidance: techniques for adaptive and dynamic feedback. In Proceedings of the 14th ACM international conference on Multimodal interaction, pages 133–140. ACM, 2012.

[24] O. Tuisku, P. Majaranta, P. Isokoski, and K.-J. R¨aih¨a. Now dasher! dash away!: longitudinal study of fast text entry by eye gaze. In Proceedings of the 2008 symposium on Eye tracking research & applications, pages 19–26. ACM, 2008.

[25] M. H. Urbina, M. Lorenz, and A. Huckauf. Pies with eyes: the limits of hierarchical pie menus in gaze control. In Proceedings of the 2010 Symposium on Eye-Tracking Research

& Applications, pages 93–96. ACM, 2010.

[26] D. Vanacken, A. Demeure, K. Luyten, and K. Coninx. Ghosts in the interface: Meta-user interface visualizations as guides for multi-touch interaction. In Horizontal Interactive Human Computer Systems, 2008. TABLETOP 2008. 3rd IEEE International Workshop on, pages 81–84. IEEE, 2008.

[27] R. Vertegaal et al. Attentive user interfaces. Communications of the ACM, 46(3):30–33, 2003.

BIBLIOGRAPHY 46

[28] M. Vidal, A. Bulling, and H. Gellersen. Pursuits: spontaneous interaction with displays based on smooth pursuit eye movement and moving targets. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, pages 439–

448. ACM, 2013.

[29] D. J. Ward, A. F. Blackwell, and D. J. MacKay. Dasher—a data entry interface using con-tinuous gestures and language models. In Proceedings of the 13th annual ACM symposium on User interface software and technology, pages 129–137. ACM, 2000.

[30] J. O. Wobbrock, B. A. Myers, and J. A. Kembel. Edgewrite: a stylus-based text entry method designed for high accuracy and stability of motion. In Proceedings of the 16th annual ACM symposium on User interface software and technology, pages 61–70. ACM, 2003.

[31] J. O. Wobbrock, J. Rubinstein, M. W. Sawyer, and A. T. Duchowski. Longitudinal evaluation of discrete consecutive gaze gestures for text entry. In Proceedings of the 2008 symposium on Eye tracking research & applications, pages 11–18. ACM, 2008.