行政院國家科學委員會專題研究計畫 成果報告
遠距點選之動作元件設計及國畫軟體控制系統開發 研究成果報告(精簡版)
計 畫 類 別 ： 個別型
計 畫 編 號 ： NSC 98-2221-E-011-057-
執 行 期 間 ： 98 年 08 月 01 日至 99 年 07 月 31 日 執 行 單 位 ： 國立臺灣科技大學工業管理系
計 畫 主 持 人 ： 李永輝
計畫參與人員： 碩士班研究生-兼任助理人員：蔡雪岑 碩士班研究生-兼任助理人員：陳宜琳 碩士班研究生-兼任助理人員：鍾任博 博士班研究生-兼任助理人員：吳淑楷
報 告 附 件 ： 出席國際會議研究心得報告及發表論文
公 開 資 訊 ： 本計畫可公開查詢
中 華 民 國 99 年 07 月 28 日
行政院國家科學委員會專題研究計畫成果報告
遠距點選之動作元件設計及國畫軟體控制系統開發 Prototype development and usability study of a remote input
device for software of Chinese-painting 計畫編號：NSC 98-2221-E-011-057 執行期間：98 年 08 月 01 日至 99 年 07 月 31 日 主持人：李永輝 國立台灣科技大學 工業管理系
計畫參與人員：吳淑楷、黃美貞、劉燕萍、陳建中 工業管理系
本計畫在延續前一國科會計畫”遠距點選輸入之手勢互動控制系統開發(NSC 96-2628-E-011-007-MY2)，該計畫發展以影像處理擷取技術建立遠距點選輸入之 手勢控制系統(Finger Trajectory Tracking System, FTTS )，提供一個在免穿戴的 條件下，在有效工作區中，利用三維指向手勢與軌跡，遙控的輸入介面。97 年 度計畫所發展FTTS，建置在繪製”毛筆及國畫”的軟體環境中，並發展具FTTS 特 色的控制介面。本計畫的工作包括：
1. 發展人性化的FTTS控制介面: 定義能呈現動作的方向性(x, y, z)與力道(速 度變化)特色的手勢元件的母集合、
2. 繪製”毛筆及國畫”的軟體系統：透過手勢辨識應用在繪製”毛筆及國畫”，其 發想為透過手勢的速度，展現”毛筆及國畫”中筆勢的勁道以及發揚”毛筆及 國畫”力與美的精神。
3. 分析該系統的使用性: 徵選20 位受試者，指派三種不同工作，自變數為輸 入設備及功能操作方式；依變數為完成時間，主觀評比適用性。
本計畫除了具輸入資訊的創新性外，還兼具中華文化的應用意涵。
關鍵字：毛筆及國畫、手勢辨識、遠距輸入控制、使用性分析
I. INTRODUCTION
To take advantage of the unique features of advanced technologies such as ubiquitous
computing, smart places, and augmented or virtual reality environments, applications must
be developed with appropriate user interfaces. As an alternative to traditional input devices,
researchers are exploring the possibility of adopting goal-directed pointing movements as an
input function to various appliances of computer games, interactive computer graphics, and
remote control for home appliances in a smart environment. Pointing is a movement of the
hand/arm towards a specific object, location, and/or a direction. Among the set of gestures performed by humans when communicated with each other and/or with machines, pointing movement has the most spatial compatibility.
In recognition pointing movements, the detections of the occurrence of the
finger-hand-arm movements and the pointing directions have to be addressed (Kai, et al., 2003). Most of pointing movement system recognition forearm orientation only. Our previous study of remote pointing accuracy in a distance of 3 meters showed that pointing accuracy to a target size of 7.5 cm in radius with no visual cue, the hitting rate was only 66.0% and the spread ranges was 7.02±5.30cm (Lee, et al., 2008). When pointing an object, the eye, the finger, and the object should be collinear. It was hypothesized in this study that pointing accuracy can be improved by the inclusion of the hand and head orientation into the determination of pointing location, (Lee, et al., 2001). Figure 1 showed the environment of this remote interaction, where hand and head orientation was recognize, and the direction vector was determined according to the extended line of the hand and head vector. What unknown is the accuracy of this goal-directed pointing movement.
Figure 1: Example of remote interaction with electronic systems.
The objective of this study is to refine a remote goal-directed pointing tracking system (GPTS) and to apply the GPTS for Chinese painting. In the study, a software for Chinese calligraphy was developed based on GPTS. There is a painting and an editing interface. All comments in the software were activated either by recognizing a natural hand posture or by an activation of a pull-down menu. In addition, a usability test was completed to justify the quality of the GPTS.
II. GOAL-DIRECTED POINTING TRACKING SYSTEM
In the study, we combine stereoscopic range information and skin-color classification to achieve a robust tracking performance. The setup consists of two fixed-baseline cameras connected to PC.
3-D Calibration
Based on data from a 2 camera system, direct linear transformation (DLT) was used to obtain 3D coordinates for the system. Measurements of the focus, positions, rotation angles, and distance parameters of the two cameras were used as inputs for the calibration.
2-D Hand/Head Area Detection
GPTS uses skin detection and background subtraction techniques to isolate the image of hand and head on each frame, which is then used for direction calculation. The skin detection algorithm is a modified version of that of McKenna and Morrison (2004).
Areas of hand and head were detected by identifying ellipse cylinder skin areas in the
ycbcr color space of the images from two cameras. Only pixels of skin-color-liked are
collected to a coordinate sheet. The most memory-saving format is to use a binary
dataset. Erosion algorithm was used to filter out the noise. Finally, a K-means
clustering mechanism (K=3) was used to cluster the coordinates sheet into 3 skin-area
sets (2-D positions of the head and 2 hands). These operations are applying onto the
both 2 images in the initial state.
2-D Hand/Head Position Tracking
It is not efficient to scan the whole stereo-image, after obtaining the initial 2-D hand and head coordinates, method of kernel based object tracking (Comaniciu et al., 2003) was adopted. When color histogram that describe the object population, a comparisons of the current and the previous target populations were conducted, update by gradient information until the correlation coefficient is large enough and then the scan stopped.
Since the moving trajectories of hand and head are nearby-differentiable, the method works efficiently in this application.
3-D Goal-directed pointing movement
The trajectories of pointing movement were smoothed by Kalman filter (Keskin, et al., 2003). Combine the paired 2-D hand and head positions from each image by DLT parameters, the 3-D coordinates of the goal directing pointing were calculated.
Sequences of hand and head information are then used for goal-directed pointing controls. The system is capable of recognizing gestures at a speed of 20 Hz. It was then the velocity and the accelerations of the pointing movements were calculated.
SYSTEM APPLICATON
To verify the advantage of GPTS, we built a remote controlled interface for Chinese calligraphy using free-hand gestures as input device. The comments were achieved by continuous monitoring the changes of the 3-D movements of the hand. Figure 2 showed the painting and an editing interface of the software for Chinese calligraphy
Figure 2: System configuration and user interfaces for the application of GPTS to Chinese calligraphy
In this application, comments in the editing interface were defined as following (from left to right):
As Input Device Calligraphy interface
Adjust Sensitivity For Using
Hand (x, y, z) Cursor
(x, y, z)
Go Painting
Back To Main Menu
Figure 3: comments in the editing interface 1. file opening,
2. file management, 3. read,
4. print, 5. pre-view, 6. supports,
7. software information, 8. close.
Comments in the painting interface as was illustrated in Figure 4 were defines as following (from top to the bottom):
1. selection of the size of the paint, 2. selection of the brush types for the pen,
3. selection of the hand gesture for gripping the pen, 4. selection of the color of the inks.
、
Figure 4: comments in the painting interface
One problem encounter in hand gesture interface is that it is difficult to stabilize the
user’s hand at a fixed position. It is therefore hard to move a pointer to follow the hand’s
position and to select or click at the intended location accurately. To solve this problem, a
crossing bar motion that occurs twice during a short time interval is made to trigger a
commend followed the design of Nakamura, et al., (2008). The hand movement of
crossing is recognized by the camera so that is can be used to press a button or to select a
menu item in the graphic user interface (UGI). Double4-corssing is a local finger gesture
that move a finger up-and-down, left-and-right; this action is simple enough that the user
does not need to train the system. Figure 5 illustrated the hand movement.
Figure 5: Hand movement of double-crossing for the confirmation of the selection Calligraphy and Chinese painting are significant traditional arts of China, and its representation of ink and water, as well as the abstract artistic conception. As the positions, orientations, and pressures of the hand movement, strokes in Chinese painting are simulated on a virtual space. The system is composed of three parts. First, input devices and user interface help user to paint intuitively. The free-hand movement contained information of orientations, brush simulations become more realistic. Secondly, the speed of hand movement was used to generate the dynamics, accordingly, it can produce the effects of brush forking. Last, the ink-water transfer model is responsible for ink and water transfer between brush and paper. Multi-layer structure of ink and water is adopted in our system, and this model can simulate Chinese painting strokes by combining algorithms of transfer, fading and spreading of ink and water. Please see Figure 6 for an illustration of the ink transformation model. Figure 7 showed some applications of the system for Chinese calligraphy
Figure 6: Multi-layer structure of ink to simulate Chinese painting strokes
Figure 7: Applications of GPTS for Chinese calligraphy
CONCLUSION
In this paper, GPTS was demonstrated to recognize movement of free-hand in 3D spaces in a distance of 3 meters. The system was designed to interact with a Chinese
0
Pen Pen
Pen
6
calligraphic interface. Other intended future work includes (a) a strong recognition engine using recurrent neural network model or hidden Markov model, (b) a more complete elements of gesture will be defined that movements in 3D spaces, moving speeds, and the strength of movement are more meaningful, and (c) interaction with a larger display at a longer distance.
Acknowledgements
This study is supported by a grant from the National Science Council, R.O.C. (Project No.
NSC 98-2221-E-011-057). The authors wish to acknowledge this financial support.
References
Comaniciu, D., Ramesh, V., and Meer, P. (2003), Kernel-based object tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 564-575.
Kai, N., Seemann, E., and Stiefelhagen, R. (2004), 3D-tracking of head and hands for pointing gesture recognition in a human-robot interaction Scenario, Automatic Face and Gesture Recognition, Proceedings. Sixth IEEE International Conference, 565-570.
Kai, N. and Stiefelhagen, R. (2003), Pointing gesture recognition based on 3D-tracking of face, hands and head orientation, ICMI’03, November, 5-7, 2003, Vancouver, British Columbia, Canada.
Keskin, C., Erkan, A., Akarun , L. (2003), Real time hand tracking and 3D gesture recognition for interactive interfaces using HMM, ICANN 2003
Lee, M. S., Weinshall, D., Cohen-Solal, E., Comenarez, A., and Lyons, D. (2001), A Computer Vision System for On-Screen Item Selection by Finger Pointing, in Proc IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, Dec 2001.
Lee, Y.H., Yeh, C.H., and Wu, S.K. (2008), Accuracy measurement of distant goal- directed hand pointing movements, Conference Proceeding of the 1
stEast Asia Ergonomics
Symposium, Nov. 12 – 14, Kitakyushu, Japan.
Lee, Y. H. and Ko, C. H. (2009), A remote, hand-free, 3-dimensional finger trajectory tracking system and its application, Conference Proceeding of IEA 2009, Aug. 9-14, Beijing, China McKenna, S. J. and Morrison, K. (2004), A comparison of skin history and trajectory-based representation schemes for the recognition of user-specified gestures, Pattern Recognition 37, 5, 999-1009
Nakamura, T., Takahashi, S., and Tanaka, J. (2008), Double-crossing: a new interaction
technique for hand gesture interfaces, Proceedings of 8
thAsia-Pacific Conference, APCHI
2008, Seoul, Korea, July 2008
2009國際人因工程年會
XVII Triennial Congress of IEA
International Ergonomic Association, 2009
李永輝
國立台灣科技大學 工業管理系
(一) 前言
人因工程學(Ergonomics)是一門應用生理、心理和工業工程，改善工作系統，
使得人員能在安全、衛生、和舒適的情況下，發揮其最大工作效率、及提高生 活品質的科學。人因工程的主要目的在增進人員的績效(如增加工作的速度、
提高工作時的安全性和增進工作時的正確性)、減少能力的浪費與疲勞、減少 人為錯誤所引發的事故、增進人力的使用以降低時間和設備的損失，以及改善 使用者或操作員的舒適程度，扮演著重要的角色。
此一會議自1961年起於瑞典舉行後，每三年舉會一次，今年適逢第十七屆，
特於北京奧運後於中國北京舉行，有其深遠的意義。多年來人因工程領域快速的 發展與擴張，吸引了許多包括在亞洲在內的開發中或已開發國家的參與。此次會 議主要由國際人因工程協會(International Ergonomic Association, IEA)所支持，
由中國人因工程學會，台灣的中華民國人因工程學會，香港人因工程學會共同合 作舉辦，更具全球華人合作的意義。提供了人因工程領域中，快速發展的方法、
工具、軟體以及設計相關的資訊與論壇。
圖一：台灣人因工程學會提供的形象設計 圖二：IEA 2009會議會場
(二)會議內容
此次會議提供相關領域各式學術以及實務之成果報告，主辦單位邀請了十三位國 際知名學者與實務人士發表專題，安排了五個場次的業界參訪，四場次的論壇 (workshop)，並875篇的論文以口頭及海報的方式發表，堪稱是此一領域的最大盛 會。邀請發表的學者以及專題包括:
Monday, August 10
(1) The transition of ergonomic from research into practice, David Caple, IEA
(2) Workplace ergonomics and occupational safety and health- an ILO perspective, Shengli Niu, ILO
(3) Psychological rehabilitation and management after Wenchuan earthquake, Kan Zhang, Chinese ergonomic society
Tuesday, August 11
(4) Ergonomic issues related to children, computers and schools, Susumu Saito, JES
(5) Aging and mobile devices: Challenges and requirements for age-sensitive electronic information designs, Martine Ziefle, Germany (6) Challenges for the aging workforce using ergonomics strategies,
Masaharu Kumarshiro, Japan
Wednesday, August 12
(7) Designing for a multicultural world: conversing with users around the world, Apala Chavan, India
(8) Methods for identifying and evaluating affective product design, Martin Helander, Singapore
(9) Digital ergonomics: Its application in product and workplace design, Mao-jiun Wang, Taiwan
Thursday, August 13
(10) Are ergonomic interventions effective and worth cost in preventing or reducing MSDs, Paulien Bongers, Netherlands
(11) Agricultural ergonomics: historical and future perspective, Fadi Fathallah, USA
(12) Tracking the Banana skin-an ergonomics approach to slip, trip, and fall prevention, Roger Haslam, UK
Friday, August 14
(13) Human factors contribution to sustainability, Colin Drury, USA 會議中來自世界各國之學者專家發表875篇的論文，大致可歸類於下列二十五個子 題，每一子題下設若干組進行發表：
1. Activity Theories for Work Analysis & Design 工作分析與設計 2. Aerospace HFE 航太人因工程
3. Affective Product Design 感性產品設計 4. Aging 高齡議題
5. Agriculture 農業人因工程 6. Anthropometry 人體計測 7. Auditory Ergonomics 聽覺工程
8. Building & Construction 營建人因工程
9. Ergonomics for Children & Education Environments幼兒教育與環境 10. Ergonomics in Design產品設計人因工程
11. Gender and Work性別與工作 12. Healthcare Ergonomics醫療與健保
13. Human Aspects of Advanced Manufacturing製造環境人因工程 14. Human Factors and Sustainable Development永續與人因工程 15. Human Simulation and Virtual Environments模擬與虛擬環境 16. Mining礦業人因工程
17. Musculoskeletal Disorders肌肉骨骼系統傷害 18. Online Communities線上互動
19. Organisational Design and Management組織人因工程 20. Process Control流程設計
21. Psychophysiology in Ergonomics心理物理 22. Safety & Health安全衛生
23. Slips, Trips and Falls滑倒與跌倒 24. Transport交通人因工程
25. Work With Computing Systems電腦人機系統
此次大會台灣與會者除本人外，尚有中華民國人因工程學會理事長林久翔教 授、清華大學王茂駿教授、王明揚教授、台科大紀佳芬教授、朝陽陳協慶、李正 隆教授以及包括台科大、清大、及國內各大學研究生群參與，共計三十多人參加，
再參與盛會的近千名與會人士中,是一相當規模，並具代表性的學術團隊。筆者研 究生於十日在“ Anthropometry“小組中由博士研究生吳淑楷代為發表
“Anthropometry data differences between adult and elderly Taiwanese” ，筆者 另於”HCI-Touch UI and Novel Interaction” 小組中主持並發表”a remote,
bare-hand, 3D finger trajectory tracking system and its application”。參與大會 以及各國學者專家的成果發表,或益意良多。由於會議場次以平行方式進行，無法 聆聽各家言論，只見各各參與人士穿梭進出各場發表會，成為場內特色。綜合而 言，此次會議特色主題有：
1. IEA與NGO合作並向開發中國家邁進: IEA為42各會員國協的聯合組 織，下挾會員人數超過20,000，近年來最大的發展在開發中國家。IEA理事長 David C Caple – IEA President以The IEA Contribution to the Transition of Ergonomics from Research to Practice為題，說明IEA如何與Non
Government Organizations合作，包括 World Health Organisation (WHO), International Labour Organisation (ILO), International Organisation for Standardization (ISO), International Commission on Occupational Health (ICOH), International Occupational Hygienist Association (IOHA),
International Council of Societies of Industrial Design (ICSID)，並說明如何 與開發中國家，由人因工程的技術引進與合作，發展人因工程的影響與貢獻。
進而由理論邁向實務應用，相關的論述請見圖三。
圖三：由理論邁向實務的人因工程
2. 高齡化社會的角色與策略: 產業醫科大學教授Masaharu Kumashiro以 Challenges for the Aging Workforce Using Ergonomics Strategies為題，說明說明 日本如何因應高齡化社會以及相關的人因工程作為，四個策略為：Strategies at the individual level (範例: Ensuring health, preventing lifestyle-related diseases, rejuvenating the physical age and estimation of functional age); Strategies at the workplace level Improvement: (範例: Workplace improvement (KAIZEN)
activities developing support equipment and tools); Strategies at the company level (範例: Evaluating workability and building workability-based wage and benefit systems, horizontal development of success stories); Strategies at the state and regional administration levels (範例: Creating a database of company success stories, publicizing the database, establishing a certified employability evaluation system)。
3. 結合感性工學與使用者為中心的設計模式：前IEA理事長、現任新加坡 南洋理工大學教授Martin G. Helander1及 Halimahtun M. Khalid，以
Citarasa Engineering for Identifying and Evaluating Affective Product Design為題，提供他們所自創的設計模式與流程，這流程是架構在一個汽車設 計，以及全球使用者需求調查的前提下發展，包括systems approach, orderly process, explicit mapping, and decision structure. 內容包括了潛在客戶的感 性訴求，Why-Why-Why 建構感性訴求的的結構，以及少記需求的演繹，
Design Equations for Citarasa Analysis (DECA) 建構需求的數學模式，再利 用資料探勘技術，依據亞洲與歐洲使用者需求，發展出商用卡車的設計元素。
完整的論述出設計的前制黑箱內容。相關的理論架構鍵圖四。
圖四：Citarasa Engineering for Identifying Affective Product Design
4. 和永續精神接軌：美國Colin G. Drury教授以Human Factors Contributions to Global Sustainability為題，提供人因工程與永續工程精神接軌的概念。他 建議人因工程專業人員，應協助政府、企業、個人，提供具有永續、節能、減 碳、更具效能的工作環境、作業方法、設備採購，以及即時資訊與回饋系統的 建置。Dr. Drury提供了具體的方法與步驟。
5. 再論人因改善對於肌肉骨骼系統傷害的價值：工業化高度細分之重複 性作業，長期工作於電腦工作站，及物料搬運作業所引起之肌肉骨骼系統傷害 之問題，如：下背痛，腕管症候群，及肩頸部之不良姿勢性負荷，為熱烈探討 之人因議題。荷蘭學者Paulien Bongers以Are ergonomic interventions effective and worth cost in preventing or reducing MSDs為題，再論人因工 程的貢獻。肌肉骨骼系統問題一直是人因工程領域的挑戰，在27個歐盟國家
中，25% 的下背痛以及23% 的肌肉痛抱怨，造成最高的請假率與40% 的補償 金支出，相當於1.6% 的全國生產淨利支出(European Agency for Safety and Health at Work, Work-related MSDs prevention report). 如此高的持續成 本，使得懷疑與檢視過去的作為成為必要的工作。有關人因工程改善對於MSD 的正面效果，相關近期的科學證據僅能提供中等以上的支持，如此保守的結論 之最核心的問題，仍環繞在MSD的does-response關係不明。Dr. Bongers呼 籲，相關研究應以可以整合的模式進行，以提供更快速、更完整、也更具說服 力的證據。
(三)與會收穫
IEA會議每兩三年召開一次，此次個人有幸獲得國科會之補助參加大會，得以參與 學術會議、協助主持、參與周邊會議，並聆賭大師級人物之風範與立論，獲益良 多。在此將所得之收穫與心得，概述於下：
1. 促成中、日人因工程學會的合作實質：延續筆者參加日本人因學會五十 週年慶的任務與投入，兩學會成員再次於IEA2009北京會面，並確認兩學會的 合作內容。參與人員包括中華民國人因工程學會理事長林久翔教授、清華大學 王明揚教授、筆者，日方則有學會會長Susumu Saito, Ph.D. President, 日本大 學Yoshinori Horie教授、關西大學 Kotani Kentaro教授等人。討論主軸在good practices 的翻譯與合作發展台灣本土的good practices的工作。作業現場推動 現場人因工程改善，一直是人因工程的核心價值，透過個案的報導、說明、與 標竿學習，是放大與複製現場改善價值的最佳手法。Ergonomic good practices 資料庫在JES不只已是網站上的重要資產，也翻譯成英文供各界引用，JES與EST 合作進行good practices中文化的工作。
2. 參與PPCOE的理監事會議：泛太平洋人因工程會議成員，也利用IEA2009 北京會議時間，會面於大會期間，並由IEA2009舉辦人王生教授作東後，於8/11 晚上舉行，主要的議題包括：明年PPCOE與台灣高雄舉辦，第二屆EAEFS於 台灣新竹舉辦，會議支持的Asian Ergonomic Journal (AEJ)期刊停辦議題。
PPCOE在現任理事長日本產業醫科大學Komarshiro的努力下，儼然成為亞洲 跨學會、跨國家、跨東北亞、東南亞、澳紐的人因工程平台，在他退休前，有 許多交棒與傳承的工作待處理。經熱絡討論後，對於明年的PPCOE成員會盡 力支持，但希望有合理的會議費用。第二屆EAEFS於台灣新竹舉辦將與PPCOE
地點衝突，且與亞洲的其他會議時間重疊，應努力往整合方向發展，新獲選擔 任IEA秘書長之清華大學教授王明揚，以其新的角色與地位，將更容易整合亞 洲的個個團體。AEJ在稿源不足的前題下，暫時停辦。並由韓國Kang Lee、筆 者、中國孫向紅博士，與馬來西亞代表組成工作小組，發展新的建議提案，在 下次的理監事會中提案。PPCOE與會學者請見圖五、圖六。
圖五 筆者與IEA2009舉辦人王生教授合影 圖六PPCOE理監事會召開
3. 申請北京清華大學的訪問學者：筆者利用在北京開會之便，擬利用教授休假期 間，安排前往北京清華大學講學。透過清華大學工業工程系、人因與工效學研 究所所長饒培倫教授的協助，獲得正面回應，現正提出書面申請中。
4. IEA的趨勢與變化：參與IEA三年一次的會員大會，適時的可以體會到此領 域變化與趨勢。明顯的有(1)開發中國國家的參與與受到重視，亞洲尤其是東南 亞國家、以及中南美洲會員參與及需求受重視，且與國際NGO有更多的互動 與資源互補。(2)跨國研究與團隊增加，包括美國、英國、法國、荷蘭、日本、
台灣、中國的人體計測專家，成立World Engineering Anthropometry Resource (WEAR)，除了提供更深入的研究與完整資料庫交換機制外，更是跨 領域、跨國際、跨理論與實務應用的最佳平台與範例。(3) 非全新但持續受關 注的議題：affective product design (感性產品設計), aging (高齡議題), ergonomics for children and education environments (幼兒教育與環境), gender and work (性別與工作), healthcare ergonomics (醫療與健保), human factors and sustainable development(永續與人因工程), online
communities(線上互動), work with computing systems(電腦人機系統)。
(四)攜回資料名稱
1. IEA 2009大會手冊及議程表 2. IEA 2009大會論文集CD一份
無研發成果推廣資料
