iCare：社群化智慧型居家照護－總計畫：iCare：社群化智慧型居家照護(3/3)

行政院國家科學委員會專題研究計畫 成果報告

iCare：社群化智慧型居家照護--總計畫：iCare：社群化

智慧型居家照護(3/3)

研究成果報告(完整版)

計 畫 類 別 ： 整合型

計 畫 編 號 ： NSC 95-2218-E-002-026-

執 行 期 間 ： 95 年 10 月 01 日至 96 年 09 月 30 日

執 行 單 位 ： 國立臺灣大學資訊管理學系暨研究所

計 畫 主 持 人 ： 曹承礎

共 同 主 持 人 ： 朱浩華、黃寶儀、許永真、苑守慈

計畫參與人員： 學士級-專任助理：喬凱妮

處 理 方 式 ： 本計畫可公開查詢

中 華 民 國 96 年 12 月 20 日

行政院國家科學委員會專題研究計畫成果報告

總計畫:

I

C

ARE

: 社群化智慧型居家照護(3/3)

計畫編號：NSC 95-2218-E-002-026

執行期限：95年10月1日至96年9月30日

主持人：曹承礎 (國立臺灣大學資訊管理學系暨研究所)

計畫參與人員： 傅立成, 許永真,朱浩華 (國立臺大資訊工程學系所)

苑守慈(國立政治大學資訊管理學系暨研究所)

黃寶儀 (國立臺灣大學電機工程學系暨研究所)

Abstract—Pervasive computing (or ubiquitous comput-ing), as envisioned by Weiser [1], is a grand vision for computing where smart digital technology can seamlessly and invisibly weave into the fabrics of our everyday lives. Currently, two approaches have emerged in designing pervasive computing for digital healthcare applications: the smart environment versus the smart people approaches. The smart environment approach focuses on automation in which digital technology is used to replace people in per-forming their everyday tasks. On the other hand, the smart people approach emphasizes on mediation in which digital technology is not used to replace people's actions, but help people better perform their everyday tasks. Persuasive computing adapts the second design approach. By embed-ding digital technology into our environments, persuasive computing hopes to modify and shape our everyday be-haviors toward smarter, healthier people. In this session, we will share our experiences in the various iCare subprojects on how we come to the path of moving from pervasive to persuasive computing.

Keywords—Pervasive computing, persuasive computing, ubiquitous computing, digital healthcare

INTRODUCTION

The iCare research initiative [2] is about embracing and embedding digital technology into our physical living to enhance our everyday experiences at home. Specifi-cally, the iCare research seeks to create intelligent digital technology that can engage and excite people into active participation of desirable physical and mental activities at home that are considered healthy, creative, productive, educational, enjoyable, etc. Our goal is to develop digital technology that empowers people with desirable behav-iors, not just supports smart environments. A current re-search focus is in applying digital technology for healthcare and self-care at home.

The iCare research initiative is an interdisciplinary research effort. Our original team members come from mostly computing background, but have since been ex-tended to cover diverse background in electrical engi-neering, computer science, information management, occupational therapy, psychology, medicine, and nursing, enabling us to tackle the research challenge of digital living from technical, design, and human aspects.

In this session, we would like to share our experi-ences through various subprojects conducted in the iCare project, and describes how we have come to the path of moving from pervasive computing to persuasive com-puting.

THE ICARE EXPERIENCE

At the start of the iCare project, an object reminder

system [3] (video:

http://mll.csie.ntu.edu.tw/video/object_tracker.avi) was prototyped to automatically track the whereabouts of objects misplaced by users at home or in an office. The motivation is for the object reminder system to address an everyday frustration in which people often forget where they misplaced things, such as glasses, cell phone, wallet, keys, remote controls, etc., resulting in wasting consid-erable amount of time looking for these misplaced ob-jects.

To create our object reminder system, we took the smart environment approach and instrumented the envi-ronment with ultrasonic positioning sensors. In addition, we created a finger ring RFID antenna that detects when an RFID-tagged object is being held and released from a user’s hand, and a wristband wearable device that com-bines a small ultrasonic positioning tag, a RFID reader, and a Zigbee radio sensor node called NTU Taroko [7]. To track the location of an object, our system first tracks the location of a user’s hand using the wristband wearable device. Then, our system detects the moment when the user’s hand releases an object, which is also the time when the RFID reader on the wristband wearable device can no longer sense the RFID tag on the hand-released object. The object’s new location is therefore inferred as the position of the hand at the hand-released moment. The object’s new location is then updated on a map shown on a PDA. Fig. 1 shows pictures of our prototype system.

When our object reminder system was shown to human experts, they quickly commented that using this system would likely produce an adverse effect that leads to continuation of a poor habit to misplace things (i.e., not putting things back in the same place). The reason is that with this digital technology, users can always find things whether they put things in/out the same place. This ad-verse effect can be found in other digital technology. A prominent example is the use of the car navigation system, which may encourage directionless-ness in drivers be-cause drivers are not compelled to the habit of remem-bering and learning roads and direction while driving. As a result, drivers may become more dependent and reli-ance on the intelligence of the digital technology rather than on their own intelligence or good habits.

iCare Project: from Pervasive to Persuasive Computing

Proceedings of 2007 CACS International Automatic Control Conference National Chung Hsing University, Taichung, Taiwan, Nov. 9-11, 2007

Fig. 1. The iCare RFID-assisted object reminder system. On the top shows the wearable wristband device. In the lower left shows the RFID tag on the object. On the lower right is a map display showing the locations of objects.

PERVASIVE TO PERSUASIVE

C

OMPUTING

The comment from the human experts about our object reminder system made us think about alternative designs for pervasive computing technology, in which a digital enhanced smart environment motivates people to become smarter (not dumber) with more desirable, healthier behaviors. For the new object reminder, this alternative design also tracks the locations of objects, but only beeps to remind users at the moments when they misplace objects. That is, rather than helping users locate misplaced objects, this alternative design incorporates a behavior modification strategy, in the form of a digital mediation, to motivate a good habit in users such that they will not misplace objects (therefore, preventing mis-placed objects from occurring). In other words, we be-lieve that this persuasive approach can be a more effec-tive use of pervasive computing technology in many of our everyday living scenarios.

In our iCare project, we have identified the following three main layers for persuasive technology shown in Fig. 2: (1) sensor infrastructure to sense and monitor human behaviors and their context, (2) activity recognition to interpret human behaviors from low-level sensor data, and (3) intelligent interaction to influence or shape hu-man behaviors. In this conference session, we will de-scribe in more details five different i-Care projects that cover these three layers.

Interaction Design

(Influence and Shape Human Behaviors)

Activity Recognition

(Recognize Human Behaviors)

Sensor infrastructure

(Monitor Human Behaviors)

Fig. 2. Three layers in Persuasive Technology

IC

ARE

E

XAMPLES OF PERSUASIVE

T

ECHNOLOGIES

To illustrate what we mean by persuasive technology, we describe four iCare subprojects which experiment with adopting persuasive computing technologies for playful behavior modification.

Playful tray

Mealtime behavior is one of the most frequently cited problems by parents of young children [4]. Despite nutritional concerns, spending excessive time to eat a meal affects the participation of children in daily school and family routines, and often contributes to negative parent-child interaction during mealtime. To address this eating behavior issue, we have designed a Playful Tray as a tool to assist occupational therapists and parents in reducing poor eating behavior in young children.

The Playful Tray [5] is embedded with an interactive game played over a weight-sensitive tray surface shown in Fig. 3 (c) and (d). By detecting weight changes of the amount of food consumed, the tray surface can recognize and track the natural eating actions of children in real time. Child eating actions are then used as inputs to play a racing game. Screenshots for the Racing game are shown in Fig. 3 (a) and (b). When starting a meal, a child selects a favorite cartoon character to compete in the Racing game. Upon detecting each eating action, one of the characters would race one step forward to the right. When the child completes the meal, the game ends and the character that has traveled the furthest distance to the right wins the game.

The Playful Tray racing game design is based on combining play-based occupational therapy and persua-sive computing. The game design provides internal con-trol, an element of playfulness, to children by given them choices to select a favorite cartoon character to compete in the race. In addition, the pace of the game is also con-trolled by children’seating behavior.Thegamedesign gives a freedom from reality, another element of play-fulness, to children by not presenting punishment for not eating or eating too slowly. The game design brings in-trinsic motivation by randomly selecting a character to race one step forward after each eating action, which children become motivated to continue playing the game until finishing the meal in order to find out the final result of the race. The Racing game is easily within the per-formance capability of children by using only their natural eating behaviors as game inputs. In addition, the immediategameresponsethrough acharacter’sracing one step forward provides extrinsic feedback to children. Finally, the randomly selected racing character after each

eating action satisfies the partial reinforcement principle to encourage repeated performance of child eating ac-tions.

Weight sensor and sensing surface Palm-top PC with touch screen

(a) (b)

(c) (d)

Weight sensor and sensing surface Palm-top PC with touch screen

(a) (b)

(c) (d)

Fig. 3. The Playful Tray prototype: (a) and (b) are screen shots for the Racing game; and (c) and (d) show the tray prototype with a Palm-top PC and weight sensing surface.

Playful toothbrush

Proper brushing is essential for cleaning teeth and gums effectively and for maintaining oral hygiene [9]. For many parents, tooth brushing is a required routine for young children before bedtime. However, it is by no means an easy task for parents to get their young children into a habit of brushing their teeth properly and thor-oughly. Thus, we have designed and implemented the Playful Toothbrush as a tool to engage young children into active participation and learn proper tooth brushing method as recommended by American Dental Associa-tion (ADA) [9].

The Playful Toothbrush [10] incorporates an inter-active game played with an augmented toothbrush shown in Fig. 4. The system is consisted of the following com-ponents: (1) The toothbrush extension, shown in Fig. 4(b), is coded with different LED marker patterns on its four faces to aid our vision-based motion recognition system. (2) A web camera, shown in Fig. 4(a), is positioned above the child head to capture the top-down view of brushing motions. By analyzing the locations and motions of the LED markers from camera images, our system recon-structs the orientation of the brush extension and rotation of the bristle, and further infers the target group of teeth being brushed. (3) A tooth brushing game, shown in Fig 4(c),takesthechild’scurrentphysicalbrushing motions as game inputs to play a game on a LCD display. The gamestartswith avirtualimageofachild’sdirty teeth. The goal is for a child to thoroughly clean these virtual dirty teeth using his/her own physical tooth brushing motions as game inputs. For example, while a child is brushing his/her outer left teeth group, it is mapped to cleaning the virtual outer left teeth group of the mirror image on the LCD screen, with visible spots of plaques falling off. Fig. 4(d) and (e) show two game screenshots.

When the child finishes cleaning all his/her teeth, the virtual teeth will become completely white accompanying with applause sound.

Fig. 4. The Playful Toothbrush prototype: (a) shows a web camera to capture the top-down view of brushing motions; (b) is the toothbrush extension with LED marker patterns; (c) demonstrates a child using our Playful Toothbrush; (d) and (e) are screenshots of the Brushing game.

Nutrition-aware Cooking Kitchen

Obesity and overweight are major contributors globally to serious diet-related chronic diseases, includ-ing cardiovascular disease, hypertension and stroke, diabetes and others [11]. To prevent obesity and over-weight, special care should be taken to maintain the calorie content of every meal within daily requirements. Packaged foods sold in grocery stores have nutritional information including number of calories. However, a recent study has indicated that most people still favor home cooking or cooking meals from scratch [12]; in Europe, 52% and the US, 44% of people prefer scratch cooking. Average family cooks may not know how many calories are in their cooked meals after raw food ingre-dients are mixed and cooked, or whether these meals are considered healthy and offer a good number of calories for their family members. One of the reasons is that the average family cook cannot easily follow the steps of calculating calories during an intense cooking activity: first they have to estimate accurately the amount (weight)

(c)

(a) (b)

(c)

of each food ingredient used (such as oil, meat, vegeta-bles and others), and then they have to look up a food calorie table to calculate the overall number of calories after cooking and mixing several ingredients in a course or a meal.

This study presents a Calorie-aware Kitchen that can provide family cooks with awareness on the number of calories in their home cooked meals, thus help family cooks make healthy meals with the appropriate amount of calories, as recommended by nutritionists.

Fig. 5 depicts our Calorie-aware Kitchen [7]. It is augmented with sensors that track the food ingredients used during cooking, and provides just-in-time digital feedback to raise healthy cooking awareness. For in-stance, when a user prepares a meal, the kitchen presents calorie information whenever the user performs a cook-ing action that changes the amount of food cook-ingredients on the kitchen counter or the stove, such as by adding meat, pouring in oil, etc. Given the number of calories of each ingredient, an average family cook can reconsider the amounts of ingredients or the composition of a course, to achieve a good balance between healthy and delicious cooking. The developed kitchen also suggests the rec-ommended number of calories for a meal, based on the Harris-Benedict equation [25], for the entire family. Fi-nally, an element of playfulness lies in a calorie-aware cooking game, which makes cooking more enjoyable.

Fig. 5. Calorie-aware Kitchen with calorie tracking and digital feed-backs of nutritional information during cooking process. The bottom left shows weighing sensors deployed under counter, and the bottom right shows the overhead camera to capture images of the counter.

Diet-aware Dining Table

Our dietary habits affect our health in many ways. Research [19] has confirmed that dietary habits are portant factors for healthy living and have profound im-pacts on many chronic illnesses. The vast majority of the

population has chronic illnesses such as heart disease, diabetes, hypertension, dyslipidemia, and obesity. A re-cent Surgeon General Report indicated that approxi-mately 300,000 U.S. deaths are associated with obesity and overweight each year. The total cost attributed to overweight and obesity amounts to $117 billion in 2000. Proper dietary intake and related interventions are effec-tive in ameliorating symptoms and improving health [20]. Nutritious dietary is one of the most accessible means for people to prevent illness and to promote well-being [20]. Unlike traditional healthcare in which professionals assessand weigh one’sdietary intakeand then develop a plan for behavioral changes, ubiquitous healthcare technologies provide an opportunity for indi-viduals effortlessly to quantify and acknowledge their dietary intake. For example, at home patients face the cumbersome need to record everything they eat, a task which can take a minimum of 15-20 minutes per day [21]. Ubiquitous computing technologies provide a means for individuals to proactively monitor their intake and act upon it, leading to better food selection and more sensible eating.

The diet-aware dining table [18] is a dietary tracker built into an ordinary dining table as shown in Fig. 6. The motivation for creating this table is to influence of our eating behavior. We have designed and implemented a diet-aware dining table that can track what and how much we eat from the table. To enable automated food tracking, the dining table is augmented with two layers of weighing and RFID sensor surfaces. We devise a weight-RFID matching algorithm to detect and distinguish how people eat. To validate our diet-aware dining table, we have performed experiments, including live dining scenarios (afternoon tea and Chinese-style dinner), multiple dining participants, and concurrent activities chosen randomly. Our experimental results have shown encouraging rec-ognition accuracy, around 80%. We believe monitoring the dietary behaviors of individuals potentially contribute to diet-aware healthcare.

R

ELATED WORK

Many researchers have looked into how pervasive technology can manifest into our living environment to motivate behavioral change. The persuasive mirror [13] aims to motivate a lifestyle change by showing indi-viduals what they may become in the future. If a person has poor lifestyle habits such as excessive eating, smok-ing, lack of exercise, etc., the mirror will conjecture an unpleasant future-face to persuade lifestyle change. Tooth Tunes [14] is a smart toothbrush designed to mo-tivate the practice of tooth brushing in young children. The toothbrush is embedded with small pressure sensors to recognize brushing activity when the toothbrush is pressed against teeth. Upon the sensors being activated, a two-minute piece of music is played to reinforce children in continuing brushing for at least two minutes. Waterbot [15] is a device installed at a bathroom sink to track the amount of water usage in each wash. The system contains flow sensors to detect the amount of water usage. By

showing the current water usage in comparison to the average household water usage, the system encourages behavioral change toward water conservation. Out [16] designed a high-tech doll that resembles a human baby to simulate the difficulty of caring for a baby. The target users are teenagers with the goal being to prevent teen pregnancy. The doll contains an embedded computer that triggers a crying sound at random intervals. To stop the crying, a caregiver must pay immediate attention to the doll by inserting a key into the back of the baby for a specific length of time to simulate a care session. Car-Coach [17] is an educational car system that can utilize sensors in a car to detect good or bad driving habits, such as excessive braking, sudden acceleration, the use of signals when turning, etc. Subsequently, CarCoach aims to provide polite, proactive, and considerate feedback to drivers by factoring into their mental state and current road conditions.

Fig. 6. The picture on the left shows the embedded RFID and weighing table surfaces, and the picture on the right shows a dining scenario.

CONCLUSION AND FUTURE WORK

In the past two years, the NTU iCare Project has been supported by research grants from the National Science Council of Taiwan, the NTU Research Excellence Pro-gram, Intel Worldwide Education Foundation, and IBM Taiwan. The iCare 2.0, a three-year research proposal, has been approved by the National Science Council

Taiwan to focus on intelligent and playful user interaction in real-world environments. Some of the ongoing iCare 2.0 projects are described as follows:

ADL-aware Home

ADL-aware Home [22][23], Recognizing patterns of human activities is an important enabling technology for building intelligent systems for assisted cognition. Ex-isting approaches to human activity recognition often focus on mutually exclusive activities. In reality, people routinely carry out multiple concurrent activities. Our approach to activity recognition adopts a variety of probabilistic models to reason about sequential and co-temporal relationships among multiple concurrent human activities from heterogeneous sensor data, both wearable and in the environment. In addition, we apply machine learning techniques to emotion recognition us-ing physiological signals from wireless wearable sensors.

CoESM

CoESM [24] is a collaborative, context-aware ex-perience sampling method to record exex-perience samples of depressed patients. The goal of CoESM is to help a psychologist gain better understanding of their depressed patient’sphysicaland socialactivity levels.Itisdeve l-oped on a mobile device, such as a cell phone (shown in Fig. 7,) equipped with location (i.e., a GPS receiver) and activity (e.g., an accelerometer) sensors. From this mo-bile device, CoESM collects experience samples by de-livering and prompting questionnaires to patients. Con-ditions to trigger questionnaires are dynamic and context dependent. This enables an inquiring psychologist to specify conditions matching potential activating events of interests. When a patient completes a questionnaire, CoESM automatically logs a timestamp, and records the current context such as location, physical and social ac-tivity level. One of the interesting features of CoESM is collaboration. It enables caregivers (i.e., family and friends)to participatein recording patients’experience after they have had a face-to-face contact with patients. To detect a face-to-facecontact,apatient’sphoneper i-odically scans any nearby phone via Bluetooth. If a nearby phone belongs to a known participating caregiver, a questionnaire is delivered to the caregiver’s phone through SMS messaging.

P

ROJECT

W

EBSITE AND

V

IDEOS

The iCare project website is located in

http://mll.csie.ntu.edu.tw/icare. The video links for the Object Tracker, the Playful Tray, the Playful Toothbrush, the Diet-aware Dining Table, etc., can also be found on the iCare website.

A

CKNOWLEDGMENT

(

S

)

We would like to thank National Science Council in Taiwan under grants NSC 95-2218-E-002-026 and 95-2218-E-002-023, and Intel Education Taiwan for their

research funding

R

EFERENCES

[1] M. Weiser, “The computer for the 21th century”, Scientific American, September 1991.

[2] iCare project, http://mll.csie.ntu.edu.tw/icare/

[3] S. Wu, “RFID-assisted physical object location tracking system,” M.S. Thesis, Graduate Institute of Networking and Multimedia, National Taiwan University, July 2006.

[4] R.Manikam,J.Perman.“Pediatric feeding disorders,” Journalof Clinical Gastroenterology, vol. 30, 2000, pp. 34-46.

[5] J. Lo, T. Lin, H. Chu, H. Chou, J. Chen, J. Hsu, P. Huang, “Playful tray: adopting Ubicomp and persuasive techniques into play-based occupational therapy for reducing poor eating be-havior in young children,”Proceedings of the 9th International Conference on Ubiquitous Computing (UBICOMP 2007), Sept. 2007.

[6] Y. Chang, C. Huang, J. Lo, H. Chu, “A playful toothbrush to motivate proper brushing for young children”, Demo session and Adjunct Proceedings of the 9th International Conference on Ubiquitous Computing (UBICOMP 2007), Sept. 2007. [7] Pei-yu Chi, Jen-hao Chen, Hao-hua Chu, Bing-Yu Chen, “

Ena-bling nutrition-aware cooking in a smart kitchen,”in ACM CHI 2007 extended abstract, April, 2007.

[8] C. You, Y. Chen, J. Chiang, P. Huang, H. Chu, S. Lau,“Se n-sor-enhanced mobility prediction for energy-efficient localiza-tion,” ProceedingsofThird AnnualIEEE Communications So-ciety Conference on Sensor, Mesh and Ad Hoc Communications and Networks (IEEE SECON 2006), Sept. 2006.

[9] American Dental Association, “ADA.org”, 2007; http://www.ada.org/public/games/animation/index.asp.

[10] Yu-chen Chang, Chao-ju Huang, Jin-Ling Lo, Hao-hua Chu, "A Playful toothbrush to motivate proper brushing for young chil-dren," in the Demo session and Adjunct Proceedings of the 9th International Conference on Ubiquitous Computing (UBICOMP 2007), Innsbruck, Austria, September 2007.

[11] WHO (World Health Organization). Diet, Nutrition and the Prevention of Chronic Diseases. In WHO Technical Report Series #916, Joint WHO/FAO Expert Consolation (2003).

[12] Datamonitor Co. Changing Cooking Behaviors & Attitudes: Beyond Convenience. Commercial Report (2006).

[13] A. C. Andrés del Valleand A.Opalach,“Thepersuasivemirror: computerized persuasion forhealthy living,” Proc.ofInt’lConf. Human Factors in Computing Systems (CHI2005), 2005. [14] Hasbro, Inc., “TOOTH TUNES,” 2007;

http://www.hasbro.com/toothtunes/.

[15] E. Arroyo, L. Bonanni, and T. Selker, “Waterbot:exploring feedback and persuasivetechniquesatthesink”,Proc.ofConf. Human Factors in Computing Systems (CHI2005), 2005, pp. 631-639.

[16] J.W.Out,“Baby Think ItOver:Using role-play to prevent teen pregnancy,” Adolescence,vol.36(143), 2001, pp. 571-582. [17] E.Arroyo,S.Sullivan,and T.Selker,“CarCoach:a polite and

effectivedriving coach,” Proc.ofInt’lConf.Human Factorsin Computing Systems (CHI2006), 2006, pp. 357-362.

[18] Keng-hao Chang, Shih-yen Liu, Hao-hua Chu, Jane Hsu, Cheryl Chen, Tung-yun Lin, Chieh-yu Chen, and Polly Huang, “Diet-Aware dining table: observing dietary behaviors over ta-bletop surface,” in ProceedingsoftheInternationalconference on Pervasive Computing (PERVASIVE 2006), Dublin Ireland, May 2006, pages 366-382.

[19] Rosenberg, I. H. (1996). Nutrition research: an investment in the nation’shealth.NutritionReview, 54, s5-s6.

[20] S. Hankinson et al., editors. Healthy Women, Healthy Lives: A Guide to Preventing Disease. A Harvard Medical School book. Simon & Schuster Inc., 2001.

[21] J. Beidler et al. The PNA project. In Proceedings of the sixth annual CCSC northeastern conference on The journal of com-puting in small colleges, pp. 276–284. The Consortium for Computing in Small Colleges, 2001.

[22] Tsu-yu Wu, Chia-chun Lian, and Jane Yung-jen Hsu. “Joint Recognition of Multiple Concurrent Activities using Factorial

Conditional Random Fields”. In Christopher Geib and David Pynadath, editors, 2007 AAAI Workshop on Plan, Activity, and Intent Recognition, Technical Report WS-07-09. The AAAI Press, Menlo Park, California, July 2007.

[23] Chi-yau Lin and Jane Yung-jen Hsu. IPARS: Intelligent portable activity recognition system via everyday objects, human move-ments, and activity duration. In Christopher Geib Gal Kaminka, David Pynadath, editor, 2006 AAAI Workshop on Modeling Others from Observations, Technical Report WS-06-13, pages 44–52. The AAAI Press, Menlo Park, California, July 2006. [24] Li-shan Wang, Sheng-hsiang Yu, Keng-hao Chang, Sue-huei

Chen, and Hao-hua Chu,“Collaborative,context-aware experi-ence sampling for depressive patients,” in the Late Breaking Results (LBR) session and Adjunct Proceedings of the 9th In-ternational Conference on Ubiquitous Computing (UBICOMP 2007), Innsbruck, Austria, Sep 2007.

[25] Harris J, and Benedict F. A Biometric Study of Basal Metabolism in Man. In Washington D.C. Carnegie Institute of Washington (1919).