Sleep/Awake Detection

In the sleep/awake detection system, the detected region is divided into hundreds of macroblocks if the size we choose of macroblocks is 5×5 pixels (see Fig. 4.7).

Dimensions of the region in the rectangle with red edges are 265×85 pixels (i.e., 53×

17 = 901 macroblocks). Common sample rate of NIR camera is 30 frames per second, and it will waste the spaces for data if we capture image data by using the sampling rate. Because the human activity is not active in sleeping, we reduce the sampling rate to 2 frames per second for our records.

Fig. 4.8 The region of sleep/awake detection.

Table IV show the result of sleep/awake detection. An interval represents a sleep or awake video of 30 seconds. The threshold of MADI is 6 that is set by training data.

When a person is awake, our system will output 1, otherwise 0.

Table V

The result of sleep/awake detection

Awake Sleep Interval MADI Judge 1 Judge 2 MADI Judge 1 Judge 2

1 20.57 1 1 3.82 0 0

2 6.30 1 1 2.93 0 0

3 15.97 1 1 3.63 0 0

4 2.82 0 1 4.68 0 0

5 4.12 0 1 3.30 0 0

6 43.78 1 1 3.53 0 0

7 4.73 0 1 3.02 0 0

8 75.52 1 1 4.77 0 0

9 93.83 1 1 4.45 0 0

10 3.38 0 1 4.23 0 0

11 2.62 0 1 3.62 0 0

12 3.28 0 1 4.00 0 0

13 13.37 1 1 3.98 0 0

14 2.95 0 1 3.02 0 0

15 2.42 0 1 2.58 0 0

16 3.75 0 1 4.37 0 0

17 2.88 0 1 4.35 0 0

18 2.80 0 1 3.40 0 0

19 3.10 0 1 2.95 0 0

20 2.75 0 1 3.47 0 0

4.5 Sleeping Posture Recognition

Actions recognition system is utilized to classify sleeping postures in this thesis.

We set k 2.0 for the grayscale value background models and k_v 1.1 for the HSV color background models. In the HSV color space, we set L_ncc0.95 in the grayscale value space and k_H 1.3 and k_s 1.3 to detect shadow pixels. Fig. 4.3

shows results of foreground extraction in bright and dark environments. Key posture images of four sleeping posture are show in Fig. 4.8. We select different postures as templates according to degree of shrinking feet in sleeping postures, right and left foetus. Table VI show the correct rate of sleeping posture.

(a)

(b)

(c)

(d)

Fig. 4.9 Key postures of sleeping postures (a) log, (b) star-fish, (c) right-foetus, (c) left-foetus.

Table VI

The recognition rate of each sleeping posture

Log Star-fish Right-foetus Left-foetus

Person 1 100% (79/79) 100% (95/95) 96.1% (73/76) 98.2% (54/55)

Average 98.7% (301/305)

5. Conclusion

In this thesis, we implement the automatic home health care system that combine the face, action and sleep/awake recognition of a person in day and night. The test images are extracted by background subtraction in action recognition system and by Haar cascade classifier in face recognition system. Then, the test images are transformed to a new space by eigenspace and canonical space projection for better efficiency and separability. Because actions are dynamic unlike face, we gather three images with fixed interval to construct fuzzy rules for containing temporal information. In sleep/awake detection, the NIR images will are rectified by using the function of illumination variation firstly. Then, the motion estimation is utilized to quantify the activity degree of sleepers.

NIR images look similar to gray-level image. The NIR image has less information of hue and saturation components than color images. Therefore, the correct rate of face recognition in dark environment is much lower than in the bright environment. However, the correct rates of action recognition in bright and dark environment are not that different because information provided by NIR images is sufficient to extract almost complete foreground images. In the sleep/awake detection system, we also obtain very good by using motion estimation. In the future, it is necessary to find a new a new face recognition algorithm to improving the correct rate in darkness environment.

References

[1] W. H. Liao and C. M. Yang , “Video-based activity and movement pattern analysis in overnight sleep studies,” ICPR, pp.1-4, 2008.

[2] Y.T. Peng, C.Y. Lin, M.T. Sun, and C.A. Landis, "Multimodality Sensor System for Long-Term Sleep Quality Monitoring," IEEE Transactions on Biomedical Circuits and Systems, vol. 1, no. 3, pp.217–227, 2007.

[3] M. Piccardi, “Background subtraction techniques: a review,” in Proc. IEEE Int.

Conf. SMC., vol. 4, pp. 3099–3104, Oct. 2004.

[4] H. Saito, A Watanabe, and S Ozawa, “Face pose estimating system based on eigenspace analysis,” in Proc. Int. Conf. Image Processing, vol. 1, pp. 638–642, 1999.

[5] J. Wang, G. Yuantao, K. N. Plataniotis, and A. N. Venetsanopoulos, “Select eigenfaces for face recognition with one training sample per subject,” in Proc.8th Cont., Automat. Robot. Vision Conf., ICARCV 2004, vol. 1, pp. 391–396, Dec.

2004.

[6] M. M. Rahman and S. Ishikawa, “Robust appearance-based human action recognition,” in Proc. the 17th Int. Conf. Pattern Recog., vol. 3, pp. 165–168, 2004.

[7] L. X. Wang and J. M. Mendel, “Generating fuzzy rules by learning from examples,” IEEE Trans. Syst., Man Cybern, vol. 22, no. 6, pp. 1414–1427, Dec.

1992.

[8] P. Viola and M. Jones, “Robust Real-Time Face Detection,” IJCV, vol. 57, no. 2, pp. 137–154, Mar. 2004.

[9] P. Belhumeur, J. Hespanha, and D. Kriegman, “Eigenfaces vs. Fisherfaces:

Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern

Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, 1997.

[10] “OpenCV 1.0, Open Source Computer Vision Library,”

http://www.intel.com/technology/computing/opencv/, 2006.

[11] I. Haritaoglu, D. Harwood, and L. S. Davis, “W⁴: Real-time surveillance of people and their activities,” IEEE Trans. Pattern Anal. Machine Intell., vol. 22, no.8, pp. 809–830, August 2000.

[12] J. C. S. Jacques Jr., C. R. Jung, S. R. Musse, “Background subtraction and shadow detection in grayscale video sequences.” In Proc. SIGGRAPH, pp.

189–196, 2005.

[13] M. Soriano, B. Martinkauppi, S. Huovinen and M. Laaksonen, “Using the skin locus to cope with changing illumination conditions in color-based face tracking,”

in Proc. IEEE NORSIG, Kolmarden, Sweden, pp. 383–386, 2000.

[14] K. Etemad and R. Chellappa, “Discriminant analysis for recognition of human face images,” in Proc. ICASSP, pp. 2148–2151, 1997.

[15] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd edition, 1300 Boylston Street Chestnut Hill, Massachusetts USA: Academic Press, 1990. 

[16] S. B. Kang and R. Weiss, “Can we calibrate a camera using an image of a flat textureless Lambertian surface?” in Proc. ECCV, vol. 2, pp. 640–653, 2000.

[17] W. Huang, A. Phyo Wai, S. Fook Foo, J. Biswas, C. Hsia and K. Liou,

“Multimodalsleeping posture classification,” in Proc. ICPR, pp. 4336-4339, Aug., 2010.

[18] Y. C. Luo, “Extracting the Foreground Subject in the HSV Color space and Its Application to Human Activity Recognition System,” Master Thesis, Elect. and Con. Eng. Dept., Chiao Tung Univ., Taiwan, 2007.

[19] R. Gonzales and R. Woods, Digital Image Processing, 3rd ed. Pearson Education International, pp. 589–591, 2008.