Real-time human identification using a pyroelectric infrared detector array and hidden Markov models

Real-time human identification using a

pyroelectric infrared detector array and hidden

Markov models

Jian-Shuen Fang1, Qi Hao2, David J. Brady2, Bob D. Guenther2, Ken Y. Hsu1 1 _{Department of Photonics & Institute of Electro-Optical Engineering,}

National Chiao Tung University, 1001 Ta Hsueh Rd., Hsinchu 300, Taiwan 2 _{Fitzpatrick Center for Photonics and Communications,}

Duke University, Durham, NC 27707 USA [email protected]

Abstract: This paper proposes a real-time human identification system

using a pyroelectric infrared (PIR) detector array and hidden Markov models (HMMs). A PIR detector array with masked Fresnel lens arrays is used to generate digital sequential data that can represent a human motion feature. HMMs are trained to statistically model the motion features of individuals through an expectation-maximization (EM) learning process. Human subjects are recognized by evaluating a set of new feature data against the trained HMMs using the maximum-likelihood (ML) criterion. We have developed a prototype system to verify the proposed method. Sensor modules with different numbers of detectors and different sampling masks were tested to maximize the identification capability of the sensor system.

©2006 Optical Society of America

OCIS codes: (110.3080) Infrared Imaging Systems; (040.3060) Infrared Detectors

References and links

1. Anil K. Jain, Arun Ross, Salil Prabhakar, “An introduction to biometric recognition,” IEEE Trans. Circuits syst. Video technol. 14, 4-20 (2004).

2. M. Planck, "On the law of distribution of energy in the normal spectrum," Annalen der Physik 4, 533 ff (1901).

3. V. Spitzer, M. Ackerman, A. Scherzinger, D. Whitlock, "The visible human male: A technical report," J. Am. Med. Inform. Assoc. 3, 118-130 (1996).

4. N. Kakuta, S. Yokoyama, M. Nakamura, “Estimation of radiative heat transfer using a geometric human model,” IEEE Trans. Biomed. Eng. 48, 324-331 (2001).

5. Glolab Corporation, “Infrared parts manual,” http://www.glolab.com/pirparts/infrared.html.

6. U. Gopinathan, D. J. Brady, N. P. Pitsianis, “Coded apertures for efficient pyroelectric motion tracking,” Opt. Express. 11, 2142-2152 (2003).

7. “A. S. Sekmen, M. Wilkes, and K. Kawamura, “An application of passive human-robot interaction: human tracking based on attention distraction,” IEEE Trans. Syst. Sci. Cybern. A 32, 248-259 (2002). 8. Q. Hao, D. J. Brady, B. D. Guenther, J. Burchett, M. Shankar, and S. Feller, “Human tracking with

wireless distributed radial pyroelectric sensors,” IEEE Sensors Journal, to be published (2006). 9. J. S. Fang, Q. Hao, D. J. Brady, M. Shankar, B. D. Guenther, N. P. Pitsianis, K. Y. Hsu, “Path-dependent

human identification using a pyroelectric infrared sensor and Fresnel lens arrays,” Opt. Express. 14, 609-624 (2006).

10. Fresnel Technologies Inc., http://www.fresneltech.com/arrays.html.

11. L.E. Baum, “An inequality and associated maximization technique in statistical estimation for probabilistic functiona of Markov processes,” Inequality 3, 1-8 (1970).

12. L.E. Baum, T. E. Petrie, G. Soules, N. Weiss, “A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains,” Ann. Math. Stat. 41, 164-171 (1970). 13. L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,”

14. R. Durbin, S. Eddy, A. Krogh, G. J. Mitchison, “Biological sequence analysis: Probability models of proteins and nucleic acids,” Cambridge University Press, Cambridge, (1998).

15. R. Hughey, A. Krogh, “Hidden Markov model for sequence analysis: extension and analysis of the basic method,” Comput. Appl. Biosci. 12, 95-107 (1996).

16. K. H. Choo, J. C. Tong, L. Zhang, “Recent applications of hidden Markov models in computational biology,” Grno. Prot. Bioinfo. 2, 84-96 (2004).

17. W. S. Kim, R. H. Park, “Off-line recognition of handwritten Korean and alphanumeric characters using hidden Markov Models,” Pattern Recogn. 29, 845-858 (1996).

18. J. Hu, M. K. Brown, W. Turin, “HMM based online handwriting recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1039-1045 (1996).

19. S. Eickeler, A. Kosmala, G. Rigoll, “Hidden Markov Model based continuous online gesture recognition,” Proceedings of international conference on pattern recognition, 2, 1206-1208 (1998). 20. T. Jebara, A. Pentland, “Action reaction learning: automatic visual analysis and synthesis of interactive

behavior,” Proceedings of 1st international conference on computer vision systems, 273-292 (1999). 21. S. E. Levinson, L. R. Rabiner, M. M. Sondhi, “An instruction to the application of the theory of

probabilistic function of a Markov process to automatic speech recognition,” Bell Syst. Tech. J. 62, 1035-1074 (1983).

22. C. Neukirchen, G. Rigoll, “Controlling the complexity of HMM systems by regularization,” Proceeding of the 1998 onference on advances in neural information processing systems II, MIT Press, 735-743 (1999).

23. M. Faundez-Zanuy, E. Monte-Moreno, “State-of-the-art in speaker recognition,” IEEE Aerosp. Electron. Syst. Mag, 20, 7-12 (2005).

1. Introduction

A biometric system is an intrinsic pattern recognition system that ensures personal identification by evaluating the authenticity of a specific physiological or behavioral characteristic possessed by the subject. In conventional biometric systems, the complex structure of certain body parts (e.g. a human iris, human fingerprints, face, or hand geometry) are measured optically, analyzed digitally, and a digital code is created for each person. Recent advances in optical and digital technologies, biometric sensors, and matching algorithms have led to the deployment of biometric recognition systems in a variety of security application [1].

When a human walks, the motion of various components of the body, including the torso, arms, and legs, produces a characteristic signature. Human walking motion is a complex process and it is difficult to decouple the individual biomechanical contributions in a motion cycle for an analysis. From the thermal perspective, each person acts as a distributed IR source whose distribution function is determined by the shapes and the IR emissions of the components [2]. Combined with the various idiosyncrasies in how an individual carries himself, the human thermal signature will impact a surrounding sensor field in a unique way. The average human frame radiates about 100 W/m2 of power, which peaks at 9.55 mμ [3, 4]. There is a constant heat exchange between a human body and the environment due to the difference in their temperatures.

The Pyroelectric infrared (PIR) detector is sensitive in a range of 8~14 mμ and is able to detect humans within a fairly reasonable distance (<15 m). PIR detectors have been used in a wide variety of applications [6-8]. In our previous study, we used a PIR detector whose visibility was modulated by a Fresnel lens array to capture an analog feature of human walking motion [9, 10]. The spectra of the sensor response data generated by a human walking along a fixed-path were used to distinguish individuals. In this paper, a PIR detector array with masked Fresnel lens arrays is utilized to generate digital sequential data that can represent a human motion feature. This digital feature based system using PIR detectors is insensitive to the velocity over the angular velocities between 1.1 rad/s and 3.1/s [8]. A feature model is based on the statistics of the on-off patterns of the sensor array for a walker who can walk at different speeds during the training stage.

Although IR cameras with large numbers of pixels are also capable of advanced positioning and control, they are inevitably associated with high data-loads, computational

costs, and much higher system costs. Sensor systems based on the PIR detectors with coded masks, on the other hand, can achieve the desired identification capability at low data-loads, computational and system costs. These pyroelectric detectors are available in either single element or dual element versions. A single element detector responds to any temperature changes in the environment and therefore needs to be thermally compensated to remove sensitivity to ambient temperature. In this study, we used the dual element PIR sensors. Dual element detectors have the inherent advantage that the output voltage is the difference between the voltages obtained from each of the elements of the detector which subtracts out environmental effects [5]. Therefore, the performance of this human identification system is robust to the environmental temperature, and suitable for both indoor and outdoor working environments.

Hidden Markov models (HMMs) are a widely used tool for sequential data modeling. Although the basic theory and inference tools were developed in the late 1960s [11, 12], HMMs have been extensively applied in the last decade to such applications: speech recognition [13], DNA and protein modeling [14-16], handwritten character recognition [17, 18], gesture recognition [19], and behavior analysis and synthesis [20]. In this study, we use HMMs to model the digital features generated by a sensor module. An example sensor module with modulated visibilities is illustrated in Fig. 1. The sensor array is distributed vertically, in an expectation that each sensor can capture the thermal dynamics of a different part of a walker. The geometry is actually 1-D and the sensor is not sensitive to anything but entry and exit from the detection region. The sensor module can sample the IR fields produced by humans and convert the pyroelectric response signals into digital sequential data. For each registered subject, an HMM is built during the training phase. In the testing phase, the set of trained HMMs are then used to estimate the identity likelihoods of a newly generated signal, for either path-dependent or path-independent human identification.

Fig. 1. A sensor module (Model 4M) and its visibilities that associate detection regions and the four sensors.

Lens Array Visibility 1 [1 1 1 0 0 0 1 1 1 0] 2 [0 0 0 1 1 1 0 0 0 1] 3 [1 1 1 0 0 0 1 1 1 0] 4 [0 0 0 1 1 1 0 0 0 1]

1

2

3

4

2. Hidden Markov models and multiple hypothesis testing for human identification Training Data Feature Extraction HMM Training Feature Model 1 Testing Data Hypothesis Testing Feature Validation Feature Extraction Feature Validation . . . Feature Model K

Fig. 2. The diagram of the identification process.

Figure 2 outlines the identification process. It has two phases: training and testing. In the training phase, we construct an HMM for each registered subject. In the testing phase, the association likelihoods of an unknown sequence with a set of trained HMMs are estimated and the identity of the subject is then obtained by choosing a model with the maximum likelihood value.

2.1 Hidden Markov Models (HMMs)

Hidden Markov models (HMMs) can be characterized by a set of output distributions and a finite-state Markov chain. A first order HMM is defined by the following elements:

M: the number of observation symbols; N: the number of states;

T: the length of the observation sequence;

1 2

{ , ,..., _T}

S= s s s : the set of hidden states;

1 2

{ , ,..., _M}

V = v v v : the set of discrete observation symbols; { }ij

A= a : the state transition probability distribution, where aij=P s( t+1= j s| t =i) , 1≤i j, ≤N, 1≤ ≤t T, and 1 0 1 ij N ij j a a = ≥ ⎧ ⎪ ⎨ ₌ ⎪⎩

∑

. (1) { ( )}_{j k}

B= b v : the observation symbol probability distribution, where

( ) ( | ) j k k t b v =P v at t s = j , 1≤ ≤j N, 1≤ ≤k M , and 1 ( ) 0 ( ) 1 j k M j k k b v b v = ≥ ⎧ ⎪ ⎨ ₌ ⎪⎩

∑

. (2)

{ }π_i

Π = : the initial state probability distribution, where πi =P s(1=i), 1≤ ≤i N, and

1 0 1 i N i i π π = ≥ ⎧ ⎪ ⎨ ₌ ⎪⎩

∑

. (3)

For convenience, we can compactly denote the model parameter set by λ=( , ,A B Π). Then, an HMM can be completely specified by λ.

For the observation evaluation, let O=o o₁, ₂,...,o_Tbe an observation sequence where t

o ∈Vis the observation symbol at time t. Given a model λ and an observation sequence O, the observation evaluation problem P O( | )λ can be solved using forward-backward procedure in terms of forward and backward variables which are defined as follows:

Forward procedure:

Forward variable: α_t( )i =P o o( _{1 2}... ,o s_{t t} =i| )λ ( )

t i

α can be solved inductively: 1. Initialization: 1( )i i ib o( )1 α =π , 1≤ ≤i N. (4) 2. Induction: 1 1 1 ( ) [ ( ) ] ( ) N t t ij j t i j i a b o α+ α + = =

∑

, 1≤ ≤ −t T 1, 1≤ ≤j N. (5) Backward procedure: Backward variable: βt( )i =P o o( t+1 t+2...oT|st =i, )λ ( ) t i

β can be solved inductively: 1. Initialization: ( ) 1 T i β = , 1≤ ≤i N . (6) 2. Induction: 1 1 1 ( ) ( ) ( ) N t ij j t t j i a b o j β + β+ = =

∑

, 1≤ ≤ −t T 1, 1≤ ≤i N. (7) Finally, the observation evaluation can be written as

1 ( | ) ( ) ( ) N t t i P O λ α i β i = =

∑

, t∀ . (8)

2.2 Model training

In the training phase, the task is to find the model parameters that can fit best a set of training data. In our study, we use the expectation-maximization (EM) algorithm to find the maximum-likelihood (ML) estimate of the parameters of a HMM, given a set of observed feature sequences. This process is also known as the Baum-Welch algorithm. It can be described as follows:

(i) given an initial guess of λ =( , ,A BΠ);

(ii) the re-estimated algorithm and O are used to derive a new model λ=( , ,A B Π) with the property that P O( | )λ ≥P O( | )λ ;

(iii) replace λ by λ and repeat the re-estimation.

In order to describe the procedure for estimation of HMM parameters, we first define ( , )

t i j

ζ : the probability of being in state i at time t and state j at t+1, given the observation sequence O and the model λ, is defined by

1 1 1 ( , ) ( , | , ) ( ) ( ) ( ) ( | ) t t t t ij j t t i j P s i s j O i a b o j P O ζ λ α β λ + + + = = = = . (9)

We also define γ_t( )i as the probability of being in state i at time t, given the observation sequence O and the model λ; thus we can relate γ_t( )i to ζ_t( , )i j by summing over j by the following equation: 1 ( ) ( , ) N t t j i i j γ ζ = =

∑

. (10) Using γ_t( )i and ζ_t( , )i j , the re-estimation formulas for A, B and Πare

1 1 1 1 ( , ) ( ) T t t ij _T t t i j a i ζ γ − = − = =

∑

∑

, 1≤ ≤i N, 1≤ ≤j N, (11) 1, 1 ( ) ( ) ( ) t k T t t o v j _T t t j b k j γ γ = = = =

∑

∑

, 1≤ ≤i N , 1≤ ≤k M, (12) 1( ) i i π =γ , 1≤ ≤i N. (13) The re-estimation process iterates until the increase in (P O| )λ is small enough. The Baum-Welch algorithm is guaranteed to increase (P O| )λ with the re-estimated A, B and Π until the optimal point is reached [21].

2.3 Multiple Hypothesis Testing

After the model training process, we will obtain K HMMs if there are K registered subjects. Therefore, for an unknown observation sequence X, we will have K hypothesis {λ λ1, 2,...,λk} to test. Our HMM-based identification approach adopts the ML criterion, where an unknown sequence X is assigned to the model with the highest testing likelihood. The decision rule is

, arg max{ ( | )}

i i

i

X∈λ i= p X λ , 1≤ ≤i K, (14) where λi is the HMM corresponding to the ith registered object.

3. Experimental Results

3.1 Feature Generation

The most important aspect of a human identification system is to choose an appropriate feature that can distinguish individuals. In our study, we select the fixed length binary event index sequence generated by a pyroelectric sensor array as the digital human motion feature. Here, an event is defined as the thermal flux collect by a pyroelectric detector which exceeds a threshold, and can be associated with some specific motions of human subjects, such as moving across one or several adjacent detection regions. The event signals are generated by pyroelectric infrared detectors with periodic sampling masks on Fresnel lens arrays.

Figure 3 shows the experiment setup. A sensor module, which contains 8 PIR detectors, is mounted on a pillar at a height of 80cm to sample the IR radiation from a subject. The sensory data were collected when different persons walked in the field of view (FOV) of sensors. The detector signal is converted to an event signal by signal processing techniques: matched filtering, threshold testing, and low-pass filtering. A threshold value has to be chosen for each detector, proportional to the noise level of that signal channel. If a processed signal’s absolute value is larger than the threshold value, the signal value is set to ‘1’, otherwise to ‘0’. The process of event signal generation is shown in Fig. 4.

Fig. 3. The experiment setup.

Sensor Module

0 10 20 30 40 50 60 70 -1 0 1 0 10 20 30 40 50 60 70 -1 0 1 0 10 20 30 40 50 60 70 -1 0 1 0 10 20 30 40 50 60 70 -1 0 1 0 10 20 30 40 50 60 70 -1 0 1 Time (second)

Fig. 4. Event signal generation. (a) The response signals of a PIR detector. (b) Filtered signals. (c) Digitized signals. (d) Binary signals. (e) Event signals.

10 20 30 40 50 60 70 80 90 100 1 2 3 4 10 20 30 40 50 60 70 80 90 100 1 2 3 4

Fig. 5. Two 4-bit digital features (event index sequences) generated by two subjects walking along the same path.

0 10 20 30 40 50 60 70 80 90 100 0 5 10 15 0 10 20 30 40 50 60 70 80 90 100 0 5 10 15

Fig. 6. The corresponding decimal sequential signals of Fig. 5. (a) (b) (c) (d) (e) B it numbe rs B it numbe rs Number of samples Number of samples E vent ind ex E v en t ind ex Time (second)

Signal Processing & Thresholding operation

Event Index Synthesis

Counter Counter=L ? Hypothesis Testing Continue

…

Sensor Unit 2 Sensor Unit N Sensor Unit 1

Fig. 7. The flow-chart diagram of digital feature extraction.

This system was implemented using the TI’s micro-controller (MSP430149) and RF transceiver (TRF6901) module. The sensory data are processed on the embedded micro-controller and the event index sequences are transmitted to the host computer via a wireless channel. Fig. 5 illustrates two 4-bit digital features (event index sequences) generated by two persons. Fig. 6 shows the corresponding decimal sequential signals of Fig. 5. It can be seen that the digital features generated by the two persons are distinctive.

Figure 7 summarizes the procedure of digital feature extraction for the real-time human identification systems. The length of a feature sequence for real-time identification is fixed. When it reaches the preset length, the system resets itself and awaits the next batch of event sequences. These digital sequential data can be modeled in HMMs. The HMM characterizes the statistics of a finite-state sequence of training. The model parameters are initialized by a random guess and updated by the EM algorithm described in the previous section. There are two important parameters for HMM training: one is the number of states; another is the length of training sequences. The model with more states can describe more characteristics of the digital feature of an individual. However, when the number of states is increased, the computation cost will be much higher and an over-fitting problem may occur [22]. We choose the state number of an HMM for each individual after testing the identification capabilities with respect to different state numbers. When we increase the length of the training sequences, the identification rate can be improved at the expense of more training time. The selection of length of testing sequences requires a compromise between the identification rate and identification time. In the path-dependent case, we set the length to 2000 for the training sequences, and 200 for the testing sequence. In the path-independent case, we set the length to 3000 for the training sequences, and 500 for the testing sequence.

3.2 Path-dependent Recognition

For the path-dependent recognition problem, the sensory data was collected while different persons walked back and forth along a prescribed straight path, 2.5 m away from and perpendicular to the sensor. The experiment setup is shown in Fig. 3. A sensor module, which contains 4 or 8 PIR detectors and Fresnel lens arrays, is mounted on a pillar at a height of 80

cm to sample the IR radiation from the human target. The range of vertical field of view of the

sensor module (8 PIR detectors) is 53~136 cm from the ground. Within this range, the sensor module can detect IR radiation from torsos, arms, and legs of normal-height humans at the

same time. A more detailed discussion on the sensor module location can be found in our previous paper [9].

If the walker belongs to a predefined set of known walkers, it is referred to as closed-set identification. Adding a “none-of-the-above” option to closed-set identification gives open-set identification [23]. Fig. 8 (a) shows the experimental results of the verification (open-set identification). The digital features of five walkers were tested against one person’s HMM. It turns out that the person’s features can not achieve the maximum log-likelihood all the time. However, when we use the digital features of that person to check against all five persons’ HMMs (closed-set identification), the maximum likelihoods can always be achieved for that person’s HMM, that is, correct identification, shown in Fig. 8 (b). Therefore, it suggests that the proposed HMM approach is only suitable for the closed-set identification case.

The poor performance of HMM approach in the verification case might be caused by the intrinsic statistical instability of the digital feature. The short testing sequences contain less statistical information and more uncertainty, whereas the HMMs, derived from much longer training sequences, contain enough statistical information to make reliable statistical inferences. Longer testing sequences might improve the system performance in verification and open-set identification. However, from the practical point of view it is not realistic to collect a long digital sequence to recognize a person. Therefore, in this paper we only investigate the case of using the digital feature for closed-set identification.

0 2 4 6 8 10 12 14 16 18 20 -1400 -1300 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 (a) 0 2 4 6 8 10 12 14 16 18 20 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 -300 (b)

Fig. 8. Log-likelihoods of (a) five walkers’ testing data against one walker’s HMM; (b) one walker’s testing data against five walkers’ HMMs.

Number of tests Number of tests L o g -l ike lihoo ds L o g -l ike lihoo ds

(a) (b)

Fig. 9. Two sensor modules and their visibility matrices that define the detection regions of the four sensors. (a) Model 4L; (b) Model 4H.

50 60 70 80 90 100 2 3 4 5 6 7 8 9 10 4H 4M 4L

Fig. 10. Average path-dependent identification rates as a function of the number of persons for the three types of sensor modules.

Lens Array Visibility 1 [1 1 1 1 1 0 0 0 0 0] 2 [0 0 0 0 0 1 1 1 1 1] 3 [1 1 1 1 1 0 0 0 0 0] 4 [0 0 0 0 0 1 1 1 1 1]

Lens Array Visibility 1 [1 0 1 0 1 0 1 0 1 0] 2 [0 1 0 1 0 1 0 1 0 1] 3 [1 0 1 0 1 0 1 0 1 0] 4 [0 1 0 1 0 1 0 1 0 1] Number of persons Ide ntific ation ra te

1

2

3

4

1

2

3

4

(a) (b)

(c)

Fig. 11. Three sensor modules with 8 sensor units and their visibility matrices that define the detection regions of the eight sensors. (a) Model 8L; (b) Model 8M; (c) Model 8H.

Lens Array Visibility 1 [1 1 1 1 1 0 0 0 0 0] 2 [0 0 0 0 0 1 1 1 1 1] 3 [1 1 1 1 1 0 0 0 0 0] 4 [0 0 0 0 0 1 1 1 1 1] 5 [1 1 1 1 1 0 0 0 0 0] 6 [0 0 0 0 0 1 1 1 1 1] 7 [1 1 1 1 1 0 0 0 0 0] 8 [0 0 0 0 0 1 1 1 1 1]

Lens Array Visibility 1 [1 1 1 0 0 0 1 1 1 0] 2 [0 0 0 1 1 1 0 0 0 1] 3 [1 1 1 0 0 0 1 1 1 0] 4 [0 0 0 1 1 1 0 0 0 1] 5 [1 1 1 0 0 0 1 1 1 0] 6 [0 0 0 1 1 1 0 0 0 1] 7 [1 1 1 0 0 0 1 1 1 0] 8 [0 0 0 1 1 1 0 0 0 1]

Lens Array Visibility 1 [1 0 1 0 1 0 1 0 1 0] 2 [0 1 0 1 0 1 0 1 0 1] 3 [1 0 1 0 1 0 1 0 1 0] 4 [0 1 0 1 0 1 0 1 0 1] 5 [1 0 1 0 1 0 1 0 1 0] 6 [0 1 0 1 0 1 0 1 0 1] 7 [1 0 1 0 1 0 1 0 1 0] 8 [0 1 0 1 0 1 0 1 0 1] 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

50 60 70 80 90 100 2 3 4 5 6 7 8 9 10 8H 8M 8L

Fig. 12. Average path-dependent identification rates as a function of the number of persons for the three types of sensor modules.

We assume that the mask with high spatial sampling can capture more detailed IR information generated by human motions. Fig. 9 shows two sensor modules with different sampling masks. Including the sensor module shown in Fig. 1, we have 3 types of sensor modules containing 4 detectors for path-dependent identification. These sensor modules with different spatial sampling masks are used to create detection regions of different sizes within the sensor FOVs. In the training stage, we constructed 10 HMMs for 10 persons. In the testing stage, each person was tested 20 times. The number of walkers along a fixed path to be identified is increased from 2 to 10 for each type of the sensor module, tested against 10 feature models obtained from the training stage. The average path-dependent identification rates of the three different sensor modules with respect to the group size are shown in Fig. 10. We can see that the sensor module with high spatial sampling frequency has the best identification performance and the average identification rates decrease when the group size grows from 2 to 10.

To improve the identification capability, we increased the number of PIR detectors in the sensor modules to sample more information in the IR field. Fig. 11 shows three different sensor modules with 8 detectors using different periodic sampling masks. With more detectors, we can obtain binary digital features in a higher dimension. The dynamic range of the observation for the HMM becomes 0~255 (8-bits). Fig. 12 shows the average identification rates of the three different sensor modules with respect to the person number. We can see that the model 8H has the best performance. It can achieve an average identification rate above 90% for a small group of 10 persons.

3.3 Path-independent recognition

For the path-independent case, we used the same sensor setup as in the path-independent case. Each person in a group of 10 walked randomly inside a 9m×9m room. We used the mask model 8H to capture the digital features for the path-independent identification case. Because of the randomness in paths, longer training sequences and testing sequences are needed. Fig. 13 illustrates the impact of the length of training sequences and testing sequences on the identification rate. We can see little improvement in average identification rates for lengths of training data beyond 3000. For HMMs derived from training data of length 3000, testing sequences beyond 500 in length does not increase the identification rate. Therefore, for path-independent recognition we chose the length of 3000 for the training sequences, and 500 for the testing sequences. Table 1 shows the closed-set path-independent identification results for 10 walkers. It can be seen that in identification among 10 walkers the lowest identification rate is 60%, the highest is 95%, and the average is 78.5%. Fig. 14 shows the average

Number of persons

Ide

ntific

ation ra

identification rates we obtained. Like in the path-dependent case, the identification rate drops as the size of the group increased in number of people. When the group size grows from 2 to 10, the average identification rate decreased from 92.5% to 78.5%. This human recognition system is based on the IR radiation from the human bodies. Among all the factors that affect the human heat radiation, the cloth that walkers wear is the most important one. From the initial experiment results, the system recognition capability is invariant to the clothes with similar fabric. However, a person wearing clothes with different kind of fabrics (e.g., cotton one for training and then polyester one for testing) will degrade the recognition rate.

30 40 50 60 70 80 90 100 1000 1500 2000 2500 3000 3500 4000 (a) 30 40 50 60 70 80 90 100 100 200 300 400 500 600 700 (b)

Fig. 13. Average identification rates for a group of 10 as a function of the (a) training sequences in different length; (b) testing sequences in different length.

Sequence length of training data

Sequence length of testing data

Ide ntific ation ra te Ide ntific ation ra te

Table 1. Closed-set path-independent identification results for 10 walkers.

Results Eve Jason Pai Bob Scott John Evan Arnak Mohan Yu

Eve 95% 5% Jason 90% 20% 10% Pai 75% 10% 15% 20% Bob 5% 10% 85% 5% Scott 70% 15% John 5% 60% 10% 15% Evan 5% 85% Arnak 5% 5% 15% 5% 80% Mohan 15% 65% Yu 10% 10% 80% 70 75 80 85 90 95 100 2 3 4 5 6 7 8 9 10

Fig. 14. Average path-independent identification rates as a function of the number of persons.

4. Conclusion

In our previous paper [9], the spectrum of a single PIR sensor’s temporal signal (analog feature) is used to represent the human motion features. This system is only suitable for path-dependent human identification. In this paper, we proposed a digital feature based system for closed-set human identification. PIR detector arrays are used for generating digital sequential data to represent human motion features. The digital feature’s advantages are in its less rigid training process, decreased sensitivity to walking speeds, effectiveness in the path-independent identification mode, and high data compression ratio for wireless data transmission.

An HMM is constructed for each person by an EM learning process and used as a statistical feature model. A person is identified by testing an unknown digital feature against all the HMMs and selection based on the Maximum-likelihood criterion. Different number of detectors and different sampling masks in the sensor module were also studied to improve identification rates. The identification performance can be improved by increasing the number of detectors and the spatial sampling frequency of the masks. Among all the tested sensor

Ide

ntific

ation ra

te

modules, the one containing 8 detector units and a high spatial sampling mask demonstrated the best performance. Its average identification rates for 10 persons are 91% and 78.5%, in path-dependent and path-independent cases, respectively.

Our future work will include better selection of features and algorithms for the open-set, less cloth sensitive human identification and simultaneous multiple people recognition by using multiple sensor modules.

Acknowledgments

The authors would like to express their gratitude to all the volunteers taking part in the experiments for their enthusiasm and patience. The authors also acknowledge the support of the Army Research Office through the grant DAAD 19-03-1-03552.