Brain Computer Interface-Based Smart Living
Environmental Auto-Adjustment Control System in
UPnP Home Networking
Chin-Teng Lin,
Fellow, IEEE,Bor-Shyh Lin,
Member, IEEE,Fu-Chang Lin, and Che-Jui Chang
Abstract—A brain computer interface-based smart living
environmental auto-adjustment control system (BSLEACS) is proposed in this paper. Recently, many environmental con-trol systems have been proposed to improve human quality of life. However, little research has focused on environmental control directly using the human physiological state. Based on the advantage of our technique on brain computer interface (BCI), we integrated the BCI technique with universal plug and play (UPnP) home networking for smart house applications. BSLEACS mainly consists of a wireless physiological signal acquisition module, an embedded signal processing module, a simple control protocol/power line communication environmental controller, and a host system. Here, the physiological signal acquisition module and embedded signal processing module were designed for long-term electroencephalogram (EEG) monitoring and backend analysis, respectively. The advantages of low power consumption and small volume of the above modules are suitable for smart house applications in daily life. Moreover, different from other BCI systems, the property of using only a single EEG channel to monitor cognitive state also makes BSLEACS become more practicable. BSLEACS has been verified in a practical demo room, and the environmental adjustment can be automatically controlled by the change of the user’s cognitive state. BSLEACS provides a novel system prototype for environmental control, and can be simply extended and integrated with the UPnP home networking for other applications.
Index Terms—Brain computer interface (BCI),
electroen-cephalogram (EEG), power line communication, simple control protocol, smart house, universal plug and play (UPnP).
I. Introduction
R
ECENTLY, with the advance in sensor technology and information technology, many studies are trying to de-velop commercial products to bring the convenience to people in their usual life. Therefore, a rapid growth of research on smart houses [1]–[12] is proposed and developed as a mainstream to provide various kinds of environmental control systems. Some environmental control systems in a smart house Manuscript received August 25, 2011; revised February 2, 2012; accepted March 17, 2012. Date of publication April 25, 2012; date of current version May 22, 2014.C.-T. Lin, F.-C. Lin, and C.-J. Chang are with the Department of Electrical Engineering, National Chiao Tung University, Hsinchu 30010, Taiwan.
B.-S. Lin is with the Biomedical Electronics Translational Research Center and the Institute of Imaging and Biomedical Photonics, National Chiao Tung University, Hsinchu 30010, Taiwan, and also with the Department of Medical Research, Chimei Medical Center, Tainan 71150, Taiwan (e-mail: [email protected]).
Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JSYST.2012.2192756
employed radio frequency identification (RFID), external sen-sor modules, and voice recognition as the controlled signals. RFID tag or external sensors are usually installed in different areas in advance for automatic detection of users’ motions. Moreover, by combining with universal plug and play (UPnP) home networks, users could send out service requests from their personal digital assistant, mobile phones, a wearable appliance, or external sensors to home server either with voice, graphic user interface, or motion.
Moreover, with the development of brain computer interface (BCI), it is an extremely new option to apply the physiological signals as the stimulus of environmental control system in a smart house. However, most of the existing brain computer interface-based environmental control systems, such as P300-based BCI [13]–[15] and motor-imagery-P300-based BCI [16]–[18], require the user’s active mental command to control external devices. Hence, these systems lack the capability to control devices automatically and adaptively according to the user’s current cognitive state. Moreover, most of current BCI-based environmental control systems are very inconvenient because bulky and expensive electroencephalogram (EEG) machines and personal computers are both required for physiological signals acquisition and backend analysis, which will limit the flexibility, portability, and practicability of these systems.
Therefore, the goal of this paper is to propose a cost-effective, simply extendable and easy-to-use brain computer interface-based smart living environmental auto-adjustment control system (BSLEACS) to control electric home appli-ances based on the change of user’s cognitive state (drowsiness or alertness). In BSLEACS, a wireless physiological signal acquisition module and an embedded signal processing module were also proposed. Different from other BCI systems, which are usually bulky and have to transmit an EEG signal to a backend personal computer to process the EEG signal [19], our proposed wireless physiological signal acquisition module and embedded signal processing module contain the advantages of small volume and low power consumption, and are more suitable for practical application. Moreover, by using UPnP home networking, BSLEACS can easily be integrated with electric home appliances for other applications. This paper is organized as follows. The system architecture of BSLEACS is introduced in Section II. The real-time cognitive state detection algorithm is introduced in Section III. The system performances of BSLEACS are investigated in Section IV. The 1932-8184 c 2012 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
Fig. 1. System architecture of proposed BCI-based smart living environmen-tal auto-adjustment control system.
discussions and conclusions are summarized in Sections V and VI, respectively.
II. System Architecture
The system architecture of BSLEACS, as shown in Fig. 1, mainly consists of four parts: 1) wireless physiological signal acquisition module; 2) embedded signal processing module; 3) host system; and 4) simple control protocol (SCP)/power line communication (PLC) environmental controller. Here, the wireless physiological signal acquisition module is designed to acquire and transmit an EEG signal to the embedded signal processing module wirelessly via Bluetooth. Bluetooth provides a short range wireless and secure communication between devices to eliminate the need for messy cables. By using the encryption function in the security procedures of Bluetooth, it will translate the transmitted data into secret code to avoid the contents being eavesdropped. The embedded signal processing module is designed to estimate the user’s cognitive state from his or her EEG, and provides the estimated cognitive state to the host system. The host system is designed for data storage/display, and is also served as an UPnP control point to manage the request from UPnP control device as well as the SCP/PLC environmental controller, which is used to control electric home appliances, such as day and night lamps, air conditioners, and others.
A. Wireless Physiological Signal Acquisition Module
The block diagram of the proposed wireless physiological acquisition module is shown in Fig. 2(a). It mainly consists of a front-end amplifier unit, a microprocessor unit, and a wireless transmission unit. Here, the front-end amplifier unit contains a preamplifier, a band-pass filter, and a 12-bit analog-to-digital converter (ADC). The gain of the front-end amplifier unit is set to 5040 times with a passing frequency band of 0.1– 100 Hz. EEG data digitized by ADC with the sampling rate of
Fig. 2. (a) Block diagram of wireless physiological signal acquisition module. Photographs of (b) wireless physiological signal acquisition module, and (c) EEG headband embedded with this module.
512 Hz will be stored into the memory of the microprocessor unit, and then be processed to pass through a moving average filter in the microprocessor unit to remove power-line interfer-ence before being sent to the wireless transmission unit. The wireless transmission unit consists of a printed circuit board antenna and a Bluetooth module, which is fully compliant with the Bluetooth v2.0+ EDR specification. This module operates at 31 mA with 3.7 V DC power supply, and can continuously operate over 33 h with a commercial 1100 mAh Li-ion battery. The volume of the proposed wireless physiological signal acquisition module is about 4 cm× 2.5 cm × 0.6 cm, which is small enough to be embedded into a headband as a wearable device, as shown in Fig. 2(b) and (c).
B. Embedded Signal Processing Module
The proposed embedded signal processing module that contains a powerful computation capability and can support various peripheral interfaces, as shown in Fig. 3(a), is de-veloped to perform the real-time cognitive state detection algorithm, and is also evaluated as the UPnP control device to send out the estimated cognitive state and EEG signal to host system to drive environmental controller via UPnP home networking. Here, the Blackfin embedded processor is used in the embedded signal processing unit. The operation frequency of central processing unit can run at up to 600 MHz. It contains two 16-bit multiply-and-accumulate to execute 1200 lines addition and multiplication functions and also has four independent direct memory access mechanisms to effectively reduce the processing time of core. A memory-mapped thin-film transistor liquid crystal display, which shares the same memory bus with synchronous dynamic random access mem-ory, is used in this module. Here, serial peripheral interface Flash is used to replace the parallel NOR flash to reduce the module size. Furthermore, this module also contains power management circuits. The embedded processor communicates with wireless transmission unit via universal asynchronous receiver/transmitter interface. This module can be operated with a 3.7 V DC power supply, and it can continuously operate for more than 45 h operations with a 1100 mAh Li-ion battery.
Fig. 3. (a) Block diagram of embedded signal processing module. (b) Photograph of embedded signal processing module.
The volume of the embedded signal processing module is about 6.4 cm× 4.4 cm × 1 cm, as shown in Fig. 3(b).
In this embedded signal processing module, the optimized universal boot loader was developed to perform the initial system configuration first and then to boot the Linux operation system kernel. The cognitive state detection algorithm was implemented as a multithreaded application on operation system. The received EEG data will be real-time processed, analyzed and displayed by the embedded signal processing module. When the change of cognitive state of the user is detected, the corresponding command will be transmitted either by radio frequency (RF) module or by Ethernet (a RS232-to-Ethernet adopter module is required) through UPnP protocol to the host system.
C. Host System and Environmental Controller
The host system is an UPnP/SCP bridge and is also served as the home gateway to internet network. With UPnP/SCP techniques, BSLEACS was realized to simply plug-and-play IP/non-IP consumer equipments in home networking with-out any complicated settings. In the host system computer, Windows XP was used as the operation system, and the host system program, developed on Microsoft Visual C#, was designed to provide following functions: 1) data storage and display; 2) UPnP control point to receive and reply the request from UPnP control device; and 3) SCP host to transmit control commands to environmental controller for operations.
A SCP-based environmental controller with four-channel AC/DC power line control outputs is used to control home equipments in this paper. All settings and control commands are accomplished with writing/reading three continuous regis-ters. Two or more kinds of commands can be sent from the host system to environmental controller to control the endpoints according to user’s cognitive state. In this paper, the SCP-based environmental controller is used to control the day and night lamps in the showroom. The adjustable DC outputs of environmental controller can be also employed if adjustable illumination of lights is required.
III. Methods
A. Real-Time Cognitive State Detection Algorithm
Previous studies have shown that EEG spectra in theta rhythm (4–7 Hz) and alpha rhythm (8–11 Hz) usually reflect
Fig. 4. Flowchart of real-time cognitive state detection algorithm. the changes of drowsiness and alertness [20]–[26]. When an alert person is becoming drowsy, his or her EEG power in both theta and alpha rhythms will increase. These discoveries motivate us to monitor user’s cognitive state from EEG spectra and apply it on smart living environmental auto-adjustment. In our previous study [27], an EEG-based unsupervised approach was proposed to detect the cognitive state without the requirement of labeled training dataset. Under the assumption that the subject is in an alert state during the first few minutes when taking a rest, the alert mode of the subject can be then derived by the first few minutes of EEG recording. The specific window is selected to build the alert mode by Mardia test [28]. If the subject remains alert, his or her EEG spectra in theta and alpha rhythms should match the alert model. Otherwise, his or her EEG spectra will diverge from the alert model if the subject is under drowsy state. In [27], we also observed that the alpha and theta rhythm of EEG spectra in the occipital midline (the location of Oz in the international 10–20 EEG system) can provide discriminating power and they have high correlation with cognitive state. Therefore, only a single EEG channel is used in our proposed system to monitor EEG signal in the occipital midline.
The flowchart of real-time cognitive state detection algo-rithm is illustrated in Fig. 4. In order to estimate the cognitive state, the information of theta and alpha rhythms in EEG spectra has to be preserved and extracted. According to the Nyquist criterion, the sampling rate of EEG signals related to cognitive state must be higher than 22 Hz to preserve and extract information of theta and alpha rhythms. Hence, a low-pass filter with a cutoff frequency of 32 Hz is first applied to remove 60 Hz power line noise and other high-frequency noise. Next, the sampling rate of EEG data will be down-sampled to 64 Hz to reduce the computation load. A 512-point FFT with 448-point overlap is used to obtain the EEG spectra, and then the EEG spectra in alpha and theta rhythms are extracted to build up the alert model. A new alert model for each subject in every experimental session will be constructed separately. The distribution of power spectrum in the alert state can be modeled by a multivariate normal distribution N(μ,2). Here, μand 2denote the mean vector and the variance–covariance matrix, respectively, and can be estimated by using maximum likelihood. The alert models of alpha and theta rhythms in this paper are represented by (μA, 2A) and (μT, 2T), respectively. After building the alert mode, the Mahalanobis distance from the alert mode of alpha rhythm (MDA) and that of
theta rhythm (MDT) will be calculated directly. Mahalanobis distance is a distance measure based on correlations between variables by which different patterns can be identified and analyzed. Different from Euclidean distance, it takes into account the correlations of the data set, and is scale-invariant. By estimating the Mahalanobis distance from the alert mode, the correlation between the current EEG signal and the alert mode can be effectively evaluated. Let xA and xT be EEG spectra in alpha and theta rhythms, respectively, at some time instant, then the values of MDA and MDT can be calculated by
MDA(xA) = (xA− μA)T(2A)−1(xA− μA) MDT(xT) = (xT− μT)T(T2)−1(xT− μT). (1) Finally, we use the linear combination MDC of MDT and MDA, to estimate the user’s cognitive state with the following formula:
MDC= α× MDA + (1 − α) × MDT, 0 ≤ α ≤ 1 (2) where α is a constant between 1 and 0. If the value of MDC is larger than the threshold, the subject can be treated as his or her cognitive state trends to drowsy state; otherwise, it trends to alert state. However, using the user’s instantaneous cognitive state directly to control electric home appliances is not practicable because the sensitive variation of control command may cause the discomfort of the user. In this paper, the control command is decided according to the trend of the user’s cognitive state. Here, the average of estimated cognitive states during the previous 10 min is used to estimate the trend of the user’s cognitive state.
B. Performance Evaluation for Real-Time Cognitive State Detection Algorithm
Because the user’s cognitive state is subjective and relative, it is difficult to compare it with the estimated cognitive state obtained by the real-time cognitive state detection algorithm directly. In order to evaluate the system performance, the user’s response behavior, which reflects the user’s cognitive state indirectly, is used to compare with the estimated cognitive state. However, the behavior in the past was only available by subjective examination of users’ cognitive states from a camera and then cross-referencing with questionnaire forms. Such a method cannot be used to monitor the user’s cognitive state continuously, and the results are less meaningful if the num-bers of participants are insufficient from the statistical point of view. Therefore, a lane-keeping driving experiment was designed to collect the user’s response behavior continuously to provide sufficient behavior data from limited participants. This experiment estimates the user’s cognitive state indirectly by monitoring the user’s driving response time [24], [25], [27]. Here, a virtual reality (VR)-based cruising environment was built to simulate a car driving on a four-lane highway at night. The car will randomly and automatically deviate from the center of the cruising lane. Subjects are asked to compensate for this deviation to keep this car in the center of the third cruising lane. The time points of three important events, as
Fig. 5. Illustration of driving task in lane-keeping driving experiment.
Fig. 6. Procedure of determining threshold of Mahalanobis distance for cognitive state detection.
shown in Fig. 5, are recorded to get the driving trajectory: deviation onset (the car starts to drift away from the cruising lane), response onset (participants respond to the car-drifting event), and response offset (the car returns to the center of the third lane). If the subject is alert, the response time, defined as the time duration from “deviation onset” to “response onset,” should be short. Then, the user’s response behavior can be recorded from the response time which can reflect the driver’s cognitive state indirectly. In our VR-based four-lane scene, the car will drift 1/4 of the road width per second after the occur-rence of car drift events. If the driver’s cognitive state is alert, he/she should correct the deviation within 0.2–1 s to avoid the car drifting into other lanes. Moreover, F-measure, the har-monic mean of precision [positive predictive value (PPV)] and recall (sensitivity), is used to find out the threshold of Maha-lanobis distance to decide the cognitive state in this paper. The procedure of determining the threshold is depicted in Fig. 6, and the value F of F-measure can be calculated as follows:
F = 2×precision× recall
precision+ recall. (3)
IV. Results
A. Performance of BSLEACS for Cognitive State Detection In this section, the optimum parameters for the real-time cognitive state detection algorithm are first determined. Here, the parameters of binary classification test are defined as follows: True Positive (drowsy people correctly recognized
TABLE I
Results of Testing Session for Cognitive State Detection
Subject F-measure (%) PPV (%) Sensitivity (%)
1 77.7 75.5 80 2 72.2 78.8 66.7 3 89.1 80.4 100 4 87.5 77.8 100 5 87.4 77.6 100 6 88.9 80 100 7 83.5 78.7 88.9 8 81.1 77.9 84.6 9 66.1 65.5 66.7 10 86.8 76.6 100 Average 82 76.9 88.7
as drowsy), False Positive (alert people wrongly recognized as drowsy), True Negative (alert people correctly recognized as alert), and False Negative (drowsy people wrongly recognized as alert). A total of 1370-trial response times and Mahalanobis distances from 15 subjects were analyzed to determine the maximum F-measure value with different parameters (α = 0.1–0.9 and the threshold of Mahalanobis distance = 1–15). After comparing the results with different parameters, the maximum value 77.6% of F-measure (PPV = 69.2% and sensitivity = 88.3%) was determined with α = 0.9 and threshold = 7.5 in this paper.
Next, 1000-trail response times and Mahalanobis distances from ten subjects for testing session were used to test the per-formance of this system (α = 0.9 and the threshold of Maha-lanobis distance = 7.5). The result of testing session for cogni-tive state detection is listed in Table I. It shows that most of the precision of drowsiness prediction (PPV) is between 75% and 80% and the sensitivities of subjects are over 80% except sub-ject 2 and subsub-ject 9. The averaged F-measure of ten subsub-jects is 82 % (PPV = 76.9% and sensitivity = 88.7%). For sensitivity analysis, we perform single sample t-test to investigate the performance of our system. The p-value and estimated stan-dard deviation (SD) of PPV and sensitivity are (p = 0.00029 and SD = 4.25) and (p = 0.93 and SD = 13.73), respectively, which evidences that BSLEACS can provide a stable PPV to effectively recognize current cognitive state of the user.
B. Performance of BSLEACS for Controlling Home
Appliances
BSLEACS has been constructed at the Eco-City Integrated Smart Living Technology Regional Center, National Chiao Tung University, Hsinchu, Taiwan, as the snapshot shown in Fig. 1. Here, BSLEACS is used to control day and night lamps in the showroom. For example, activities, such as exercise and working that people are under alert cognitive state, are more suitable for bright illumination. On the other hand, relaxing and sleeping that people are under drowsy state are appropriate for twilit illumination. Therefore, the control criteria of day and night lamps in this paper are defined as follows.
Criterion 1) when the trend of cognitive state is alert, the major day lamp is on and the night lamp is off. Criterion 2) when the trend of cognitive state is drowsy, the major day lamp is off and the night lamp is on.
In order to evaluate the system performance of BSLEACS for controlling home appliances, a total of 75-trial system re-sponses and questionnaire results from 15 subjects were cross-referenced and analyzed. The parameters of binary classifica-tion test are defined as follows: True Positive (control criterion 2 was correctly performed when the subject felt drowsy), False Positive (control criterion 2 was wrongly performed when the subject felt alert), True Negative (control criterion 1 was correctly performed when the subject felt alert), and False Negative (control criterion 1 was wrongly performed when the subject felt drowsy). The result of testing session for system control performance is listed in Table II. The F-measure of system control performance is 75.27% (PPV = 70% and sensitivity = 81.40%). The experimental result evidences that BSLEACS can effectively control home appliance according to the user’s cognitive state.
V. Discussion
The specification comparisons between BSLEACS and other environmental control systems are listed in Table III. Chaya et al. [11] presented a voice-controlled smart house in 1993. They first categorized the home environment into several areas, such as energy management, communication, security, convenience, and entertainment, and then a set of corresponded voice commands within the defined areas in a house were recorded. An example of telling smart house to turn off the lights might be “energy management, light, off.” Such a voice-controlled smart house is inconvenient for users in that they must predefine their home environments and record the corresponding voice commands. Corcoran et al. [12] proposed an UPnP home network infrastructure to provide services to user of a wireless home network from a mobile phone, or a wearable appliance to overcome the inconvenience in [11]. User could send out the service request to home server either with voice or user interface, which could overcome the inconvenience of predefined areas and voice commands. However, the cost of such a PDA-based smart appliance is expensive and difficult for operations. Hwang et al. [10] introduced a RFID-based multiuser access control algorithm in an UPnP smart home. The user has to take a RFID tag and many additional RFID readers have to be installed in different areas, such as bed room, kitchen and living room, in advance for automatic detection of users’ moving-in or moving-out a specific region. Helal et al. [2] proposed a wear-less smart floor technology with pressure sensor to detect inhabitant location in a house. Liau et al. [3] also proposed a wear-less inhabitants tracking system in a cluttered home environment via floor load sensors. However, the cost of the smart floor is about USD 4.00 per square-foot that is quite expensive for practical implementation in a house. These studies provided the environmental control technologies either with voice or position detection by floor pressure sensor. However, all of the above systems cannot adjust environment automatically according to the change of users’ physiological state.
Some brain computer interface-based control systems, which are designed to control external devices by using users’
TABLE II
Results of Testing Session for System Control Performance
System Control Output Control Criterion 1 Control Criterion 2 Cognitive state (questionnaire) Drowsy 8 (FN) 35 (TP)
Alert 17 (TN) 15 (FP)
TABLE III
Comparison Between BSLEACS and Other Environmental Control Systems
EC System Chaya [11] Corcoran [12] Hwang [10] Helal [2] BSLEACS Major technique Voice 1. Voice/UI 1. RFID 1. RFID 1. EEG processing
recognition 2. UPnP 2. UPnP 2. Floor sensor 2. UPnP 3. External sensor 3. SCP/PLC
Controlled signal Voice Voice/UI RF RF EEG
Nondisabled users Yes Yes Yes Yes Yes
Disabled users Yes Yes No No Yes
Aged people Yes Yes Yes Yes/No Yes
Low vision people Yes No Yes/No Yes/No Yes
Hearing impaired people Yes/No Yes/No Yes Yes Yes
Voice impaired people No No Yes Yes Yes
T
ype
of
smart
house
Cognitive impaired users Yes/No Yes/No Yes/No Yes/No Yes
Health monitoring device No No No No Yes
Wearable No No Yes No Yes
Cost − − − >$41 <$302
1Unit price for smart floor sensor per square-foot.
2Bill of material cost for embedded signal processing module.
TABLE IV
Comparison Between BSLEACS and Other BCI Systems
MindBalance Graz-BCI SSVEP BCI
BCI System [13] [17] Multimedia BSLEACS
Control [15]
EEG signal SSVEP Motor imagery SSVEP Alpha and theta rhythms 3 x EEG
Channels 2 x EEG 3 x EEG 1 x EEG 1 x EEG
4 x EMG
Transmission Cable Cable RF transmission Bluetooth Power supply Power line Power line Power line 3.7 V Li battery Backend signal Personal Personal Embedded signal Processing unit computer computer FPGA board Processing
Module Control mode Active mental Active mental Active mental Adaptation
command command command
physiological state, also have been developed in previous studies. The specification comparisons between BSLEACS and other BCI-based control systems are listed in Table IV. Lalor et al. proposed MindBalance system to control videogame [13]. MindBalance gains 1-D control of the character’s balance on a tightrope by using SSVEP generated in response to phase-reversing checkerboard patterns. Leeb et al. developed Graz-BCI system for virtual reality control [17]. The user can decide how they wanted to explore the virtual apart-ment by using their motor imagery. However, both of the above BCI systems have to send EEG signal to backend personal computer to process EEG signal. Bulky and expensive EEG machines and personal computers are required for real-time EEG processing. Shyu et al. proposed a FPGA-based SSVEP BCI control system [15]. Instead of personal computer, the real-time SSVEP BCI algorithm was implemented in a FPGA board. All of the above BCI-based control systems
require the user’s active mental command to control external devices.
BSLEACS uses EEG signal as the device control signal that is suitable for any alive human, and the environmental adjustment can be controlled automatically according to the change of the user’s cognitive state. Here, only a single EEG channel is required to monitor the user’s cognitive state. It can avoid the inconvenience of other BCI-based control systems for wearing many EEG electrodes on his or her head, and also makes BSLEACS become more practicable in daily application. Moreover, different from other bulky EEG machines and personal computers, the portability of the proposed wireless physiological signal acquisition module and embedded signal processing module are more suitable for daily application. Furthermore, the cost of the embedded signal processing module is less than USD30, which can be cheaper for mass production.
VI. Conclusion
A brain computer interface-based smart living environ-mental auto-adjustment control system was proposed in this paper. The wireless physiological signal acquisition module is small enough to be embedded into a headband as a wearable EEG device, and provides the advantages of mobility and long-term EEG monitoring (over 33 h by using 1100 mA Li-ion battery). The embedded signal processing module, which provides powerful computations, was designed to recog-nize the user’s cognitive state and was also implemented as an UPnP control device. Different from other BCI-based control systems [29], BSLEACS can process EEG signal without transmitting EEG signal to backend personal computers. Such flexibility and the advantages of low power consumption and small volume of the wireless physiological signal acquisition module and embedded signal processing module are suitable for various kinds of smart applications in daily life.
Based on the unsupervised approach proposed in our previ-ous study, a real-time cognitive state detection algorithm was also implemented in the embedded signal processing module to recognize the user’s cognitive state continuously and to change the environment setting automatically when cognitive state changes. Different from other BCI-based control systems which require many EEG channels to extract sufficient EEG feature, BSLEACS only needs single EEG channel to recog-nize cognitive state by monitoring EEG signal in the location of Oz of the international 10–20 EEG system. Avoiding the in-convenience of that the user has to wear many EEG electrodes on his or her head, also makes BSLEACS become more practi-cable in daily application. For 75-trial test results, the PPV and sensitivity of BSLEACS for controlling home appliances are 70% and 81.40%, respectively. BSLEACS has been verified in a practical environment and shows that the lights/lamp can be successfully and automatically adjusted in real time based on the change of the user’s cognitive state. BSLEACS provides a novel system prototype for environmental control, and can be generalized for other applications that are implemented and constructed in an UPnP-based smart house.
References
[1] L. Zhang, H. Leung, and K. C. C. Chan, “Information fusion based smart home control system and its application,” IEEE Trans. Consumer
Electron., vol. 54, no. 3, pp. 1157–1165, Aug. 2008.
[2] S. Helal, W. Mann, H. El-Zabadani, J. King, Y. Kaddoura, and E. Jansen, “The gator tech smart house: A programmable pervasive space,” IEEE
Comput., vol. 38, no. 3, pp. 50–60, Mar. 2005.
[3] W.-H. Liau, C.-L. Wu, and L.-C. Fu, “Inhabitants tracking system in a cluttered home environment via floor load sensors,” IEEE Trans. Autom.
Sci. Eng., vol. 5, no. 1, pp. 10–20, Jan. 2008.
[4] D. H. Stefanov, Z. Bien, and W.-C. Bang, “The smart house for older persons and persons with physical disabilities: Structure, technology ar-rangements, and perspectives,” IEEE Trans. Neural Syst. Rehabilitation
Eng., vol. 12, no. 2, pp. 228–250, Jun. 2004.
[5] S. H. Kim, J. S. Kang, H. S. Park, D. Kim, and Y.-J. Kim, “UPnP-ZigBee internetworking architecture mirroring a multi-hop zigbee net-work topology,” IEEE Trans. Consumer Electron., vol. 55, no. 3, pp. 1286–1294, Aug. 2009.
[6] C.-S. Li, Y.-M. Huang, and H.-C. Chao, “UPnP IPv4/IPv6 bridge for home networking environment,” IEEE Trans. Consumer Electron., vol. 54, no. 4, pp. 1651–1655, Nov. 2008.
[7] T.-W. Jo, Y.-D. You, H. Choi, and H.-S. Kim, “A bluetooth-UPnP bridge for the wearable computing environment,” IEEE
Trans. Consumer Electron., vol. 54, no. 3, pp. 1200–1205, Aug.
2008.
[8] C. Rus, K. Kontola, I. D. D. Curcio, and I. Defee, “Mobile TV content to home WLAN,” IEEE Trans. Consumer Electron., vol. 54, no. 3, pp. 1038–1041, Aug. 2008.
[9] B. A. Miller, T. Nixon, C. Tai, and M. D. Wood, “Home networking with universal plug and play,” IEEE Commun. Mag., vol. 39, no. 12, pp. 104–109, Dec. 2001.
[10] T. Hwang, H. Park, and E. Paik, “Location-aware UPnP AV session manager for smart home,” in Proc. IEEE 1st Int. Conf. Networked Digit.
Technol., Jul. 2009, pp. 106–109.
[11] C. Yerrapragada and P. S. Fisher, “Voice controlled smart house,” in Proc. IEEE Int. Conf. Consumer Electron., Jun. 1993, pp. 154–155.
[12] P. M. Corcoran, J. Desbonnet, P. Bigioi, and I. Lupu, “Home network infrastructure for handheld/wearable appliances,” IEEE Trans. Consumer
Electron., vol. 48, no. 3, pp. 490–495, Aug. 2002.
[13] E. C. Lalor, S. P. Kelly, C. Finucane, R. Burke, R. Smith, R. B. Reilly, and G. McDarby, “Steady-state VEP-based brain computer interface control in an immersive 3-D gaming environment,” EURASIP J. Appl.
Signal Process., vol. 2005, no. 19, pp. 3156–3164, Jan. 2005.
[14] B. Rebsamen, C. Guan, H. Zhang, C. Wang, C. Teo, M. H. Ang, and E. Burdet, “A brain controlled wheelchair to navigate in familiar environments,” IEEE Trans. Neural Syst. Rehabilit. Eng., vol. 18, no. 6, pp. 290–598, May 2010.
[15] K.-K. Shyu, P.-L. Lee, M.-H. Lee, M.-H. Lin, R.-J. Lai, and Y.-J. Chiu, “Development of a low-cost FPGA-based SSVEP BCI multimedia control system,” IEEE Trans. Biomed. Circuits Syst., vol. 4, no. 2, pp. 125–132, Apr. 2010.
[16] G. Pfurtscheller and C. Neuper, “Motor imagery and direct brain-computer communication,” Proc. IEEE, vol. 89, no. 7, pp. 1123–1134, Jul. 2001.
[17] R. Leeb, F. Lee, C. Keinrath, R. Scherer, H. Bischof, and G. Pfurtscheller, “Brain–computer communication: Motivation, aim, and impact of exploring a virtual apartment,” IEEE Trans. Neural Syst.
Rehabilitat. Eng., vol. 15, no. 4, pp. 473–482, Dec. 2007.
[18] Y. Li, J. Long, T. Yu, Z. Yu, C. Wang, H. Zhang, and C. Guan, “An EEG-based BCI system for 2-D cursor control by combining Mu/Beta rhythm and P300 potential,” IEEE Trans. Biomed. Eng., vol. 57, no. 10, pp. 2495–2505, Oct. 2010.
[19] M. Cheng and S. Gao, “EEG-based cursor control system,” in Proc.
IEEE Annu. Int. Conf. Eng. Med. Biol., Oct. 1999, p. 669.
[20] W. Klimesch, “EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis,” Brain Res. Rev., vol. 29, nos. 2–3, pp. 169–195, 1999.
[21] S. Makeig, T.-P. Jung, and T. J. Sejnowski, “Awareness during drowsi-ness: Dynamics and electrophysiological correlates,” Canad. J. Exp.
Psychol., vol. 54, no. 4, pp. 266–273, 2000.
[22] M. A. Schier, “Changes in EEG alpha power during simulated driving: A demonstration,” Int. J. Psychophysiol., vol. 37, no. 2, pp. 155–162, 2000.
[23] C. T. Lin, R. C. Wu, S. F. Liang, T. Y. Huang, Y. C. Chen, and T. P. Jung, “Estimating driving performance based on EEG spectrum analysis,”
EURASIP J. Appl. Signal Process., vol. 2005, no. 19, pp. 3165–3174,
Mar. 2005.
[24] C. T. Lin, R. C. Wu, S. F. Liang, W. H. Chao, Y. J. Chen, and T. P. Jung, “EEG-based drowsiness estimation for safety driving using independent component analysis,” IEEE Trans. Circuits Syst. I, vol. 52, no. 12, pp. 2726–2738, Dec. 2005.
[25] T.-P. Jung, S. Makeig, M. Stensmo, and T. J. Sejnowski, “Estimating alertness from the EEG power spectrum,” IEEE Trans. Biomed. Eng., vol. 44, no. 1, pp. 60–69, Jan. 1997.
[26] S. Makeig and T.-P. Jung, “Changes in alertness are a principal compo-nent of variance in the EEG spectrum,” Neuroreport, vol. 7, no. 1, pp. 213–216, 1995.
[27] N. R. Pal, C. Y. Chuang, L. W. Ko, C. F. Chao, T. P. Jung, S. F. Liang, and C. T. Lin, “EEG-based subject and session-independent drowsiness detection: An unsupervised approach,” EURASIP J. Adv. Signal Process., vol. 2008, no. 519480, p. 11, 2008.
[28] K. V. Mardia, “Mardia’s test of multinormality,” in Encyclopedia of
Statistical Sciences, vol. 5, S. Kotz and N. L. Johnson, Eds. New York:
Wiley, 1985, pp. 217–221.
[29] Y. S. Kim, H. J. Baek, J. S. Kim, H. B. Lee, J. M. Choi, and K. S. Park, “Helmet-based physiological signal monitoring system,” Eur. J. Appl.
Chin-Teng Lin (S’88–M’91–SM’99–F’05) received the B.S. degree from National Chiao Tung Univer-sity (NCTU), Hsinchu, Taiwan, in 1986, and the Masters and Ph.D. degrees in electrical engineering from Purdue University, West Lafayette, IN, in 1989 and 1992, respectively.
He is currently the Provost with NCTU. Dr. Lin was elevated to a fellow of the IEEE for his contributions to biologically inspired information systems in 2005. He served on the Board of Gover-nors of the IEEE Circuits and Systems Society from 2005 to 2008, the IEEE Systems, Man, Cybernetics Society from 2003 to 2005, and the IEEE Computational Intelligence Society from 2008 to 2010. He was the Chair of the IEEE Taipei Section from 2009 to 2010.
Bor-Shyh Lin (S’01–M’04) received the B.S. de-gree from National Chiao Tung University (NCTU), Hsinchu, Taiwan, in 1997, and the M.S. and Ph.D. degrees in electrical engineering from National Tai-wan University, Taipei, TaiTai-wan, in 1999 and 2006, respectively.
He is currently an Assistant Professor with the Institute of Imaging and Biomedical Photon-ics, NCTU. His current research interests include biomedical circuits and systems, biomedical signal processing, and biosensors.
Fu-Chang Lin received the B.S. and M.S. degrees in electrical engineering from National Cheng-Kung University, Tainan, Taiwan, in 1995 and 1997, re-spectively. He is currently pursuing the Ph.D. degree with National Chiao Tung University, Hsinchu, Tai-wan.
He is currently a Senior Application Constant with Synopsys Taiwan, Ltd., Hsinchu. His current research interests include artificial neural networks and biomedical signal processing.
Che-Jui Chang received the B.S. degree in electri-cal engineering from Chang-Gung University, Gueis-han, Taiwan, in 2007, and the M.S. degree in electrical and control engineering from National Chiao Tung University (NCTU), Hsinchu, Taiwan, in 2009.
His current research interests include brain– computer interface designs, biomedical amplifier cir-cuit designs, and embedded system processing.
![Fig. 4. Flowchart of real-time cognitive state detection algorithm. the changes of drowsiness and alertness [20]–[26]](https://thumb-ap.123doks.com/thumbv2/9libinfo/7513082.117933/3.1263.112.607.115.394/flowchart-cognitive-state-detection-algorithm-changes-drowsiness-alertness.webp)
