4.1 Introduction
The growing number of traffic accidents had become a serious concern to the society in recent years. Accidents caused by driver’s drowsiness behind the steering wheel have a high fatality rate because of the marked decline in the driver’s abilities of perception, recognition and vehicle control abilities while being sleepy. For instance, the National Highway Traffic Safety Administration (NHTSA) conservatively estimates that 100,000 police-reported crashes are the direct result of driver fatigue each year. This results in an estimated 1,550 deaths, 71,000 injuries and
$12.5 billion in monetary losses. The National Science Foundation (NSF) also reported in 2002 that 51% of adult drivers had driven a vehicle while feeling drowsy and 17% had actually fallen asleep. Preventing traffic accidents caused by drowsiness is highly desirable but requires techniques for continuously monitoring, estimating, and predicting the level of alertness of drivers and delivering effective feedbacks to maintain their maximum performance (Pilutti and Ulsoy, 1999). Therefore, we demonstrated an EEG-based drowsiness estimation method in long-term driving in this chapter. In our past research, we had already found the parietal and occipital brain sources were highly correlated with drowsiness via ICA-based signal process (Lin et al., 2005 & 2006). The method combined ICA, EEG log band power spectrum, correlation analysis, and linear regression models to indirectly estimate driver’s drowsiness level. Here, we use the same method to estimate subject drowsiness level, except we employ MEMS sensors rather than conventional wet ones to acquire continuous EEG data to demonstrate the potential uses of the MEMS sensors during long and routine recordings in the VR-based dynamic driving environment.
In realistic environment, it is not humane and convenient to acquire the EEG
signal with skin preparation even it had better performance. Using electrolytic gel is not only uncomfortable and inconvenience, but also can cause itchy feeling, and sometimes make skin red and swollen during long-term EEG-measurement. Hence, we also have developed an EEG-based drowsiness estimation algorithm that consisted of signal acquisition, power spectrum estimation, Principal Component Analysis (PCA)-based signal process, and multivariate linear regression to estimate the driver’s drowsiness level in the VR-based dynamic driving environment.
4.2 Experimental Setup
Ten subjects (ages from 20 to 40 years, 29.8±5.9 years old) participated in the VR-based highway driving experiments. To maximize the opportunities to get valuable data for our study, all driving experiments were conducted in the early afternoons after lunch. Statistics (Jung et al., 1997; Makeig and Jung, 1995) showed that people often get drowsy within one hour of continuous driving during these periods, indicating that drowsiness is not necessarily caused by long driving-hours.
The driving speed is fixed as 100 km/hr and the car is randomly and automatically drifted away from the center of the cruising lane to mimic the consequences of a non-ideal road surface. On the first day, participants were told of the general features of the driving task, completed necessary informed consent material, and then started with a 15 to 30-minute practice to keep the car at the center of the cruising lane by maneuvering the car with the steering wheel. Subjects reported this amount of practice to be sufficient to reach a performance asymptote on the task. After practicing, participants wore wired EEG cap with electrodes and began 1-hour lane-keeping driving task. Participants returned on a different day to complete the other 1-hour driving session for cross-session test. While the subject was alert in the experiment, his/her response time was short and the deviation of the car was small; otherwise the
subject’s response time and the car deviation would be slow and long. In this driving experiment, the VR-based freeway scene showed only one car driven on the road without any other event stimuli to simulate a monotonous and unexciting task that could easily make drivers fallen asleep.
The flowchart of data analysis for estimating the level of drowsiness based on the EEG power spectrum is shown in Figure 4-1. For each subject, after collecting EEG signals and driving deviation in 1-hour simulated driving session, the EEG data were first preprocessed using a simple low-pass filter with a cut-off frequency of 50 Hz to remove the line noise and other high-frequency noise. Then, we applied ICA to decompose EEG-signals into temporally independent stationary sources and calculated the moving-averaged log power spectra of all ICA components.
33-channel
Figure 4-1: Flowchart of the drowsiness detection system
In general, the drowsiness-related regions are mainly in the parietal and occipital lobes. If we acquire the EEG signal from these haired regions of the hindbrain to detect drowsiness level in realistic environment, it is uncomfortable and inconvenient
to acquire the EEG signal from the conventional gel-based sensors on the scalp. To overcome this limitation, we employ the self-stabilized MEMS sensor to replace the conventional ones. The self-stabilized MEMS sensor is expected to circumvent the high impedance characteristics of the stratum corneum (SC) and then skin preparation and electrolytic gel application are not required. Due to the limitation of the current MEMS technology, self-stabilized MEMS sensor is not sufficient to penetrate human hairs to contact stratum germinativum (SG) or even SC. The hair elasticity also makes it difficult to fix the sensor on the scalp. Therefore, the self-stabilized MEMS sensors in this study are placed at non-hairy sites, such as Fp1 and Fp2 on the forehead to on-line estimate the driver’s drowsiness level in real-world application.
4.2.1 Experimental Environment
A VR-based dynamic driving simulation environment is designed and built for interactive driving experiments. It includes three major parts: (a) the 3D highway driving scene based on the virtual reality technology, (b) the driving cabin simulator mounted on a 6-DOF dynamic Stewart motion platform (as shown in Figure 2-1 (a) &
(b)) and (c) the EEG acquisition system with 13-channel sensors (as shown in Figure 4-2).
Figure 4-2: A VR-based dynamic driving environment for interactive driving experiments
Acquired EEG signals are analyzed by power spectral density analysis, PCA-based signal processing, and linear regression model to estimate subject’s driving performance as shown in Figure 4-3. The subject’s performance is defined as the deviations between the center of the vehicle and the center of the cruising (3rd) lane. While the subject was alert in the experiment, his/her response time was short and the deviation of the car was small; otherwise the subject’s response time and the car deviation would be slow and long. In this driving experiment, the VR-based freeway scene showed only one car driven on the road without any other event stimuli to simulate a monotonous and unexciting task that could easily make drivers fallen asleep. These physiological and behavioral data are continuously and simultaneously measured and recorded by the WTK program and the acquisition system.
Figure 4-3: Flowchart for processing the EEG signals. (1) A low-pass filter was used to remove the line noise and higher frequency (>50Hz) noise. (2) Moving-averaged spectral analysis was used to calculate the EEG log power spectrum of each channel advancing at 2-sec steps. (3) Two EEG channels with higher correlation coefficients between subject’s driving performance and EEG log power spectrum were further selected. (4) PCA was trained and used to decompose selected features and extract the representative PCA-components as the input vectors for the linear regression models (LRM). (5). LRMs were trained in one training session and used to continuously estimate the individual subject’s driving performance in the testing session.
Noise
The time series of recorded driving performance were smoothed using a causal 90-s square moving-averaged filter (Sterlade, 1993; Treisman, 1984) advancing at 2-sec steps to eliminate variance at cycle lengths shorter than 1-2 minutes since the driving performance tended to vary irregularly with cycle lengths of 4 minutes and longer (Jung et al., 1997; Makeig and Jung, 1995). The EEG data recorded by the MEMS (or wet) electrode pairs were first preprocessed using a simple low-pass filter with a cut-off frequency of 50 Hz to remove the line noise and other high-frequency noise. After moving-average power spectral analysis, we obtained EEG log power spectrum time series for the 5 MEMS (or wet) electrodes. Then, we applied Karhunen-Loeve Principal Component Analysis (PCA) to the resultant EEG log spectrum betweem 1 and 40 Hz to extract the directions of largest variance for each session. Projections of the EEG log spectral data (PCA components) along the subspace formed by the eigenvectors corresponding to the largest 50 eigenvalues were used as inputs to a multiple linear regression model (Chatterjee, 1986) for each individual subject to estimate the time course of his/her driving error (Bishop, 1995).
Each model was trained using the features extracted only from the training session and tested on the data from a separate testing session.
4.2.2 EEG Data Acquisition
Figure 4-4 shows the placements of five MEMS/conventional sensor pairs at the frontal locations. The first and fifth MEMS sensors are placed at Fp1 and Fp2 according to the international 10-20 electrode placement system (Thakor, 1999). We also placed three additional MEMS sensors evenly spaced between these two MEMS sensors and labeled them as MEMS2, MEMS3, and MEMS4. Corresponding conventional wet electrodes were placed 1 cm above the MEMS EEG sensors (cf.
Figure 4-2). The contact impedance between the MEMS/wet electrode and skin was
calibrated to be less than 5 kΩ. The EEG was recorded from these 5 MEMS and 5 wet electrodes, referenced against linked mastoids (A1, A2) by the Neuroscan NuAmps Express system (Compumedics Ltd., VIC, Australia, as shown in Figure 4-2), band-passed between 0.5 and 100Hz with a 60Hz notch filter, and recorded with 16-bit quantization level at a sampling rate of 500 Hz and then down-sampled to 250 Hz for the simplicity of data processing.
Figure 4-4: Forehead positions of conventional wet electrodes (circle) and MEMS EEG sensors (square)
4.2.3 Lane Keeping Driving Task
In the long-term driving, the car cruised with a fixed velocity of 100 km/hr on the VR-based highway scene and it was randomly drifted either to the left or to the right away from the cruising position with a constant velocity. The subjects were instructed to steer the vehicle back to the center of the cruising lane as quickly as possible. Figure 4-5 shows the time course of a typical deviation event that embedded in the long-term lane-keeping driving task.
Figure 4-5: An example of the deviation event. The car cruised with a fixed velocity of 100 km/hr on the VR-based highway scene and it was randomly drifted either to the left or to the right away from the cruising position with a constant velocity. The subjects were instructed to steer the vehicle back to the center of the cruising lane as quickly as possible.
Firstly, we need to quantify the volunteer’s drowsiness level in this experiment.
When subjects fall drowsy, they often exhibits relative inattention to environments, eye closure, less mobility, failure to motor control and making decision (Brookhuis et al., 2003). Hence, the vehicle deviations were defined as the subject’s drowsiness index. The VR-based four-lane straight highway scene was applied in the experiment.
Figure 4-6 shows an example of the driving performance represented by the vehicle deviation trajectories.
Figure 4-6: An example of the driving performance that represented by the digitized vehicle deviation trajectories
4.3 Experimental Results
In order to demonstrate the potential applications of the MEMS electrodes for long and routine EEG recording in operational environments, we investigated the quality of the EEG signals recorded by the MEMS EEG sensors placed at Fp1 and Fp2 for estimating subjects’ drowsiness in a sustained-attention driving experiment.
The EEG signals recorded by five MEMS sensors were fed into an EEG-based drowsiness estimation system (Lin et al., 2005) as shown in Figure 4-3 to estimate driver’s driving performance, an indication of driver’s drowsiness level.
4.3.1 Comparison Performance between MEMS-based and Standard Wet Sensor Figure 4-7 plots the raw EEG signals measured by the proposed MEMS EEG sensors and wet electrode pairs (only the leftmost and rightmost MEMS/wet pairs are shown here). As can be seen, the EEG signals recorded by the MEMS sensors are extremely comparable to those obtained by the corresponding wet electrodes.
Figure 4-7: Raw EEG Data Recording by MEMS sensors and Standard Wet Sensors
Figure 4-8 over-plots the EEG power spectra of 5 MEMS/wet sensor pairs. As it can been seen, they are extremely similar especially in low frequency bands (1-30Hz),
indicating that the signals obtained by proposed MEMS EEG sensors matched well with the EEG signals recorded by the conventional wet electrodes.
4.3.2 Correlation Analysis Results
The correlation coefficients between the subject’s driving performance and the log power spectra of all ICA components at each frequency band are further evaluated to form a correlation spectrum. The normalized log sub-band power spectra of top two ICA components with the highest correlation coefficients in some critical bands are
(a) (b)
(e)
(d) (c)
Figure 4-8: The EEG power spectra of 5 MEMS / Wet sensor pairs (a) MEMS1 / Wet1 sensor pair (b) MEMS2 / Wet2 sensor pair (c) MEMS3 / Wet3 sensor pair (d) MEMS4 / Wet4 sensor pair (e) MEMS5 / Wet5 sensor pair
further selected as the input features of the conventional linear regression model to estimate the individual subject’s drowsiness level. Figure 4-9 shows the correlation of driving performance and EEG power spectra from the different two subjects. We can easily find that alpha band (8-13 Hz) is highly correlated with drowsiness. These results are consistent with the related studies (Jung et al., 1997; Makeig and Jung, 1996) and alpha band will be a good indicator to detect drowsiness level.
Figure 4-9: Correlation of driving performance and EEG power spectra from the different two subjects
Table 4-1 shows the scalp topographies and spectrum correlation of each subject between driving performance and ICA power spectra of the top two ICA components.
The correlations are particularly strong at central and posterior areas, which are consistent with related studies in the driving experiments (Makeig and Inlow, 1993;
Makeig and Jung, 1996). The relatively high correlation coefficients near θ-band or α-band may be suitable for drowsiness estimation, as the subject’s cognitive state might fall into stage one of the non-rapid eye movement sleep. As can been seen that the best drowsiness-correlated components (best matching) differ in each subject, in general their scalp topographies are all within the ambit of central lobe to occipital lobe.
Table 4-1: Correlation spectra between smoothed driving errors and ICA power spectra of first 2 ICA components of each subject
SUBJECT1 SUBJECT2 SUBJECT3 SUBJECT4 SUBJECT5
Figure 4-3 shows the flowchart of the EEG-based drowsiness estimation algorithm. EEG signals are analyzed by power spectral density analysis, PCA-based signal processing, and linear regression model to estimate subject’s driving performance. Figures 4-10 to 4-13 compare the drowsiness-estimation performance obtained by MEMS EEG sensors and wet electrodes in the sustained-attention driving tasks. They show the minute scale fluctuation of the each driver’s driving performance index in 1-hour driving session. In each figure, the blue and red traces represent the acquired and estimated driving errors, respectively, and all of these figures are the results on the testing data.
Figures 4-10(a) and (b) show the estimated driving error in Session #2 of Subject 1 using the EEG signals recorded by the conventional wet electrodes and the MEMS EEG sensors, respectively. The estimators were trained with the EEG signals from Session #1 to estimate the driving error of Session #2 of Subject 1 (the blue traces in Figure 4-10). Conversely, Figures 4-11 (a) and (b) show the estimated driving error of Subject 1 using EEG data from Session #2 as the training dataset and those from
Session #1 as the testing dataset.
(a) (b)
Figure 4-10: Estimated and actual driving error of Session #2 of Subject 1 using the EEG signals recorded by (a) the conventional wet electrodes and (b) MEMS EEG sensors, respectively. The estimators were trained with the EEG signals from Session
#1 to estimate the driving error of Session #2 (the blue traces).
(a) (b)
Figure 4-11: Estimated and actual driving error of Session #1 Subject 1 using the EEG signals recorded by (a) the wet and (b) MEMS electrodes, respectively. The estimators were trained with the EEG signals of Session #2 to estimate the driving error of Session #1 (the blue traces).
Similarly, Figures 4-12 and 4-13 show estimated and actual driving errors made by another subject (Subject 2). Table 4-2 shows the comparison results of correlation coefficient between the actual and estimated driving error time series using MEMS sensors and conventional wet sensors for 5 different subjects.
(a) (b)
Figure 4-12: Estimated and actual driving error of Session #2 of Subject 2 using the EEG signals recorded by (a) the wet and (b) MEMS electrodes, respectively. The estimators were trained with the EEG signals from Session #1 to estimate the driving error of Session #2 (the blue traces).
(a) (b)
Figure 4-13: Estimated and actual driving error of Session #1 of Subject 2 using the EEG signals recorded by (a) the wet and (b) proposed electrodes, respectively. The estimators were trained with the EEG signals of Session #2 to estimate the driving error of Session #1 (the blue traces).
Table 4-2: Performance of testing patterns for electrode-skin-electrode impedance (ESEI) measurement
Session 1 estimates Session 2 Session 2 estimates Session 1 Wet sensor MEMS sensor Wet sensor MEMS sensor
Subject 1 0.95 0.96 0.90 0.92
Subject 2 0.96 0.96 0.85 0.88
Subject 3 0.82 0.83 0.84 0.86
Subject 4 0.92 0.94 0.81 0.83
Subject 5 0.85 0.86 0.85 0.88
As can be seen in Figures 4-10 to 4-13 and Table 4-2, the estimated driving errors based on EEG spectra matched well with the actual errors, consistent with our recent report in the same driving tasks using whole-head 32-channel EEG (Lin et al.
2005). The results demonstrated the feasibility of accurately estimating subject task performance based on EEG signals collected from the frontal non-hairy sites.
Furthermore, the estimation accuracy based on the EEG collected by the MEMS sensor is comparable to that based on the signals collected by conventional wet sensor, indicating the feasibility of using MEMS sensors that do not require skin preparation or conductive pastes to acquire high-quality EEG signals in operational environments.
In Table 4-2, the correlation coefficient between the two time series (using Session 1 to estimate Session 2) is r = 0.96 of subject 1, r = 0.96 of subject 2, r = 0.83 of subject 3, r = 0.94 of subject 4, and r = 0.86 of subject 5, in the testing session 2.
The correlation coefficient between the two time series (using Session 2 to estimate Session 1) is r = 0.92 of subject 1, r = 0.88 of subject 2, r = 0.86 of subject 3, r = 0.83 of subject 4, and r = 0.88 of subject 5, in the testing session 1. The average performance of the estimation system can be reached over 90% by using five self-stabilized MEMS sensors at forehead area.
4.4 Discussion
The lack of availability of EEG monitoring system without use of conductive gels applied to the scalp has long thwarted both military and civilian applications of EEG monitoring in the workplace. In this chapter, MEMS sensors with microprobe array structure bring EEG monitoring to the operational environment without requiring scalp gel or other scalp preparation. Our experimental results demonstrated that the MEMS sensors have advantages in electrode/skin interface impedance, signal intensity and size over the conventional wet sensors. Furthermore, we find that alpha band (8-13 Hz) is highly correlated with drowsiness. It means alpha band will be a good indicator to detect drowsiness level. Hence, we employed the MEMS sensor to collect continuous EEG signals in realistic 1-hour sustained-attention experiments to test the feasibility of using MEMS sensors in operational environments. The EEG-based drowsiness estimation system consists of the MEMS sensor array, power spectrum estimation, PCA-based EEG signal processing, and multivariate linear regression to estimate driver’s drowsiness level in a VR-based dynamic driving
The lack of availability of EEG monitoring system without use of conductive gels applied to the scalp has long thwarted both military and civilian applications of EEG monitoring in the workplace. In this chapter, MEMS sensors with microprobe array structure bring EEG monitoring to the operational environment without requiring scalp gel or other scalp preparation. Our experimental results demonstrated that the MEMS sensors have advantages in electrode/skin interface impedance, signal intensity and size over the conventional wet sensors. Furthermore, we find that alpha band (8-13 Hz) is highly correlated with drowsiness. It means alpha band will be a good indicator to detect drowsiness level. Hence, we employed the MEMS sensor to collect continuous EEG signals in realistic 1-hour sustained-attention experiments to test the feasibility of using MEMS sensors in operational environments. The EEG-based drowsiness estimation system consists of the MEMS sensor array, power spectrum estimation, PCA-based EEG signal processing, and multivariate linear regression to estimate driver’s drowsiness level in a VR-based dynamic driving