1.1 Motivation
Drivers’ fatigue has been implicated as a causal factor in many accidents.
Preventing accidents caused by drowsiness has become a major focus of active safety driving in recent years. During the past years, driving safely has received increasing attention of the publics due to the growing number of traffic accidents because of the marked decline in the drivers’ abilities of perception, recognition and vehicle control abilities while sleepy. Therefore, it requires an optimal human estimation system to online continuously detect drivers’ cognitive state related to abilities in perception, recognition, and vehicle control. The difficulties in developing such a system are lack of significant index for detecting drowsiness and complicated noise interferences in a realistic and dynamic driving environment. Development of the drowsiness monitoring technology for preventing accidents behind the steering wheel has become a major interest in the field of safety driving. Thus, developing accurate and non-invasive real-time driver drowsiness monitoring system would be highly desirable, particularly if this system can be further integrated into an automatic warning system.
It is known that abundant information on physiological changes such as eye activity measures, heart rate variability (HRV), or particularly, the electroencephalogram (EEG) activities can relate with drowsiness (Vuckovic et al., 2002; Roberts et al., 2000). Previous studies (Stern et al., 1994; McGregor and Stern, 1996) showed that the eye blink duration and the blink rate typically increases while blink amplitude decreases as function of the cumulative time, and the saccade frequencies and velocities of electrooculogram (EOG) decline when people get drowsy. Although approaches based on EOG signals showed that eye-activity
variations were highly correlated with the human fatigue and can accurately and quantitatively estimate alertness levels, the step size (temporal resolution) of those eye-activity based methods is relatively long (about 10 seconds) to track slow changes in vigilance (Orden et al., 2000). Contrarily, the step size of the EEG-based methods can be reduced to about 2 seconds to track second-to-second fluctuations in the subject’s performance (Orden et al., 2001; Jung et al., 1997; Makeig and Jung, 1996).
Since the computer power becomes faster and faster, it is practicable and appealing to know what information about human cognitive state and behavior are available through analyzing complex EEG signals. Hence, we constructed a virtual-reality (VR) based highway-driving environment to study drivers’ cognitive changes during a long-term driving. A lane-keeping driving experiment was designed to indirectly quantify the driver’s drowsiness level and a drowsiness estimation system combining the EEG power spectrum analysis, the principle component analysis (PCA) and the linear regression model was developed. Independent Component Analysis (ICA) was used in the similar experiments (Comon, 1994; Girolami, 1998; Lee et al., 1999) to locate the optimal electrode placements for each individual. A total of 10 frequency bands in 2 ICA components are selected and fed to the linear regression models to estimate driver’s performance.
1.2 Statement of the Problem
Biomedical signal monitoring systems have been rapidly advanced with electronic and information technologies in recent years. Electroencephalogram (EEG) recordings are usually obtained by placing electrodes on the scalp with a conductive gel or paste, each of which is attached to a wire that is then connected to an external signal acquisition device. The tethering caused by this method of recording prohibits experiments in real operational environments. Furthermore, most of the existing
physiological signal monitoring systems can only record the signals without the capability of automatic analysis. In this study, we develop a portable and real-time Brain Computer Interface (BCI) that can acquire and analyze EEG signals in real-time to monitor human physiological as well as cognitive states and, in turn, provide warning signals to the users when needed. In order to widely application in the realistic environment, we should consider the effects of kinesthetic perception on the BCI system. Therefore, we first constructed the unique virtual reality-based dynamic driving environment to investigate EEG activation on kinesthetic perception and under different cognitive states. Then, the neural human machine interface/interaction in realistic environment will be well-established by combining the findings of EEG activation with the portable and real-time BCI system.
Kinesthetic perception is one of the most important sensations to human beings.
The vestibular system thus plays an important role in our lives. We usually overlook the contributions of the vestibular system to our lives, simply because it doesn’t give us the sense of this vivid and harmonic world the way our eyes and ears do. However, we would not have a complete sensation without the perception of motion. Vestibular system is one of the most important sensory apparatus for detecting the perception of motion. One of the most experienced kinesthetic perceptions in our life is the motion associated with driving. Almost all of the existing EEG correlated research studies of perceiving kinesthetic stimuli focus on the brain dynamics of the subjects receiving visual and/or auditory stimulus, very few one focus on the subject perception of kinesthetic stimulus such as car drivers, airplane pilots, etc.
After the investigation of kinesthetic perception, we would use the portable real-time BCI system as the base platform and increase some actual functions such as low-power consumption for portability and high computational capability to process EEG signal. The portable and real-time BCI system consists of a 4-channel bio-signal
acquisition/amplification module, a wireless transmission module, a dual-core signal processing unit and a host system for display and storage. The embedded dual-core processing system with multi-task scheduling capability was proposed to acquire and process the input EEG signals in real-time. In addition, the wireless transmission module, which eliminates the inconvenience of wiring, can be switched between radio frequency (RF) and Bluetooth according to the transmission distance. Finally, the real-time EEG-based drowsiness monitoring and warning algorithms were implemented and integrated into the system to close the loop of the BCI system. The practical online testing demonstrates the feasibility of using the proposed system with the ability of real-time processing, automatic analysis and on-line warning feedback in real-world operation and living environments.
Based on the neural human machine interface/interaction in the future work, we will perform more realistic experiments on the unique VR-based dynamic driving environment which can simulate vehicle driving and 3D surrounded scenes to investigate the EEG correlated activities of the driver (such as, distraction, carsickness (motion sickness), etc.). It will widen the fundamental biomedical and brain science research and spawns new industry opportunities to provide the solutions of real-life problems.
1.3Organization of Dissertation
This dissertation is organized as follows. Chapter 2 describes the virtual reality-based dynamic driving environment, electroencephalogram (EEG) signal acquisition system, Independent Component Analysis (ICA), event-related potential (ERP), and event-related spectral perturbation (ERSP). Chapter 3 explores EEG activation of kinesthetic perception under our unique virtual-reality-based dynamic driving environment. Chapter 4 investigates EEG activation under different cognitive
states and develops drowsiness estimation technology by using Micro Electro Mechanical Systems sensor (MEMS sensor). Based on the drowsiness estimation technology, Chapter 5 develops the portable brain computer interface to real-time detect drivers’ drowsiness. At last, we make some conclusions in Chapter 6.