基於多重生理訊號參數之即時無線瞌睡偵測系統

國 立 交 通 大 學

生醫工程研究所

碩

碩

碩

碩

士

士

士

士

論

論

論

論

文

文

文

文

基於多重生理訊號參數之

即時無線瞌睡偵測系統

Real-time Wireless System based on Multiple Bio-signal

Parameters for Drowsiness Detection

研 究 生：劉育航

指導教授：林進燈 教授

中

中

中

基於多重生理訊號參數之即時無線瞌睡偵測系統

Real-time Wireless System based on Multiple Bio-signal Parameters for

Drowsiness Detection

研 究 生：劉育航 Student：Yu-Hang Liu

指導教授：林進燈 Advisor：Chin-Teng Lin

國 立 交 通 大 學

生 醫 工 程 研 究 所

碩 士 論 文

A Thesis

Submitted to Institute of Biomedical Engineering College of Computer Science

National Chiao Tung University in partial Fulfillment of the Requirements

for the Degree of Master

in

Computer Science

July 2010

Hsinchu, Taiwan, Republic of China

Real-time Wireless System based on Multiple Bio-signal

Parameters for Drowsiness Detection

Student: Yu-Hang Liu

Advisor: Dr. Chin-Teng Lin

Institute of Biomedical Engineering

College of Computer Science

National Chiao Tung University

Abstract

In recent years, traffic accident is one of the critical reasons to cause deaths of drivers. Drivers’ drowsiness has been implicated as a causal factor in many accidents because of the marked decline in drivers’ perception of risk and recognition of danger, and diminished vehicle handling abilities. Consequently, if the mental state of drivers could be real-time monitored, drowsiness detection and warning could effectively avoid disasters such as vehicle crashes in working environments. Some previous researches used non-physiological method, as eye closure with CCD image tracking, such as the pupil recognition, blink detection or identification of the drivers head shaking frequency. However, for CCD image tracking, users couldn’t move for free, and the images detecting performance were easily be interfered by external flash light. Other studies used physiological parameters to increase the accuracy of drowsy detection, like pulse wave analysis with neural network, electrooculography (EOG), electromyography (EMG), and electroencephalogram (EEG) measurement. In this study, we proposed a real-time wireless system for drowsiness detection. A wearable, wireless and real-time bio-signal acquisition system was designed for long-term

monitoring. In the other hand, not only EEG but also EOG signals were acquired by our system to increase the accuracy of drowsiness detection. Furthermore, an algorithm of drowsiness detection was also proposed to reduce the computation complexity, and was implemented in a portable DSP module with bio-feedback as bio-stimulator or buzzer. In order to estimate the level of drowsiness, a lane-keeping driving experiment was designed and the drowsiness level of drivers was indirectly assessed by the reaction time under Virtual Reality Driving Simulation Environment. The advantage of this unsupervised algorithm can remove the differences between individual and environment in different people or measurements. For the purpose of verifying the accuracy and feasibility of our proposed unsupervised algorithm, drowsiness status estimated by driving performance was compared with the results obtained by our proposed unsupervised algorithm. The results of comparison showed that our algorithm can detect driver’s drowsiness status precisely. In addition, our system can be successfully applied in practice to prevent traffic accidents caused by drowsy driving.

KEYWORD: drowsiness detection, electroencephalogram,electrooculography, portable bio-signal acquisition system, DSP module, Virtual Reality Driving Simulation Environment, driving performance, unsupervised algorithm Acknowledgements

基於多重生理訊號參數之即時無線瞌睡偵測系統

基於多重生理訊號參數之即時無線瞌睡偵測系統

基於多重生理訊號參數之即時無線瞌睡偵測系統

基於多重生理訊號參數之即時無線瞌睡偵測系統

學生：劉育航

指導教授：林進燈 博士

國立交通大學生醫工程研究所

摘要

摘要

摘要

摘要

近年來，交通意外是一個造成駕駛死亡的至關重要原因，其中駕駛者的精神 狀況不佳所造成車禍意外佔了絕大多數比例，所以開車駕駛瞌睡監控問題是我們 嘗試克服之處，試著以人為方式來減少車禍發生。近年來相關的開車監控研究引 進了生理參數來做為開車即時瞌睡狀況的比較依據，如心電圖、眼電波圖、肌電 圖或腦波圖等，較影像辨識來得直接與精確，使用者可以不必受影像定位之問題 影響，本論文即針對生理參數中之腦波以及眼電波參數做進一步的探討。我們設 計了一套無線可攜式的多重生理訊號擷取系統以及包含生理回饋機制如電刺激 器等的數位訊號處理平台，再搭配非監督式分析演算法來做即時的瞌睡判斷。使 用非監督式演算法的優勢在於可移除掉不同人、不同次測量中個別跟環境差異 性。本論文藉由虛擬實境模擬環境所記錄下開車偏移量來當作瞌睡程度的參考， 並與所發展的非監督式分析法的相互比對關係來證明此演算法對瞌睡程度偵測 的功效與可行性，最後實現在數位訊號處理平台上。經由實際測試，可以成功在 駕駛者有睡意時，利用電刺激器或是警示音提醒駕駛保持清醒，確保開車時的安 全。

關鍵字: 瞌睡監控、腦波圖、眼電波圖、無線可攜式生理訊號擷取系統、數位訊 號處理平台、虛擬實境模擬環境、開車偏移量、非監督式分析法

誌謝

誌謝

誌謝

誌謝

本論文的完成，首先要感謝指導教授林進燈博士這兩年來的悉心指導，讓我 學習到許多寶貴的知識，在學業及研究方法上也受益良多。另外也要感謝口試委 員們的建議與指教，使得本論文更為完整。 其次，感謝實驗室的學長林伯昰在研究上的指導。同學智賢、璽文、佩瑄、 聖翔、敬婷、佳鈴，在過去兩年研究生活中同甘共苦、相互扶持，及學姐依伶、 學長家達、孟修、煒忠、寓鈞、哲睿、有德、家欣、怡然與倫德、學弟妹們俊甫、 麒宇與琬茹，在研究過程中所給我的鼓勵與協助。尤其是哲睿學長、依伶學姐及 伯昰學長，在研究理論及程式技巧上給予我相當多的幫助與建議，亦師亦友，讓 我獲益良多。也同樣感謝實驗室助理在許多事務上的幫助。 感謝我的父母親對我的教育與栽培，並給予我精神及物質上的一切支援，使 我能安心地致力於學業，也要感謝你們對我不斷的關心與鼓勵。還要謝謝我的妹 妹以及 Angel 和圓圓的陪伴，使我回家時的生活充滿歡笑。此外，我也很感謝我 的朋友柏任、宗穎、念平、哲羽以及可涵，同樣謝謝你們的鼓勵以及支持，讓我 在沮喪或低潮時重新有動力，使我能全心投入論文之中，真心的感謝你們。 謹以本論文獻給我的家人及所有關心我的師長與朋友們。

Chapter 1 Introduction...1

1.1 Drowsiness Detection ...1

1.2 Previous Research ...2

1.3 Motivation...7

1.4 Organization of Thesis ...9

Chapter 2 Material and Method...10

2.1 EEG Signal Acquisition ... 11

2.2 EOG Signal Acquisition...13

2.3 Virtual Reality Driving Simulation Environment ...14

2.4 EEG Preprocessing ...17

2.5 EOG Preprocessing...18

2.6 Unsupervised Analysis of EEG and EOG...19

Chapter 3 Hardware Frameworks ...23

3.1 System Overview ...23

3.2 Portable Bio-signal Acquisition System ...24

3.2.1 Front-End Filter Circuit ...26

3.2.2 Analog to Digital Converter and Digital Controller ...29

3.2.3 Power Management ...36

3.2.4 Wireless Transmission ...38

3.3 DSP Module...39

3.3.1 DSP Framework ...39

3.3.2 The Circuit of Stimulator ...41

3.4 Hardware System Specification ...43

Chapter 4 Unsupervised Approach ...46

4.1 Driving Performance...46

4.2 Construction of the Alertness Model ...48

4.3 Computation of the Deviation from the Subject ...51

4.4 Computation of the Blink Duration from the Subject...52

Chapter 5 Results and Discussion ...55

5.1 Performance of Portable Bio-signal Acquisition System and Bio-stimulator of DSP Module...55

5.2 Driving Performance and Unsupervised Analysis ...60

5.2.1 Results of Unsupervised Analysis...60

5.2.2 Relationship between Driving Performance and Unsupervised Analysis ....64

5.2.4 Test and Verify the Optimized Threshold ...74

5.2.5 Comparison between MDT/MDA Only and with Blink Duration...75

5.2.6 DSP Module Programming ...77

Chapter 6 Conclusions and Future Works ...81

6.1 Conclusions...81

List of Figures

Fig. 1-1: The role of driver status monitor [13] ...3

Fig. 1-2: Flowchart of EEG processing in drowsy estimation system [19] ...6

Fig. 1-3: Scan NuAmps Express system (Compumedics Ltd., VIC, Australia) ...6

Fig. 2-1: A typical BCI system architecture...10

Fig. 2-2: International 10-20 system ...13

Fig. 2-3: The recommended derivation of EOG ...14

Fig. 2-4: The overview of surrounded VR scene.. ...15

Fig. 2-5: The digitized highway scene.[56]. ...16

Fig. 2-6: Illustration of synchronization between the driving trajectory and bio-signal data...17

Fig. 2-7: Steps of EEG preprocessing...18

Fig. 2-8: Illustration of 5-second moving window with 4.88-second overlap...18

Fig. 2-9: Steps of EOG preprocessing ...19

Fig. 2-10: The flowchart of the EEG and EOG analysis method. ...22

Fig. 3-1: Illustration of hardware framework of our system...24

Fig. 3-2: Diagram of portable bio-signal acquisition unit...25

Fig. 3-3: The RG decides the gain of preamplifier, and the high pass filter of preamplifier decided by RG and C. ...27

Fig. 3-4: Simulation of preamplifier’s gain response (EEG) ...27

Fig. 3-5: High-pass filter and Low-pass filter circuits ...28

Fig. 3-6: Simulation results in EEG channel of amplifier and band-pass filter ...28

Fig. 3-7: Simulation results in EOG channel of amplifier and band-pass filter ...29

Fig. 3-8: MSP430 Architecture [51] ...30

Fig. 3-9: Operating flow chart in MSP430F1611 ...30

Fig. 3-10: Timer_A up mode for interrupt function of MSP430F1611 ...31

Fig. 3-11: ADC12 Block Diagram [51] ...32

Fig. 3-12: Diagram of the sampling and conversion with timer A trigger ...33

Fig. 3-13: Data format...33

Fig. 3-14: Result of noise cancellation by using moving average ...35

Fig. 3-15: Power supply circuit in portable bio-signal acquisition system...37

Fig. 3-16: Charging circuit in our portable bio-signal acquisition system ...38

Fig. 3-17: PCB Blue Tooth antenna [64] ...39

Fig. 3-18: The block diagram of DSP system [64] ...41

Fig. 3-19: The schematic circuit of Slave board ...42

Fig. 3-20: The front-end analog and digital control circuit...43

Fig. 4-1: The example of deviation event and car trajectories...47

Fig. 4-2: Blink features ...52

Fig. 5-1: The result between two conditions...56

Fig. 5-2: The result of arbitrary wave test...57

Fig. 5-3: The position of Fengchi ...58

Fig. 5-4: Result of EEG raw data and corresponding frequency spectrum ...58

Fig. 5-5: Results of EEG raw data: (a) before bio-stimulation, (b) during bio-stimulation and (c) after bio-stimulation. ...59

Fig. 5-6: Example 1 of driving performance and unsupervised analysis...61

Fig. 5-7: Example 2 of driving performance and unsupervised analysis...62

Fig. 5-8: Example 3 of driving performance and unsupervised analysis...62

Fig. 5-9: Example 4 of driving performance and unsupervised analysis...63

Fig. 5-10: Process of sorting analysis ...65

Fig. 5-11: The relationship between MDA/ MDT/ Duration and reaction time ...66

Fig. 5-12: The relationship between sensitivity and specificity ...67

Fig. 5-13: Positive predictive value vs. threshold of “Combination” where the numeric in parentheses was sequentially expressed as (MDT, MDA, Duration) ....70

Fig. 5-14: Sensitivity vs. threshold of “Combination” where the numeric in parentheses was sequentially expressed as (MDT, MDA, Duration) ...71

Fig. 5-15: F-measure vs. threshold of “Combination” where the numeric in parentheses was sequentially expressed as (MDT, MDA, Duration) ...73

Fig. 5-16: The result of comparison between MDT/MDA only and with blink duration ...77

Fig. 5-17: The flowchart of DSP module program ...78

Fig. 5-18: The user interface’s flowchart ...80

Fig. 5-19: The block diagram of dataflow ...80

Fig. 6-1: (a) top view, (b) exploded view of the proposed dry foam EEG electrode. .83 Fig. 6-2: Frequency characteristic of the proposed dry foam electrodes on ...85

Fig. 6-3: Impedance variation of dry foam electrode and conventional wet electrode under long-term EEG measurement...86

Fig. 6-4: Placements and results of measurements by using different types of electrodes ...88

List of Tables

Table 2-1: Common band of EEG ...12

Table 2-2: Feature of blink behaviors ...14

Table 3-1: Specification for various kinds of bio-sensors ...25

Table 3-2: The spec of portable bio-signal acquisition system ...44

Table 3-3: The spec of DSP Module ...44

Table 5-1: The description of binary classification test ...67

Table 5-2: The results of binary classification test ...74

Table 5-3: 10 subjects’ results of validation with MDT=0.3, MDA=0.3, Duration=0.4 and threshold set as 31.5 ...75

Chapter 1 Introduction

1.1 Drowsiness Detection

In recent years, traffic accident is one of the critical reasons to cause deaths of drivers. World Health Organization report released that the global traffic accidents killed 1.2 million lives each year and caused millions of people were injured [1]. The report stated that a daily average of 1000 persons aged 25 years of age because of the people killed in traffic accidents, of which 90 percent of the victims took place mainly in Africa and Asia, low-income countries. The report said that the 19-year-old and 15-year-old groups to the cause of death, traffic accidents ranked first, far exceeding the number of AIDS deaths. It showed that the traffic safety is the very urgent issues that need to straighten and improve.

The cause of accidents is often imputed to driver’s mental state. A human in drowsiness often exhibits relative inattention to environments, eye closure, less mobility, failure to motor control and decision making [2]. Therefore, those accidents which caused by falling drowsiness usually not only endanger themselves but also involve the public. Many studies have pointed out that a driver’s drowsiness can cause serious traffic accidents [3]-[6]. In 2002, the National Highway Traffic Safety Administration (NHTSA) reported that about 0.7% of drivers have been involved in a crash that they attribute to drowsy driving, amounting to an estimated 800,000 to 1.88 million drivers in the past five years [7]. The National Sleep Foundation (NSF) also reported that 51% of adult drivers had driven a vehicle while feeling drowsy and 17% had actually fallen asleep [8].

drowsiness / departure from alertness in drivers has become an important area of researches. If the mental state of drivers can be real-time monitored directly, drowsiness detection and warning can effectively avoid disasters such as vehicle crashes in working environments. Recently, with the development of brain computer interface, real-time monitoring the mental states of drivers and detecting drowsiness have become feasible.

1.2 Previous Research

Drowsiness leads to decline in drivers’ abilities of perception, recognition, and vehicle control and hence monitoring of drowsiness in derivers is very important to avoid road accidents [9]. Some researches used non-physiological method, as eye closure with CCD image tracking [10]-[16]. And others used physiological parameters to increase the accuracy of drowsy detection, like pulse wave analysis with neural network [20], the electrooculography (EOG) and the electromyography (EMG) measurement [17], [18], and theelectroencephalogram (EEG) [19]-[21].

In 2003, Hamada et al. proposed a driver status monitor system by using CCD camera, as shown in Fig. 1-1 [13]. The CCD camera was installed in the car and focused on the user’s eyes. The driver status monitor detected drowsiness from the change in the duration of eye closure during blinking and inattention from the change in the gaze direction. Using CCD camera to contribute the urgency system was a very difficult work here. There were some critical points inside, and needed to overcome. For instance, user couldn’t move for free, the images detecting performance were easily be interfered by light, and the largest problem was that the system is too big, complex, and expensive to implement. The algorithm of eye tracking also needed to

use edge detecting to train data, and hence to build up a neural network to classify the drowsy status.

Fig. 1-1: The role of driver status monitor [13]

An alternate is to detect the moment from alertness to drowsiness by using physiological parameters. In 2005, Thum et al. used EOG as an alternative to video-based systems in detecting eye activities caused by drowsiness [18]. Rapid eye movements (REM), which occurred when one is awake, and slow eye movements (SEM), which occurred when one is drowsy, can be detected through EOG. The results showed that the detection rate for eye activities caused by drowsiness was more than 80 %. However, REM and SEM are difficult to measure when users are driving because users can not close his/her eyes when they are driving a vehicle on the road, and then SEM is hard to measure. In addition, REM and SEM are tending to the level of sleep stage not the indicator of drowsiness detection, so they can not be used as the parameters of on-line process.

In 2003, Caffier et al. proposed that the spontaneous eye blink is considered to be a suitable ocular indicator for fatigue diagnostics [24]. To evaluate eye blink parameters as a drowsiness indicator, they developed a contact free method for the measurement of eye blinks by using an infrared sensor clipped to an eyeglass frame recorded eyelid movements continuously. The parameters blink duration and

reopening time in particular change reliably with increasing drowsiness. The results demonstrate that the measurement of eye blink parameters provided reliable information about drowsiness. In 2008, Jammes et al. in order to automatically score the drowsiness level, they developed a software for identifying blinks in EOGs as their first step [23]. They recorded vertical EOG signals by surface electrodes placed above and below the eyes. The analysis of EOG velocity based on expert rules was the originality of their blink detection algorithm and more than 97.7% of blinks were detected by their algorithm. The drowsiness scale they selected was Karolinska Drowsiness Score (KDS) which would score when signs of drowsiness, i.e. long duration or small amplitude blinks were detected. Comparing the results of KDS and the results of their automatic scoring, and then they found out the correlation of these results was high. It demonstrated that blink duration and amplitude are important parameters for drowsiness detection.

Brain Computer Interface (BCI) is an interface between human and computers or machines. It is based on the translation of the specific brain activity generated by a specific thought of a human to control machines, to communicate with the outside world directly, to convey the message, and independent operations, as well as self-care purposes. BCI can be divided into three distinct modes: invasive, partially-invasive, and non-invasive BCI. Non-invasive BCI is the main stream of BCI research which has advantages of both easy application and absence of procedural risks, such as infection or cortical micro-lesions. There are several approaches to non-invasively acquire brain activities, such as magentoencephalography (MEG), positron emission tomography (PET), functional magnetic resonance imaging (fMRI), electroencephalography (EEG) and et al. EEG is the mainstream of non-invasive BCI, because of its much fine temporal resolution,

ease of use, portability and low set-up cost. In particular, higher temporal resolution becomes the great temptation to use EEG techniques as a direct communication channel from the brain to the real world [27]-[42].

In EEG system, it was different from other physiological parameters, and moreover it owned intuitive and specific characteristics, such as alpha, theta or beta band power followed subject’s own mental state. In addition, the EEG system usually needed to collect enough EEG data to analyze. The supervised methods which previously study often had been used to train a learning data, and usually implement in off-line EEG analysis. Previous studies which used supervised methods developed several kinds of brain computer interface for drowsiness detection [19], [20]. When the subject changed the state from alertness to drowsiness, the alpha rhythm will increase and beta rhythm will decrease [21]. In 2005, a drowsy estimation system was developed by combining independent component analysis (ICA), power-spectrum analysis, correlation evaluations, and linear regression model to estimate a driver’s cognitive state when he/she drove a car in a virtual reality (VR)-based dynamic simulator [19]. Its flowchart of EEG processing was shown in Fig. 1-2. In the above studies, an EEG machine, Scan NuAmps Express system (Compumedics Ltd., VIC, Australia), was used to measure EEG, as shown in Fig. 1-3. It is not small, light, and wearable. Moreover, the above algorithms for drowsiness detection requires mass computation complexity, thus, they are not easy to be implemented in a portable DSP device.

Fig. 1-2: Flowchart of EEG processing in drowsy estimation system [19]

Fig. 1-3: Scan NuAmps Express system (Compumedics Ltd., VIC, Australia) In the supervised mode, supervised learning methods such as artificial neural network (ANN) could be used to classify different states of vigilance. But stimulus may introduce some noise. So in [43], the author proposed a semi-supervised learning algorithm which can quickly label huge amount of data. Here another author proposed another kind of semi-supervised learning method based on probabilistic principle component analysis (PPCA) to distinguish wake, drowsy and sleep in driving simulation experiment. After training with data of around 20 min (6–8 min for each state), they could directly use our method as a real time classifier to estimate driver’s

vigilance state [44]. Although this method could greatly reduce the training time, but it still must used in off-line analysis. In our target, we wanted to find a non-training and unsupervised method, and easily implement to an on-line detecting system.

1.3 Motivation

To avoid tragedies, a real-time physiological signal monitoring system of drowsiness detection is required to prevent traffic accidents. However, users’ mobility is limited by the inconvenience of traditional BCI (heavy and large EEG machine). Besides, recent studies used one kind of bio-signal as EEG or EOG only to detect drowsiness. Nevertheless, the accuracy of detection is not high enough with one physiological signal used only. Thus, an inexpensive, convenient, portable, wireless and multi-parameter of bio-signal used platform with long battery life that can be carried indoors or outdoors are desired.

In this study, not only a real-time wireless brain computer interface for drowsiness detection was proposed, but also parameters of EOG were used. The spontaneous eye blink can be determined by EOG measurement and is considered to be a suitable ocular indicator of fatigue and alertness level [22, 24]. It offers several advantages, since it represents a normal, simply observable and easily accessible phenomenon that reflects the influence of central nervous activation without voluntary manipulation[24]. Recent studies show that the analysis of eye blinks can provide some information for physiologists to quantify drowsiness. There are many parameters of eye blinks, such as duration, frequency, closing time, opening time, amplitude and so on. The blink duration and opening time are especially subject to characteristic modifications with increasing drowsiness. Besides, frequency

(percentage of eyelid closure) was also used as the indicator by previous research to determine user was drowsy or not [22-26]. To increase accuracy of drowsiness detection, parameters of EOG signal were used in our study and we would stress the importance of correlation between drowsiness with EOG signal.

A wearable, wireless and real-time bio-signal acquisition system was designed for long-time EEG/EOG monitoring, and a DSP module with bio-feedback as buzzer or bio-stimulator was also introduced. An algorithm of drowsiness detection based on [45] was proposed to reduce the complexity of computation. Different from previous ICA-based algorithm of drowsiness detection, it used the statistics properties of alpha/theta rhythm and blink duration in alert state to build up the alert model. Consequently, a derivation from the alert model can be used to detect drowsiness. The most useful advantage of this algorithm was that the differences between individual and environment in different people or measurements could be removed, and every analysis was independent. Moreover, with the advantage of low computational complexity, it is easy to be implemented in our portable DSP module.

1.4 Organization of Thesis

In Chapter 2, it will describe that what are EEG and EOG signals, virtual reality driving simulation environment, and algorithms implemented in this thesis, which including EEG/EOG preprocessing and unsupervised approach. In Chapter 3, it will introduce how to implement a wireless portable bio-signal acquisition system and DSP module in hardware design. In Chapter 4, it will explain the detail of driving performance, unsupervised algorithm, and how to accomplish them. In chapter 5, it will introduce the driving performance sorting analysis then the method of driving performance and unsupervised approach will be verified with 10 real experimental subjects’ driving trajectories and corresponding EEG and EOG signals, the procedures and results of verification will be described in this chapter. Finally it will have conclusions and future works in Chapter 6.

Chapter 2 Material and Method

We developed the BCI system according to the steps of Fig. 2-1. The portable bio-signal acquisition system which we designed was used in input device of BCI. The EEG and EOG raw data continually transmitted to DSP module, hence, the following three steps: signal preprocessing, features extraction, and classifier were processed in DSP module. The algorithm we chose was according to unsupervised approach (N. R. Pal, 2008 [45]) and automatic EOG analysis (B. Jammes, 2008[23]). The user interface can output real-time EEG and EOG signals and the results of drowsy detection. If the results were determined as drowsiness by algorithm, DSP module would call the buzzer to output a warning voice or generate stimulation by the muscular bio-stimulator to wake user up as a bio-feedback application.

Input Device Signal Preprocessing Features Extraction Classifier Output User Interface Applications DSP module Bio-feedback Closed Loop

Fig. 2-1: A typical BCI system architecture

In off-line analysis, we wanted to verify the relationship between users’s driving trajectories and corresponding EEG and EOG signals. Before analyzing, we assumed

that driving trajectories were directly proportional with variance of theta, alpha spectrum and blink duration. So we designed a driving simulation experiment and used our portable bio-signal acquisition system to observe and record driving information and actual EEG and EOG raw data at one time. There were 10 subjects’ EEG and EOG raw data recorded and every trail was at least 25 minutes. Our analysis included two parts: one was to analyze the driving trajectories, and another was to analyze the corresponding EEG and EOG signals. The first step of driving trajectories processing was to analyze the driving performance. On the other hand, we also analyzed EEG and EOG signals. First, we used FFT to get the theta and alpha band information, and then used both two information built up an alert model, computing covariance matrix and mean vector of theta and alpha spectra. Furthermore, compute MDT and MDA continually by using unsupervised method. Second, we calculated the derivative of EOG signals. The EOG velocity was acquired, and then a threshold was applied to select blinks. Next, the duration of these blinks was computed as the reference of drowsiness. After finishing whole data analysis, we used binary classification test, sensitivity and specificity, to verify the drowsiness hit rate. Every experimental trial was separated and sorted, hence, the corresponding MD* (MDT and MDA) and duration of blinks were also sorted. Defining the threshold of both information which been processed to decide the drowsiness or alertness, and to analyze the drowsy accuracy.

2.1 EEG Signal Acquisition

Electroencephalography (EEG) is the recording of electrical activity along the scalp produced by the firing of neurons within the brain. In clinical contexts, EEG

refers to the recording of the brain's spontaneous electrical activity over a short period of time, usually 20–40 minutes, as recorded from multiple electrodes placed on the scalp [55]. When measuring from the scalps, recorded EEG signal is about 10-100uV for a typical adult human. And a common system reference electrode is connected to the other input of each different amplifier. These amplifiers amplify the voltage between the active electrode and the reference (typically 1,000–100,000 times, or 60–100 dB of voltage gain). The EEG is typically described in terms of rhythmic activity and transients. The rhythmic activity is divided into bands by frequency. The common band of EEG is shown as Table 2-1. Following the classification of EEG, Theta and Alpha band are related to drowsiness. Thus, when the subjects become drowsy, both bands will increase their power.

Table 2-1: Common band of EEG Type Frequency (Hz) Normally

Delta <4 Slow wave sleep for adults

Theta 4~7 Drowsiness, idling, or arousal in children and adults Alpha 8~12 Relaxed, reflecting, or closing the eyes

Beta 12~30 Alert or working

There are high correlation between drowsiness and EEG obtained from the location of OZ in the international 10–20 EEG system [56]. Therefore, in this study, we only monitored EEG in the location of OZ. Here, three EEG electrodes were used. One was input, one was reference, and the other was ground. According to a modified International 10–20 EEG system and refer to right ear lobe as depicted in Fig. 2-2. We used the following notations: F: Frontal lobe. T: Temporal lobe. C: Central lobe. P: Parietal lobe. O: Occipital lobe. "Z" refers to an electrode placed on the mid-line. The input data was placed on OZ, ground was fixed on the center of forehead, and reference was pasted behind the right ear.

Fig. 2-2: International 10-20 system

Raw EEG data were recorded with 12-bit quantization level at the sampling rate of 256 Hz. And a simple moving average filter was used to remove 60 Hz power line noise and other high-frequency noise.

2.2 EOG Signal Acquisition

Electrooculography (EOG.) is a technique for measuring the resting potential of the eyeball. Mostly, there are two electrodes placed above and below the eye, and the resulting signal is called vertical EOG. If the eye is moved from center position toward left or right, then one of the electrodes would see the positive side of the eyeball and the other would see the negative side. There would be a potential difference between the electrodes. If we assumed the resting potential as a constant, then the potential difference become a measure for the eye position called eye movement measurements [46]. Eye movement measurements usually used as a reference of stages of sleep which included three main stages called: awake, REM and NREM. Eye movement is significantly difference during these three stages, so lots of research of sleep used this measurement to observe variation. In this study, the

vertical EOG was derived using three electrodes: input, reference and ground. One electrode was placed above the right eye as the input and the other below the right eye as the reference signal. There was also an electrode as ground fixed on the center of forehead (the same as the ground signal of EEG). A sampling rate of 256Hz was used; because of the principal measurement range of EOG is about 50uV-3500uV, the gain was set 2,000 times. Besides, the parameters of EOG which used in our study were blink behaviors, so feature of these blink behaviors was listed in Table 2-2. The recommended derivation of EOG was shown as Fig 2-3.

Table 2-2: Feature of blink behaviors Behavior Description

Blink amplitude A typical blink has an amplitude of 400uV Blink duration Nearly 200ms – 400ms for one blink

Blink frequency About 15-20 times per minutes for a relaxed person

Reference

Input Ground

Fig. 2-3: The recommended derivation of EOG

2.3 Virtual Reality Driving Simulation Environment

In this study, a lane-keeping driving experiment was utilized to investigate driving performance under different levels of drowsiness. Here, a virtual reality (VR)-based cruising environment was developed to simulate a car driving at 100 km/hr on a straight four-lane highway at night [19], [57]. During the driving

experiments, all scenes move according to the displacement of the car and the subject’s maneuvering of the wheels which make the subject feel like driving the car on a real road. The VR environment was shown in Fig. 2-4.

Fig. 2-4: The overview of surrounded VR scene. The VR-based highway scenes are projected into surround screen with seven projectors.

In our experiments, the driving speed was fixed at 100 km/hr and the car was drifted away from the center of the cruising lane automatically and randomly by the system to mimic the effects of a non ideal road surface. The driver was asked to maintain the car along the center of the cruising lane. All subjects involved in this study had good driving skill and hence when the subject was alert, his/her response time to the random drift was short and the deviation of the car from the center of the lane was small. However, when the subject became not alert / drowsy, both the response time and the car’s deviation were high. Note that, in all our experiments, the subject’s car was the only car cruising on the VR-based freeway. Although, both response time and the deviation from the central line are related to the subject’s driving performance, in this study, we use the response time as a measure of

performance of the subjects. The driving task was shown in Fig. 2-5.

Fig. 2-5: The digitized highway scene. The width of highway is equally divided into 256 units and the width of the car is 32 units. An example of the deviation event, the car cruised with a fixed velocity of 100 km/hr on the VR-based highway scene and it was randomly drifted either to the left or to the right away from the cruising position with a constant velocity. The subjects were instructed to steer the vehicle back to the center of the cruising lane as quickly as possible [56].

In order to synchronize the records of driving trajectory and raw bio-signal data, a C# program was designed to record both of them at the same sampling rate. The driving trajectory produced from the VR-based cruising environment environment program, and raw bio-signal data obtained by portable bio-signal acquisition system were transmitted to C# program via RS232 and Bluetooth respectively. After finishing

the experiment, both driving trajectory and raw bio-signal data were saved in a text file. Thus, we could investigate the correlation between driving performance and results of unsupervised approach. The illustration of synchronization between the driving trajectory and bio-signal data was shown in Fig. 2-6.

Fig. 2-6: Illustration of synchronization between the driving trajectory and bio-signal data

2.4 EEG Preprocessing

The EEG preprocessing steps were shown in Fig. 2-7. First, a simple moving average filter (low-pass filter with a cutoff frequency of 32 Hz) was used to remove 60 Hz power line noise and other high-frequency noise. In order to simplify the computation, raw EEG data were down-sampled to sampling rate of 128 Hz. Then a 640-point Hanning window was designed to save 5 seconds EEG information and the frequency resolution was set as 256-point, as Fig. 2-8 shown. In our study, Hanning window was chosen because the frequency resolution and spectral leakage are both good for each application. Finally, the power in the frequency band of alpha rhythm (8 ~ 12Hz) and theta rhythm (4 ~ 7Hz) was extracted.

Fig. 2-7: Steps of EEG preprocessing

Fig. 2-8: Illustration of 5-second moving window with 4.88-second overlap

2.5 EOG Preprocessing

EOG preprocessing was introduced in this section. Comparing to EEG preprocessing, EOG preprocessing was simpler to implement. Behavior of EOG emphasized in our study was blink, so the steps of EOG preprocessing were aimed at

preprocessing for blink detection. First, we had to define the initial threshold parameters. The values of threshold were empirically selected after analyzing a set of EOG signals recorded as user was in alert state before experiments. According to these threshold parameters then we could acquire the closing time, the opening time of the blinks and other parameters of blink. Second, a 10-Hz low pass filter was used in order to remove frequencies above 10-Hz. Finally, the derivative of the filtered signal called EOG velocity need to be computed. To combine EOG velocity and initial threshold parameters then we could define if the event of EOG raw data was a blink or not, and acquired the parameters we needed. The steps of EOG preprocessing were shown in Fig. 2-9.

Fig. 2-9: Steps of EOG preprocessing

2.6 Unsupervised Analysis of EEG and EOG

It is recognized that the changes in EEG spectra in the theta band (4~7Hz) and alpha band (8~12Hz) reflect changes in the cognitive and memory performance [58]. Other studies have reported that EEG power spectra at the theta band [59], [60] and/or alpha band [61], [62] are associated with drowsiness, and EEG log power and subject’s driving performance are largely linearly related. Besides, blink of EOG is also recognized as an important parameter for detecting drowsiness[23]. There are lots of parameters could be extracted from blinks, for instance: amplitude, duration and so on. Blink duration is an significant information as a result of it becomes longer when

subjects become drowsy, and the changes of blink amplitude reflect the drowsiness level too; the amplitude is small when subjects feel drowsy[24],[25].

As above researches said, these findings have motivated us to derive the alert model of the driver using not only the alpha-band and theta-band EEG power spectrum computed using OZ channel output recorded in the first few minutes of driving, but also the duration of blinks from subjects to increase the accuracy of drowsiness detection. The unsupervised analysis of EEG is introduced first. The choice of the OZ channel is explained in the Experimental Results section. We emphasize that the few minutes of data used to find the alert model are not necessarily collected from the very beginning of driving session because different factors, such as walking of driver by a few meters to reach the garage, may influence the EEG signal generated at the very beginning. The specific window to be used for generation of the alert model is selected by Mardia test [63]. We assume that if the subject/driver is in an alert state, then the EEG power spectra relating to theta band (as well as that relating to alpha band) would follow a multivariate normal distribution. The parameters of the multivariate normal distributions characterize the models. Using the alpha-band and theta-band EEG power, we identify two normal-distribution based models. Then, we assess the deviation of the current state of the subject from the alert model using Mahalanobis distance (MD). We assume that when the subject continues to remain alert, his/her EEG power should resemble the sample data used to generate the model and hence would match the alert model or template. If the subject becomes drowsy, then its power spectra in the alpha band (and also in theta band) will deviate from the respective model and hence MD will increase. With a view to reducing the effect of spurious noise, MDs are smoothed over a 20-sec moving windows, the window is moved by 0.125-sec steps.

Next, we focus on the analysis of EOG signal. In our study, lots of blink behaviors are extracted by our algorithm in the beginning. To base on section 2.5, after EOG velocity and initial threshold parameters are all prepared, we could use the closing and opening threshold to define closing time and opening time as parameters. Besides, computing the difference between the starting point of closing time with end point of opening time then we could acquire the value of blink duration. On the other hand, blink amplitude is computed by using maximum value of one blink to subtract the value of EOG baseline. After above procedure, amplitude of blinks, duration of blinks, closing time of closures and opening time of closures are all acquired. However, some of these parameters could not reflect the variation when subjects become drowsy, for example: the difference of closing time of closures is little between alert and drowsiness[22]. At last, duration is the only parameter we used in our study to determine the subjects are drowsy or not. We assumed when subjects are remaining alert, duration of blinks would be short. In contrast, when subjects feel drowsy, duration of blinks would become longer. 6-second averaged signal of duration before every event of driving performance occurred is extracted by our algorithm to reduce the effect of spurious noise and to be the index for detecting drowsiness.

We then study the relationship between smoothed Mahalanobis distance, duration of blinks and subject’s driving performance by computing the correlation between them. Fig. 2-10 shows the overall flow of the EEG and EOG data analysis. In this figure, after the models are identified, the preprocessed alpha band and theta band power data directly go to the blocks for computation of MDA and MDT respectively. Besides, blink duration is computed at the same time. The block for computation of “Combination” makes a linear combination of MDT, MDA and duration of blinks.

Finally, all of these parameters are used in correlation analysis with the driver’s performance. Alpha Band Spectra Theta Band Spectra Determining the Theta Model by Mardia Test Determining the Alpha Model by Mardia Test Computing MDT Computing MDA Combination Driving Experiment Behavior Analysis Preprocessing Compare with the Driving Performance Building the alert model

Computing parameters for alert model

EOG Velocity Computing

Blink Duration Driving Trajectories Bio-Signal Driving Performance

Chapter 3 Hardware Frameworks

In this chapter, we focus on this portable system hardware. Following the design flowchart, we will introduce the design methods of hardware circuits and firmware structures steps by steps.

3.1

System Overview

In order to online-measure and analyze EEG and EOG signals, the whole hardware framework of our system mainly contains two sub-systems: One is portable bio-signal acquisition system and the other is DSP module. First, EEG and EOG signals were measured by our portable acquisition module continually. After EEG and EOG signals were acquired, these tiny signals would be amplified. Noise except the frequency band of these bio-signals would be removed by filters in our portable acquisition module. Then filtered EEG and EOG signals would be digitized by analog-to-digital converter and transmitted to the DSP module via Bluetooth. In this study, Linux kernel µClinux was used as the operation system in DSP module to handle user’s applications. The major tasks of DSP module were to receive EEG and EOG signals via Bluetooth, and to execute the program of online drowsiness level detection, which monitored the variation of power of users’ alpha rhythm and theta rhythm. Besides, the difference of blink duration was also monitored. The program of

online drowsiness level detection would collect EEG data under alertness for first 3 minutes to build EEG alert model, and then calculated drowsiness level by assessing the power variation of alpha and theta rhythm every 2 seconds. In addition, the duration of every blink was also calculated by our module. If the power or duration variation exceeded the threshold of alert model, the DSP module would send warning

tone of buzzer or generate an electrical stimulation to wake up user. The whole hardware framework is shown as Fig. 3-1.

Bio-signal

Filter & Amplify

ADC

Portable Bio-signal Acquisition System

Unsupervised Algorithm Buzzer Bio-stimulator Bluetooth

DSP module

Bluetooth TFT-LCD Display

Fig. 3-1: Illustration of hardware framework of our system

3.2 Portable Bio-signal Acquisition System

The portable bio-signal acquisition unit combines the power, amplifier, band pass filter, ADC, wireless controller, and data encoding into one. It is a light weight, wireless monitor for recording physiological signals. It owns 4-channel bio-signal measurement, includes EEG x2 and EOG x2. The portable bio-signal acquisition unit mainly contains four parts: (1) front-end filter circuit, (2) analog to digital converter, and digital controller, (3) power management circuit and (4) wireless transmission. The diagram of the portable bio-signal acquisition unit is shown as Fig. 3-2.

Fig. 3-2: Diagram of portable bio-signal acquisition unit

In our study, only one EEG and one EOG channel of this portable system were used. They were both amplified by bio-amplifier consisted of high-pass, low-pass filters and amplifier. The amplified signals then organized by Micro-processor MSP430. When the signal passed through the high-pass filter, all unnecessary low-frequency noise would be reduced or eliminated, as they passed through the low-pass filter, all unnecessary high-frequency noise would be reduced or eliminated. According to AASM recommendations and difference physiological potentials characteristics, the specification for EEG and EOG channels was shown in Table 3-1.

Table 3-1: Specification for various kinds of bio-sensors

Sensor Input Signal Range Gain (Operation voltage:3V) H.P. Corner F. (Hz) L.P. Corner F. (Hz) Sampling rate (Hz) EEG 20uV-200uV 4500 0.1 45 256 EOG 50uV-3500uV 2000 0.3 45 256

3.2.1 Front-End Filter Circuit

The front-end circuit consisted of preamplifier, and band-pass filter. In some references, other circuit designs preferred to use unit gain filters and one variable gain amplifier. Moreover, they didn’t use a high-pass filter to cut-off the noise in low frequency band. To improve them, we designed a 3 stages high pass filter and 2 stages low pass filter to get the clear EEG and EOG information without noise. Hence, adding the gain into filter tried to minimize the total size.

A. Preamplifier

Instrumental amplifier LT1789-1 was used as the first stage of analog amplifier. LT1789-1 owns an ultra low input current and a high common-mode rejection ratio (CMRR) about 90dB. A high CMRR is important in applications that the signal of interest is represented by a small voltage fluctuation superimposed on a (possibly large) voltage offset, or when relevant information is contained in the voltage difference between two signals. Instrumental amplifier LT1789-1 provided not only the function of gain, but also that of one stage high pass filter by adding a capacitor. The output voltage of the LT1789-1 is referenced to the voltage on the reference terminal. The preamplifier circuit design is shown in Fig. 3-3 and the simulation of preamplifier’s gain response (EEG) is in Fig. 3-4.

Fig. 3-3: The RG decides the gain of preamplifier, and the high pass filter of preamplifier decided by RG and C.

10-4 10-3 10-2 10-1 100 101 102 103 0 1 2 3 4 5 6 7 8 9 X: 8.125 Y: 7.043 f (Hz) G a in ( d B )

Fig. 3-4: Simulation of preamplifier’s gain response (EEG)

B. Band pass filter

In this thesis, operational amplifiers were used to achieve the function of band-pass filter; the feature of op AMP is suitable for amplifying low-frequency signal. The AD8609 is quad micro-power rail-to-rail input and output amplifiers and low dc offset was chosen to be band pass filter. Fig.3-5 shows High-pass filter and Low-pass filter circuits. The 3dB cutoff frequency of high pass was decided

Gain = 1 + (200k/R

G

)

by passive components R3, R4, C1 and C2. 3 4 1 2 1 2 H f R R C C

π

=

Fig. 3-5: High-pass filter and Low-pass filter circuits

The passive components R7, R8, C3, and C4 decide the 3dB cutoff frequency

6 7 3 4 1 2 L f R R C C

π

= .A circuit of band-pass filters and amplifier is designed as shown in Fig. 3-5 and gain will be determined by passive components R1, R2, R5 and R7. The simulation results of each channel are shown as Fig.3-6 and Fig.3-7.

Fig. 3-7: Simulation results in EOG channel of amplifier and band-pass filter

3.2.2 Analog to Digital Converter and Digital Controller

For the data acquisition system, it needs a controller to organize the working of ADC and encode the digital data to Bluetooth module by UART port.The MSP430 is particularly well suited for wireless RF or battery powered applications. The MSP430 incorporates a 16-bit RISC CPU, peripherals, and a flexible clock system that interconnect using a von-Neumann common memory address bus (MAB) and memory data bus (MDB) shown as Fig. 3-8. The clock system is designed specifically for battery-powered applications. Dedicated embedded emulation logic resides on the device itself and is accessed via JTAG using no additional system resources.We configure with built-in 16-bit Timer_A, a fast 12-bit A/D converter, one universal serial synchronous/asynchronous communication interfaces (USART) and 4M Hz external oscillator to development our design[51].

Fig. 3-8: MSP430 Architecture [51]

Timer_A triggers Analog to Digital Converter, and buffers the output data of ADC until buffer full. And then all buffer data will be transmitted via USART. The operating flow chart in MSP430F1611 was shown in Fig. 3-9.

Fig. 3-9: Operating flow chart in MSP430F1611

A. Timer Interrupt

The interrupt function of MSP430F1611 is based on inner timer/counter register, called Timer_A, to count a specific time value. The counter value TACCR0 had to be set first, as shown in Fig. 3-10. When the timer counted to the TACCR0 value, the TACCR0 CCIFG interrupt flag would be set. When the timer counted from TACCR0 to zero, the TAIFG interrupt flag would be set. In our portable bio-signal acquisition

module, 4MHz crystal oscillator was used as system clock of MSP430F1611. Thus, if the sampling rate of our acquisition module is set to 256 Hz, TACCR0 has to be set to 15625.

15625 256

4M

TACCR0 = =

Fig. 3-10: Timer_A up mode for interrupt function of MSP430F1611

B. Analog to Digital Converter

In this system, by passing the signal through wireless, it needs an analog to digital converter to convert the continuous signal to discrete number. To suit with the filtered and amplified signal from front-end circuit, built in ADC of MSP430 was chosen to be an analog to digital converter.

Fig. 3-11: ADC12 Block Diagram [51]

Fig. 3-11 shows ADC12 Block Diagram. The ADC12 module supports fast, 12-bit analog-to-digital conversions. The module implements a 12-bit SAR core, sample select control, reference generator and a 16 word conversion-and-control buffer. The conversion-and-control buffer allows up to 16 independent ADC samples to be converted and stored without any CPU intervention [51]. The ADC12 inputs are multiplexed with the port P6 (A0-A7) pins, which are digital CMOS gates. An analog-to-digital conversion is initiated with a rising edge of the sample input signal SHI. The signal SHI will be set by interrupt routine of timer A at 256Hz. The ADC12 module is configured by three control registers, ADC12CTL0, ADC12CTL1 and ADC12MCLTx. Those registers are set to enable core, select conversion clock, set conversion mode, sample and input channels define. In our system, we used the

channels is sampled and converted once. Each conversion requires 6 ADC12CLK cycles; include conversion and result restored into ADC12MEMx conversion memory registers. Fig. 3-12 shows a diagram for sampling time and conversion time of ADC with trigger by timer A. Here, the total sampling and conversion time less then 7812 clocks. Therefore, the conversion time of ADC is fast enough to fit the requirement of the sampling rate of the whole system. The ADC result of each channel will be 12 bits long in the form of an unsigned integer whose value is: 4095* Ax -

Vr-Vr+ - Vr-

Fig. 3-12: Diagram of the sampling and conversion with timer A trigger

When conversion results are written to a selected ADC12MEMx, the corresponding flag in the ADC12IFGx register is set. An interrupt request is generated if the corresponding ADC12IEx bit and the GIE bit are set. After ADC12IFGx register set, the interrupt service routine of ADC started. In the interrupt service routine, we buffered ADC12MEMx. Next, a moving average filter was used to remove 60-Hz power line interference, and then filtered signal data was encoded before wireless transmission [Fig. 3-13].

FF 62 Channel 1_low byte Channel 1_high byte Channel 2_low byte Channel 2_high byte

C. Moving Average

Moving average, also called rolling average or running average, is usually used to analyze a set of data points by creating a series of averages of different subsets of the full data set. Moving average can be applied to any data set, however, it is most commonly used with time series data to smooth out short-term fluctuations and highlight longer-term trends or cycles. The choice between short- and long- term, and the setting of moving average parameters depends on the requirement of application. Mathematically, moving average is a type of convolution and is similar to a low-pass filter used in signal processing. The moving average filter is optimal for a common task: reducing random noise while retaining a sharp step response. This makes it as the premier filter for time domain encoded signals.

Given a sequence

{ }

1

N i i

a ₌ , the output of an n-moving average is a new sequence

{ }

1 1

N n i i

s ₌− + defined as the average of subsequences of n terms. The formula of moving averaging was shown as followings.

1 1 1i n i j j s a n + − = =

∑

(3-1) Therefore, the sequences sn of n-moving averages when n=2,3 can be expressed

as

(

)

2 1 2 2 3 1 1 , ,..., 2 n n s = a +a a +a a₋ +a

(

)

3 1 2 3 2 3 4 2 1 1 , ,..., 3 n n n s = a + +a a a + +a a a ₋ +a₋ +a (3-2) Fig. 3-14 shows the results of noise cancellation by using moving average. A function generator was used to generate sin wave, and our portable bio-signal acquisition system was used to record this signal. If our portable acquisition module

was close to some electric instruments, the signal recorded from the acquisition module would be easily influenced by noise of 60 Hz power line. In the above figure of Fig. 3-14, it showed that the original sin wave had been contaminated by 60Hz power-line noise. After filtering by using moving average with 5-point moving window, we found moving average could effectively remove power-line noise, as shown in the below figure of Fig. 3-14.

4.27 60 256 60 _ _of window=SampleRate = = Num (3-3) 3000 3020 3040 3060 3080 3100 3120 3140 3160 3180 3200 50 100 150 200

Befor Moving average

A m p Sample 3000 3020 3040 3060 3080 3100 3120 3140 3160 3180 3200 50 100 150 200

After Moving average

A

m

p

Sample

Fig. 3-14: Result of noise cancellation by using moving average

D. UART Interface

In asynchronous mode, USART connected MSP430 to external systems via two external pins, URXD and UTXD. In UART mode, USART transmitted and received characters at a bit rate asynchronously to another device. Timing for each character was based on the selected baud rate of USART. In our study, the transmitter and receiver used the same baud rate. For initializing UART, RX and TX had to be enable first, and then decided the baud rate of UART and disable SWRST. The required

division factor N for determining baud rate was listed as followings:

BRCLK N

baud rate

= (3-4) Here, BRCLK was 4 MHz, and baud rate was 115200 bit/s. After initializing UART, the micro-controller could transmit data filtered by moving average to BLUE TOOTH module via UART.

3.2.3 Power Management

Power Management circuit in our portable bio-signal acquisition system includes two parts: one is power supply circuit, and the other is charging circuit.

A. Power Supply Circuit

In our portable bio-signal acquisition system, the operating voltage VCC was at 3V, and the virtual ground of analog circuit was at 1.5V. In order to provide stable 1.5V and 3V voltage, a regulator LP3985 was used to regulate battery voltage to 3V. LP3985 is a micro-power, 150mA low noise, and ultra low dropout CMOS voltage regulator. The maximum output current can support 550mA. Furthermore, the turn-on time can reach 200µs. A voltage divider circuit was used to divide 3V voltage into 1.5V, and a unity amplifier constructed from AD8628 was used to provide a voltage buffer. The total power supply circuit was shown in Fig. 3-15.

Fig. 3-15: Power supply circuit in portable bio-signal acquisition system

B. Charging Circuit

The charging circuit BQ24010DRC had integrated power FET and current sensor for 1-A charging applications. The maximum charging current can arrive at 1A. The battery’s power would be detected automatically by charging circuit and switched to charging mode when battery’s power was not enough. BQ24010DRC also protected battery to avoid over charging or over driving [64]. The charging circuit was shown in Fig. 3-16.

Fig. 3-16: Charging circuit in our portable bio-signal acquisition system

3.2.4 Wireless Transmission

Bluetooth is a wireless protocol utilizing short-range communication technology to facilitate data transmission over short distances from fixed and/or mobile devices. The intent behind the development of Bluetooth was the creation of a single digital wireless protocol, capable of connecting multiple devices and overcoming issues arising from synchronization of these devices. In this study, Bluetooth module BM0203 was used. BM0203 is an integrated Bluetooth module to ease the design gap and uses CSR BuleCore4-External as the major Bluetooth chip. CSR BlueCore4-External is a single chip radio and baseband IC for Bluetooth 2.4GHz systems including enhanced data rates (EDR) to 3Mbps. It interfaces to 8Mbit of external Flash memory. When used with the CSR Bluetooth software stack, it provides a fully compliant Bluetooth system to v2.0 of the specification for data and voice communications. All hardware and device firmware of BM0203 is fully

compliant with the Bluetooth v2.0 + EDR specification. Bluetooth operates at high frequency band to transmit wireless data, so it can be perfect worked by using a PCB antenna, as shown in Fig. 3-17.

Fig. 3-17: PCB Blue Tooth antenna [64]

3.3 DSP Module

The goal of our DSP module is not only to build a back-end analysis platform but also to provide a bio-feedback mechanism by buzzer or stimulator. This module has greatly powerful calculating ability and supports various peripheral interfaces. After measuring and pre-processing EEG and EOG signals by our portable bio-signal acquisition system, these signals would be transmitted to this DSP module via Bluetooth module. Then, DSP module would process and analyze them. As soon as a drowsy state is detected, this module would warn the user by buzzer or stimulator. Furthermore, it can also use other peripheral interfaces to implement lots of applications [64].

3.3.1 DSP Framework

A powerful digital signal processor Analog Device BF533-STAMP was used in this DSP module, and its CPU speed can be up to 600MHz. There are two 16-bit

MAC Multiply-And-Accumulate used to execute 1200 lines addition and multiplication functions. Besides, DSP contains many independent DMA, Direct Memory Access, to effectively reduce the processing time of core. The system block diagram was shown in Fig. 3-18. In our study, Bluetooth module and UART both worked in the same UART interface.

TFT-LCD, an optional element depends on user’s requirement, which is worked by using Memory Mapping, shared the same Memory Bus with SDRAM. In order to reduce the size of platform, traditional parallel NOR Flash was replaced with SPI Flash, and it shared with micro SD Socket, too. Furthermore, the DSP module also owned power management and charging circuits. Micro SD Socket provided the interface scalability, such as micro SD Card, Sensor, ADC, Wireless Card, etc. There is also a buzzer placed on the platform worked via GPIO interface for application. As a matter of fact, our DSP module is divided into two parts: the Master board and the Slave board. The elements introduced above are all placed in the Master board, for instance: SDRAM, micro SD socket, buzzer and so on. There are three circuits of stimulator placed on the Slave board; it also worked via GPIO interface from the Master board. In our application for drowsiness detection and warning, a buzzer and a stimulator are used as a bio-feedback to warn users when they become drowsy. The stimulation which generated by the circuit of stimulator will be introduced in the next session.

BF-533 DSP TFT-LCD 128x160 24bits SDRAM 16MB Keypadx3 Buzzer Stimulator MicroSD socket Power DC/DC Charger Bluetooth UART Port SPI Flash 4MB UART Bluetooth USB

UART Memory Bus

SPI GPIO

Fig. 3-18: The block diagram of DSP system [64]

3.3.2 The Circuit of Stimulator

The circuit of stimulator was placed on the Slave board to generate the stimulation for warning the user. The Master board and the Slave board communicated with each other via GPIO interface. It is a muscular bio-stimulator, which is small, portable set, designed for those aiming at look improvement in the beginning. Stimulator also provides muscles' stimulation and invigoration so it was used as the bio-feedback mechanism to warn user. The schematic circuit of Slave board was shown in Fig. 3-19.

Fig. 3-19: The schematic circuit of Slave board

In every circuit of stimulator, there were two sources of pulse for user to select: Timer interface or GPIO interface. It was all depend on user’s application. In our study, the GPIO interface was chosen. As soon as a drowsy state was detected by the DSP module, 150µSec. pulses at about 80 Hz frequency would be generated and send to the Slave board via GPIO interface. Q1 acted as a buffer and Q2 inverted the polarity of the pulses and drove the Transformer. The amplitude of the output pulses was set by R1 and roughly displayed by the brightness of LED D1. D2 protected Q2 against high voltage peaks generated by T1 inductance during switching. The output signals would be generated by J1 and J2, which were directly connected to the electrodes. User only need taped the electrodes to the skin at both ends of the chosen muscle then the stimulator could be worked. The output voltage is about 60V positive and 150V negative but user is no need for fear of electric-shock danger, because the output current is so small and would be safe[72].

3.4 Hardware System Specification

A. Portable bio-signal acquisition system

Fig. 3-20 is the front-end analog circuit and digital control circuit of our portable bio-signal acquisition system. There are three leads in our portable bio-signal acquisition module for each channel, includes EEG or EOG input, reference, and virtual ground of the front-end analog circuit. The electrodes connected with the leads of virtual ground and EEG reference were placed on user’s forehead and behind right ear respectively. On the other hand, the reference of EOG was placed below right eye. The specification of portable bio-signal acquisition system was listed in Table 3-2.