Virtual Reality Driving Simulation Environment

In this study, a lane-keeping driving experiment was utilized to investigate driving performance under different levels of drowsiness. Here, a virtual reality (VR)-based cruising environment was developed to simulate a car driving at 100 km/hr on a straight four-lane highway at night [54], [62]. During the driving experiments, all scenes move according to the displacement of the car and the subject’s maneuvering of the wheels which make the subject feel like driving the car on a real road. The VR environment is showing in Fig. 2-3.

Fig. 2-3: The overview of surrounded VR scene. The VR-based highway scenes are projected into surround screen with seven projectors.

In all our experiments we have kept the driving speed fixed at 100 km/hr and system automatically and randomly drifts the car away from the center of the cruising lane to mimic the effects of a non ideal road surface. The driver is asked to maintain the car along the center of the cruising lane. All subjects involved in this study have good driving skill and hence when the subject is alert, his/her response time to the random drift is short and the deviation of the car from the center of the lane is small.

But, when the subject is not alert / drowsy, both the response time and the car’s deviation are high. Note that, in all our experiments, the subject’s car is the only car cruising on the VR-based freeway. Although, both response time and the deviation from the central line are related to the subject’s driving performance, in this study, we use the car’s deviation from the central line as a measure of performance of the subjects. The driving task is showing in Fig. 2-4.

Fig. 2-4: The digitized highway scene. The width of highway is equally divided into 256 units and the width of the car is 32 units. An example of the deviation event. The car cruised with a fixed velocity of 100 km/hr on the VR-based highway scene and it was randomly drifted either to the left or to the right away from the cruising position with a constant velocity. The subjects were instructed to steer the vehicle back to the center of the cruising lane as quickly as possible [61].

In order to synchronize the records of driving trajectory and raw EEG data, a JAVA program was designed to record both of them at the same sampling rate. The driving trajectory produced from the VR-based cruising environment environment program, and raw EEG data obtained by portable EEG acquisition module were transmitted to JAVA program via RS232 and Blue tooth respectively. After finishing

the experiment, both the driving trajectory and raw EEG data were saved in a text file.

Then, we can investigate the correlation between driving performance and results of unsupervised approach. The illustration of synchronization between the driving trajectory and EEG data was shown in Fig. 2-5.

Fig. 2-5: Illustration of synchronization between the driving trajectory and EEG data

2.3 EEG Preprocessing

The EEG preprocessing steps were shown in Fig. 2-6. First, a simple moving average filter (low-pass filter with a cutoff frequency of 32 Hz) was used to remove 60 Hz power line noise and other high-frequency noise. In order to simplify the computation, raw EEG data were down-sampled to sampling rate of 64 Hz. Then, a 512-point moving window we designed to save the 8 seconds EEG information, as Fig. 2-7 shown. Finally, the power in the frequency band of alpha rhythm (8 ~ 11Hz) and theta rhythm (4 ~ 7Hz) was extracted.

Fig. 2-6: Steps of EEG preprocessing

Fig. 2-7: Illustration of 8-second moving window with 7-second overlap

2.4 Unsupervised Analysis

It is recognized that the changes in EEG spectra in the theta band (4~7Hz) and alpha band (8~11Hz) reflect changes in the cognitive and memory performance [63].

Other studies have reported that EEG power spectra at the theta band [64], [65] and/or alpha band [66], [67] are associated with drowsiness, and EEG log power and subject’s driving performance are largely linearly related.

As the above researches said, these findings have motivated us to derive the alert models of the driver using the alpha-band and theta-band EEG power spectrum computed using OZ channel output recorded in the first few minutes of driving. The

choice of the OZ channel is explained in the Experimental Results section. We emphasize that the few minutes of data used to find the alert model are not necessarily collected from the very beginning of driving session because different factors, such as walking of driver by a few meters to reach the garage, may influence the EEG signal generated at the very beginning. The specific window to be used for generation of the alert model is selected by Mardia test (explained later) [68]. We assume that if the subject/driver is in an alert state, then the EEG power spectra relating to theta band (as well as that relating to alpha band) would follow a multivariate normal distribution.

The parameters of the multivariate normal distributions characterize the models.

Using the alpha-band and theta-band EEG power, we identify two normal-distribution based models. Then, we assess the deviation of the current state of the subject from the alert model using Mahalanobis distance (MD). We assume that when the subject continues to remain alert, his/her EEG power should resemble the sample data used to generate the model and hence would match the alert model or template. If the subject becomes drowsy, then its power spectra in the alpha band (and also in theta band) will deviate from the respective model and hence MD will increase. With a view to reducing the effect of spurious noise, MDs are smoothed over a 90-sec moving windows, the window is moved by 1-sec steps [61]. We then study the relationship between smoothed Mahalanobis distance and subject’s driving performance by computing the correlation between the two. Fig. 2-8 shows the overall flow of the EEG data analysis. In this figure, note that, after the models are identified, the preprocessed alpha band and theta band power data directly go to the blocks for computation of MDA and MDT, respectively. MDT and MDA are measure of deviations of the subject’s present state from the respective models, this will be

analysis with the driver’s performance.

Fig. 2-8: The flowchart of the EEG analysis method.

Chapter3 Hardware Frameworks

In this chapter, we focus on this portable system hardware. Following the design flowchart, we will introduce the design methods of hardware circuits and firmware structures steps by steps.

3.1 System Overview

In order to online measure and analyze EEG signals, the whole hardware framework of our BCI mainly contains two sub-systems: One is portable EEG acquisition module, and the other is DSP module. First, EEG signal was measured by our portable EEG acquisition module continually. After amplifying tiny EEG signals, noise except the frequency band of EEG would be removed by filters in our portable EEG acquisition module. And then, filtered EEG signals would be digitized by analog-to-digital converter, and be transited to the DSP module via Bluetooth. Here, Linux kernel µClinux was used as the operation system in DSP module to handle user’s applications. The major tasks of DSP module are to receive EEG signals via Bluetooth, and to execute the program of online drowsiness level detection, which monitor the variation of power of users’ alpha rhythm and theta rhythm. The program of online drowsiness level detection would collect EEG data under alertness for first 3 minutes to build EEG alert model, and then calculated drowsiness level by assessing the power variation of alpha and theta rhythm every 2 seconds. If the power variation exceeded the threshold of alert model, the DSP module would send warning tone of buzzer to wake up users. The whole hardware framework is shown as Fig. 3-1.

Fig. 3-1: Illustration of hardware framework of our BCI system

3.2 Portable EEG Acquisition Module

In order to be as small as possible and be easily wearable, a portable, distributive and wireless EEG headband system was designed to measure EEG signals. To reduce noise on PCB board produced by digital control circuit, the analog amplifier and digital control circuit were separated into two PCB boards. Following the previous researches [69], [70] worked, this general system was designed to minimize the circuit’s size, use a simple microcontroller to handle programs, implement filter more accurate, and etc. We also referenced some circuit designs [71]-[76]. Those circuit designs followed portable and wireless rules, separating the circuit into client and server model. In this session, we interested in the client circuit design. The portable EEG acquisition module system mainly contains five parts: (1) front-end filter circuit, (2) analog to digital converter, (3) digital controller, (4) power management circuit and (5) wireless transmission. The system block diagram is shown in Fig. 3-2.

Fig. 3-2: Block diagram of Portable EEG acquisition module

3.2.1 Front-End Filter Circuit

The front-end circuit consisted of preamplifier, and band-pass filter. The total gain of front-end circuit was set as about 5040 times with the frequency band of 0.1~100 Hz. In some references, other circuit designs liked to use unit gain filters and one variable gain amplifier. Moreover, they didn’t use a high-pass filter to cut-off the noise in low frequency band. To improve them, we designed a 3 stages high pass filter and 2 stages low pass filter to get the clear EEG information without noise. Hence, adding the gain into filter tried to minimize the total size.

A. Preamplifier

Here, instrumental amplifier LT1789-1 was used as the first stage of analog amplifier. LT1789-1 owns an ultra low input current and a high common-mode rejection ratio (CMRR) about 90dB. A high CMRR is important in applications that the signal of interest is represented by a small voltage fluctuation superimposed on a (possibly large) voltage offset, or when relevant information is contained in the voltage difference between two signals. Here, instrumental amplifier LT1789-1 provided not only the function of gain, but also that of one stage high pass filter by

2.25 times. The instrumental amplifier circuit was shown in Fig. 3-3, and the simulation of preamplifier’s gain response was in Fig. 3-4.

1 200 2.25

Fig. 3-3: Circuits of preamplifier

Fig. 3-4: Simulation of preamplifier’s gain response

B. Band pass filter

In order to more precisely reserve relevant EEG signal, a band pass filter with frequency band of 0.1Hz ~ 100Hz and with gain of 1588.75 times was designed. The band-pass filter consisted of a 2nd-order high-pass filter and a 2nd-order low-pass

filter. Here, OP-AMP AD8607 was used to construct the band-pass filter. AD8607 also owns high CMRR (about 100dB), low input current, low distortion, and no phase reversal. The band-pass filter circuit was shown in Fig. 3-5, and the simulation of those gain response was in Fig. 3-6.

Fig. 3-5: Circuits of band-pass filter

10^-4 10^-3 10^-2 10^-1 10⁰ 10¹ 10² 10³ 10⁴ 10⁵ 10⁶

The system specification of portable EEG acquisition module is listed below.

The damping ratio of second order filter was set to 0.5, thus, the Bode diagram could be smoother in two sides of 3dB point as Table 3-1 descript. The final simulation of gain response was shown in Fig. 3-7.

Table 3-1: System specification of IA, HP, and LP filter for portable EEG acquisition module

Orders Type Gain Corner Freq. Damping

Instrumental amplifier Quasi HP 2.25 0.099

High-pass filter HP 43.7 0.099 0.5

Fig. 3-7: Simulation of gain response of the portable EEG acquisition module

3.2.2 Analog to Digital Converter

The analog amplifier circuit and digital control circuit of our portable EEG acquisition module were placed individually in two PCB boards. There are some leading wires to connect both. A 12-bits analog-to-digital converter (ADC) AD7466 was used to convert continuous EEG signal of analog amplifier circuit to digitized EEG signal. Here, the micro-controller (MSP430F1611) was used to control ADC AD7466. The handshake mode between MSP430F1611 and AD7466 was shown in Fig. 3-8. The command signals and serial digitized EEG signal were transmitted via the serial peripheral interface (SPI) of MSP430F1611. The micro-controller MSP430F1611 outputs SCLK and CS signals in specific sampling rate 512Hz, and then digitized EEG signal would deliver into MSP430F1611. Each converting interval needed 16 cycles to complete transmission of digitized data, here, the data in first 4 cycles were zero, and the others were real 12-bit digitized data based on MSB.

Fig. 3-8: Handshake mode between AD7466 and MSP430F1611

Moreover, according to equation (3-1), the conversion time is about 4.7 μs, and 8-bit digitized data were transmitted every transmission cycle. And the maximum frequency of input signal of ADC was 100Hz. After calculating in equation (3-2) and (3-3), the result conforms to the equation (3-1). Thus, this system needn’t a sample

The TI micro-controller MSP430F1611 was utilized to control other parts of circuits in portable EEG acquisition module. It owns many advantages for medical application, includes ultra-low power consumption, 16-bit RISC architecture, 125 ns instruction cycle time, five power saving modes, and diversification of peripheral communication interface. The functional block diagram of MSP430F1611 was shown in Fig. 3-9.

Fig. 3-9: Functional block diagram of MSP430F1611

MSP430F1611 catch digitized EEG signal from ADC AD7466 via serial

peripheral interface with sampling rate 512Hz, and then digitized EEG data were stored into memory of MSP430F1611. Next, a moving average filter was used to remove 60-Hz power line interference before wireless transmission. The operating flow chart in MSP430F1611 was shown in Fig. 3-10.

Fig. 3-10: Operating flow chart in MSP430F1611

A. Timer Interrupt

The interrupt function of MSP430F1611 is based on inner timer/counter register, called Timer_A, to count a specific time value. The counter value TACCR0 had to be set first, as shown in Fig. 3-11. When the timer counted to the TACCR0 value, the TACCR0 CCIFG interrupt flag would be set. And when the timer counted from TACCR0 to zero, the TAIFG interrupt flag would be set. In our portable EEG acquisition module, 4MHz crystal oscillator was used as system clock of MSP430F1611, and the sub-system master clock was set to 2MHz. Therefore, the operating cycle of program in MSP430F1611 would follow sub-system master clock.

Thus, if the sampling rate of our EEG acquisition module is set to 512 Hz, TACCR0 has to be set to 3906.

TACCR0 = 2M 512 = 3906

Fig. 3-11: Timer_A up mode for interrupt function of MSP430F1611

B. SPI Mode

In synchronous mode, the USART of MSP430F1611 connects to external systems via three or four pins: SIMO, SOMI, UCLK, and STE, as shown in Table 3-2.

Table 3-2: Definition and function for pins of SPI mode SPI Mode Operation

SIMO Slave in, master out SOMI Slave out, master in

UCLK USART SPI clock

STE Slave transmit enable. Not used in 3-pin mode.

The master configuration of USART was shown in Fig. 3-12. The data transmission function of USART was initiated when transmitted data were moved to the transmit data buffer UxTXBUF. If the TX shift register was empty, then data in UxTXBUF would be moved into the TX shift register. When transmitted data were received, the received data were moved from the RX shift register to the received data buffer UxRXBUF and the receive interrupt flag URXIFGx would be set, that indicates the RX/TX operation was completed.

In our portable EEG acquisition module, the micro-controller MSP430F1611 can

cascade-connect with four front-end circuits via SPI, as shown in Fig. 3-13. Therefore, the handshake connection between Master module and Slave module needs 3-pin to transmit data: CS, SOMI, and UCLK. And there are four CS signal lines, one SOMI, and one UCLK signal line inside the leading wire.

Fig. 3-12: USART Master and external Slave

Fig. 3-13: Illustration for connection between four front-end circuits and digital control circuit

C. Moving Average

Moving average, also called rolling average or running average, is usually used to analyze a set of data points by creating a series of averages of different subsets of the full data set. Moving average can be applied to any data set, however, it is most

the setting of moving average parameters depends on the requirement of application.

Mathematically, moving average is a type of convolution and is similar to a low-pass filter used in signal processing. The moving average filter is optimal for a common task: reducing random noise while retaining a sharp step response. This makes it as the premier filter for time domain encoded signals.

Given a sequence

{ }

_i ^N₁, the output of an n-moving average is a new sequence a i₌

{ }

si i^{N n}₌₁^{− +}¹ defined as the average of subsequences of n terms. The formula of moving

averaging was shown as followings.

Fig. 3-14 shows the results of noise cancellation by using moving average. Here, a function generator was used to generate sin wave, and our portable EEG acquisition module was used to record this signal. If our portable EEG acquisition module was close to some electric instruments, the signal recorded from EEG acquisition module would easily be influenced by noise of 60 Hz power line. In the above figure of Fig.

3-14, it showed that the original sib wave had been contaminated by 60Hz power-line noise. After filtering by using moving average with 9-point moving window, we found moving average could effectively remove power-line noise, as shown in the below figure of Fig. 3-14.

_ _ SampleRate60 51260 8.53

Num of window= = = (3-6)

3000 3020 3040 3060 3080 3100 3120 3140 3160 3180 3200

Fig. 3-14: Result of noise cancellation by using moving average

D. UART Interface

In asynchronous mode, USART connected MSP430 to external systems via two external pins, URXD and UTXD. In UART mode, USART transmitted and received characters at a bit rate asynchronously to another device. Timing for each character was based on the selected baud rate of USART. Here, the transmitter and receiver used the same baud rate. For initializing UART, RX and TX had to be enable first, and then decided the baud rate of UART and disable SWRST. The required division factor N for determining baud rate was listed as followings:

BRCLK

N =baud rate (3-7)

Here, BRCLK was 4 MHz, and baud rate was 115200 bit/s. After initializing

3.2.4 Power Management

Power Management circuit in our portable EEG acquisition module includes two parts: one is power supply circuit, and the other is charging circuit.

A. Power Supply Circuit

In our portable EEG acquisition module, the operating voltage VCC was at 3V, and the virtual ground of analog circuit was at 1.5V. In order to provide stable 1.5V and 3V voltage, a regulator LP3985 was used to regulate battery voltage to 3V. Here, LP3985 is a micro-power, 150mA low noise, and ultra low dropout CMOS voltage regulator. The maximum output current can support 550mA. Furthermore, the turn-on time can reach 200μs. And a voltage divider circuit was used to divide 3V voltage into 1.5V, and a unity amplifier constructed from AD8628 was used to provide a voltage buffer. The total power supply circuit was shown in Fig. 3-15.

Fig. 3-15: Power supply circuit in portable EEG acquisition module

B. Charging Circuit

The charging circuit BQ24010DRC had integrated power FET and current sensor for 1-A charging applications. The maximum charging current can arrive to 1A. The battery’s power would be detected automatically by charging circuit and switched to charging mode when battery’s power was not enough. BQ24010DRC also protected battery to avoid over charging or over driving [77]. The charging circuit was shown in

在文檔中即時無線瞌睡偵測腦機介面系統 (頁 24-0)