System Architecture

The block diagram of the developed EEG-based BCI system is showed in Figure 5-1, which includes five units: (1) signal acquisition and amplification unit, (2) wireless data transmission unit, (3) embedded signal processing unit, (4) host system for data storage and real-time display, and (5) warning device.

Figure 5-1: The block diagram of the proposed Brain Computer Interface

The three-layer sensing module provides 4-ch biomedical signal acquisition, amplification and wireless transmission functions. The signal acquisition and amplification unit is placed on the top side of layer-1, and the 8-bit A/D converters are designed on the bottom side of layer-1. Layer-2 is the CPLD module that controls A/D and wireless modules. For wireless transmission, RF3100 module is arranged in layer-3, and the Bluetooth module is placed on the top side of layer-1. The size of the sensing module is 4.5cm×6.5cm×2.5cm, and the weight of the module with a Li-ion battery is 51g. The sensing module (including signal acquisition, amplification and wireless units) is designed to operate at 300mA with 3.7-V DC power supply and its power consumption is about 1.11W. The module can continuously operate for at least 45 hours with a commercial 16000mA/h Li-ion battery. Besides, the EEG signal processing unit (OMAP 1510) and the host system (personal computer) is powered with AC.

5.2.1 Signal Acquisition and Amplification Unit

The signal acquisition and amplification unit is applied to measure the EEG signal and to filter out the artifacts as shown in part (a) of Figure 5-2. The EEG amplifying circuit consists of a pre-amplifier (a differential amplifier) with the gain of 100, an isolated amplifier to protect subject, a band-pass filter that was composed of a low-pass filter and a high-pass filter to reserve 1-100Hz signals, a differential amplifier which had the gain of 10 or 50 (that can be chosen by a switch). The gain of the pre-amplifier (100) is larger than the amplifier (a gain of 10) because of the EEG signal is in micro-Volt level, and thus larger amplification is needed before filtering.

The capacity we used in the band-pass filter can compensate the DC-offset, thus there are no mechanism designed for the problem. The sensing module that carried by the subjects is designed to operate with a 3.7-Volt DC power supply and the DC voltage can be either supplied by a battery or AC power line. Therefore, a 60Hz notch filter is also included to eliminate the effect of the line noise in case we have to run the system with AC power.

Figure 5-2: The detail architecture of the Brain Computer Interface

5.2.2 Wireless Data Transmission Unit

Part (b) of Figure 5-2 shows the wireless data transmission unit which includes 8-bit A/D converters (parallel output, sampling rate = 768 Hz, AD-7575, Analog Device, Inc.), a Complex Programmable Logic Device (CPLD) and wireless modules.

The acquired signal is first converted from analog to digital, and then transmitted through the wireless modules. The ALTERA FLEX10K EPM 7128STC100-7 CPLD is employed to control the A/D converter and encode the data for the wireless modules. Two different transmission methods can be selected in the wireless module

of the designed BCI system according to the transmission distance in applications.

Although Bluetooth module is most commonly used in medical/clinical settings where short-distance transmission is required, however, sometimes long-distance transmission is desirable in the settings. In addition to drivers’ drowsiness estimation, the system is expected to be applied in various fields such as home cares, clinical physiological signal monitoring, exercise training, etc. Thus, we also integrated a custom-made radio-frequency transmission module with longer operation range in the developed system. RF 3100/3105 (Ancher Technology, Inc.) module is a transparent module that integrates low transmission power and high transmission rate (76800 bps) designs. The comparison of Bluetooth and RF3100/3105 is shown in Table 5-1.

Table 5-1: The Comparison of the Bluetooth and RF 3100/3105

Mode Bluetooth RF 3100/3105

Frequency Band 2.4GHz 915MHz

Transmission Distance 10m 200-600m

Transmission Direction Full-Duplex Half-Duplex

Modulation Method

Frequency Shift Keying

Frequency Shift Keying Transmission Power 0dBm (1mW) 12dBm

Interface UART、USB UART

The transmission rate is set as 19200 bps only in our final design to prevent transmission error and it can still provide 295 Hz sampling rate for 4-ch signal transmission. This setting is quite enough for general EEG signal acquisition since the most concerned frequency band of EEG signals is during 1~60 Hz. The EEG signals

are recorded at the higher sampling rate to preserve the original signal as well as possible for various applications in addition to drowsiness estimation. Thus the signals are recorded at a higher sampling rate and down-sampled in the EEG signal analysis unit.

5.2.3 Dual-core Processing Unit

It is expected that the portable biomedical devices should provide more advanced functions such as real-time feedback to the users in addition to on-line monitoring.

Therefore, more complex processing methods have been proposed for physiological analysis and they will produce more impacts if can be implemented in a real device or product. A dual-core processing unit is adopted as a platform that EEG signal processing methods as well as the intelligent technology can be implemented on it for different applications due to its powerful computation power. The operating core is Texas Instruments (TI) Open Multimedia Architecture Platform (OMAP) 1510, which is composed of an ARM925 processor and a TMS320C55x DSP processor. The DSP core was used to process EEG data and the ARM925 was used to communicate with other devices such as wireless transmission modules and TCP/IP network. The DSP Gateway is used as the cooperation structure for the communication between the two cores since these two cores have different functions as shown in part (c) of Figure 5-2.

DSP gateway is the software that makes ARM core possible to use resource of DSP core by Application Program Interface (API), and works like a small real-time kernel which manages the resource and data flow in the DSP core. With this mechanism, the DSP processor is on only when the system needs to process the EEG data. The Linux operating system (OS) is built to manage the resource of ARM core (Kobayashi and Takahashi, 2003). The functions of ARM core can be divided into three parts: (1) wireless module control, (2) TCP/IP control, and (3) DSP gateway

driver. The ARM core was selected for these tasks due to its excellent interface control ability. The process flow and task distribution in the embedded system are shown in Figure 5-3.

Figure 5-3: Software structure of the embedded system and the data processing flow

There are two processing flows running at the same time including EEG data acquisition and communication and EEG signal processing. The data processing flows are described as follows. (1) After receiving EEG data from wireless device, Task A transmits the data to network. The EEG data are then stored in the shared memory. (2) After the EEG data are stored, Task B enables the DSP module and sends data to DSP.

(3) After DSP receives the EEG data, DSP processes the data with Hanning Window and Short-Time FFT analysis. (4) After EEG analysis, DSP sends the result to ARM and ARM performs the other processes and saves the result to the share memory. (5)If

the driver drowsiness is detected through EEG analysis, the embedded system will send the triggering signal to the warning device via wireless transmission.

Since the proposed BCI system is designed to work in real-time, the signal receiving task should continue while EEG signal is on processing. An embedded multi-task scheduling mechanism system is used to manage these tasks and to ensure the accurate sampling rate for EEG signal acquisition and data process/analysis in real time (Liu Jane, 2000). The tasks are divided into three types according to their working frequency: (1) Task A- wireless device and TCP/IP control; (2) Task B- call DSP task and transmit EEG data to IPBUF buffer; (3) Task C- receiving data from IPBUF buffer and further processing of the DSP processed data. The time series diagram of the multi-task scheduling system is shown in Figure 5-4.

Figure 5-4: Time series diagram of multi-task scheduling mechanism

The working frequency of buffer IPBUF data transmission is much smaller than the working frequency of wireless device and TCP/IP control. Thus we allowed the system continuously to receive signals from the wireless module and output to the display unit through TCP/IP. The system can decide when to process other tasks by itself. With the architecture, the ARM core will not only hold and wait but also keep transmitting data from the wireless module to the display unit when the DSP core is processing the EEG signals. Inter-process communication (IPC) is also an important issue for our scheduling system since the tasks in our system are not completely

independent. In our system, ARM-Linux was used to manage tasks. Linux provides three methods for IPC including: message queue, semaphore, and shared memory.

Message queue and semaphore are not efficient enough for the proposed embedded system. Three modified communication methods are employed in the proposed BCI system for inter-process communication: (a) a novel synchronization mechanism, (b) arbitration method, and (c) sharing memory buffer (IPBUF) between processing cores.

Traditionally, the synchronization procedure is enabled when two tasks are accessing one memory block at the same time. The memory is blocked when one task is writing or reading on it thus no other task can access to the memory. The synchronization procedure unlocks the blocked memory when the first task finishes writing/reading and then sends a signal to inform other waiting tasks. It’s obvious that the mechanism can largely decline the processing speed of the processor. A new synchronization mechanism is designed to deal with the simultaneous memory access by both receiving EEG data from EEG acquisition system (Task A) and sending EEG data to DSP (Task B). When task A is accessing the memory, Task B will be idle and waste some time in waiting. Therefore, we use two blocks of memory with the same size to reduce the waiting time. When Task A is storing EEG data on memory M1, Task B can get the EEG data from memory M2 at the same time. With the modified procedure, these two tasks can execute concurrently and reduce the waiting time caused by synchronization control. Although the method consumes double memory size to complete the procedure, the required extra memory is less than 4K bytes.

Besides, we use arbitration flag register instead of semaphore due to that the speed of flag register based on shared memory is the fastest IPC method in Linux platform. In addition, using flag register can reduce the amount of memory needed because one declared variable can contain many flags.

5.2.4 Host System for Data Storage and Real-Time Display

The structure of the host system is shown in part (d) of Figure 5-2. The host system has two functions including data storage and real-time EEG signal display.

The data size of continuous EEG recordings is beyond the storage capacity of the embedded system. Thus, we have implemented a network file system to store EEG signals. Additionally, we built a graphic user interface (GUI) to show the bio-medical signals in real-time as shown in Figure 5-6 (c). The connection between the host system and the embedded system is TCP/IP protocol.

5.2.5 Warning Device

The warning device is combined in our system as shown in Figure 5-2 (e). Visual signal and audio signal can be presented to the BCI users as the feedback warning signals. The audio signals are more effective in our prior study for driver drowsiness warning since it is easier to detect audio signals than visual signals for the driver when he/she is drowsy. The efficiency of audio signals with different frequencies:

500Hz, 1750Hz, and 3000 Hz were tested in the prior study and the audio signal of 1750Hz achieves the best results. The triggering signals are sending from the dual-core EEG signal processing unit to the warning device through wireless transmission modules. The RF3100/3105 modules are half-duplex, which means the modules cannot transfer and receive signals at the same time. Since the signals are transmitted as packages, a package of warning signal can be transmitted in the time period between two packages of the acquired EEG signals for transmission. The time period between two packages might be too short for the reverse-direction transmission if the transmission frequency is set too high. To deal with the problem, the transmission frequency is set lower to leave some time duration for the data transmission from the other end.

5.3 Real-time Driver’s Drowsiness Detection and Warning

With combining on-line EEG recording and wireless transmission ability, the proposed BCI system is designed for real-time physiological signal analysis. Thus a real-time drowsiness detection method combined with an on-line warning feedback is implemented in the developed BCI system for demonstration. A dynamic operating environment is also built up to test and verify the robustness of the BCI system.

5.3.1 Experimental Design

The virtual-reality (VR) based dynamic driving environment reported in chapter 2 was used to investigate those changes on drivers’ cognitive states in long-hour driving tasks. However, the traditional wet electrode requires electrical gel to increase the conductivity between the electrode and the scalp. Thus it takes time for preparation and the subjects need to wash their hair after recording. For the convenience in practical applications, we place five dry electrodes (Chiou et al., 2006) on the driver’s forehead and the distance between two near electrodes is 1.5 cm to acquire the EEG signals for the BCI system. The EEG features related to alertness changes can be extracted from EEG signals acquired by the electrodes at the forehead according to our experiments. Therefore, placing electrode array on the forehead is a feasible and convenient strategy for drivers’ drowsiness estimation. Six dry electrodes are used to acquire EEG signals from the driver including five electrodes placed on the subject’s forehead and one placed behind the subject’s left ear. The five electrodes we placed on subject’s forehead including a common ground electrode and 4 EEG electrodes. The ground electrode is directly connected to the ground of power supply.

The EEG signals we used in further analysis are measured between the 4 EEG electrodes and the reference electrode. The dry electrodes and the sensing module can be embedded in a hat as shown in Figure 5-1. The combination of the dry electrodes

and the sensing module has gradually improved the convenience and the future applicability of the developed system.

5.3.2 Data Processing Flow and Analyzing System Design

The analysis procedure implemented in the dual-core signal processing unit is shown in Figure 5-5. The acquired EEG signals are first down-sampled to 64 Hz to reduce the calculation loading of the system and a 64-points Hanning-window is then applied to smooth the signals. The short-time Fourier transform is used to extract the time-frequency characteristics of the EEG signals and a 90-second moving average filter is applied to eliminate the noise.

Figure 5-5: Flowchart of the proposed EEG signal analysis procedure

We use principal component analysis (PCA) on the EEG power spectrum to reduce the data dimension and the computational loading of the embedded system.

The EEG features (dimension = 20) extracted by PCA are then fed into a linear regression model to estimate the driver’s drowsiness levels. The EEG signals collected in the first session were used to construct his/her drowsiness estimation model including the PCA method through off-line training by the PC. The model

including the PCA matrix is then load into OMAP 1510 to process and analysis the subject’s EEG signals in the other days in on-line and real-time for testing.

The main tasks of the embedded processor OMAP1510 was to process EEG data, wireless receiver control, and TCP/IP control. Thus we distributed these tasks into DSP core and ARM core to retain satisfied performance. According to the characteristics of the processors, the calculation of driving error estimation needed to process a long period of EEG data, so it was implemented in ARM processor. On the other hand, the Hanning windowing and short-time FFT needs heavy computation and thus implemented in the DSP core to balance the computation load. The remaining processes were implemented in ARM, because they need relatively less computations.

5.4 Experimental Results and Discussion

The feasibility of the proposed BCI system is tested in three aspects. First, we test the basic functions including bio-medical signal amplification/acquisition and wireless transmission of the system. The embedded multi-task scheduling system will then be tested and compared with the system without scheduling. Finally, the accuracy of the embedded drowsiness monitoring system will be evaluated.

5.4.1 Test of the Basic Functions

The basic functions of the developed BCI system should be tested before any further applications. The sub-systems are designed in modules to make the system more flexible for different applications. Thus the test of the system can be divided into two phases, the sub-system testing and the overall-system testing. The acquisition/amplifying unit is tested for its dependability by three steps. First, a sin wave with frequency of 5 Hz and 30uV vibration amplitude generated by the EEG simulator is used for simulation test. Then EEG signals of eye blinking were test

because the amplitude of the signals will be larger and it is easy to be recognized from acquired signal. Finally, we want to measure α wave to confirm the signal we measured is real EEG signal. When subjects take rest and close their eyes, we can continuously measure α wave which with frequency band between 8-12 Hz. Figure 5-6 (b) shows the α wave measured by the developed system.

Figure 5-6: Testing results of the acquisition/amplifying unit and the developed GUI monitoring interface

After tested and verified the subsystems, we test and verify the whole system online. In order to show the processed results of our system, we developed a Java graphic user interface (GUI) to receive the processed results of the embedded system by TCP/IP network and plot them on the display screen as shown in Figure 5-6 (c).

The left two frames are two-channel EEG raw data (Frame A, B), and the right frames show the estimated drowsiness levels (Frame C, D).

5.4.2 Embedded Multi-Task Scheduling

The multi-task scheduling mechanism was developed to ensure the accurate sampling rate for EEG signal acquisition and data analysis in real time. In order to test the performance of the embedded multi-task scheduling system, two-channel EEG

(a)

(b) (c)

data with sampling rate of 65Hz were fed to the BCI system. The time consumptions of BCI system executing 1000 cycles of signal processing procedure in DSP with and without embedded multi-task scheduling are compared and the test results are shown in Table 5-2. It takes 2425 seconds to complete the 1000 cycles without multi-task scheduling, whereas the system with multi-task scheduling needs only 1933 seconds.

Table 5-2: The Comparison of the Performance Considering the time of data

receiving

Without considering the time of data receiving

It is noted that the execution time is mainly used in receiving data. It takes 1838

It is noted that the execution time is mainly used in receiving data. It takes 1838