國 立 交 通 大 學
電子工程學系 電子研究所碩士班
碩 士 論 文
適用於中風復健之即時低複雜度腦機介面開發
Real Time Low Complexity BCI Interface for Stroke
Rehabilitation
研究生：周宗本
指導教授：張添烜 博士
適用於中風復健之即時低複雜度腦機介面開發
Real Time Low Complexity BCI Interface for Stroke
Rehabilitation
研 究 生: 周宗本 Student: Tsung-Pen Chou 指導教授: 張添烜 博士 Advisor: Tian-Sheuan Chang
國 立 交 通 大 學 電子工程學系 電子研究所碩士班
碩 士 論 文
A Thesis
Submitted to Department of Electronics Engineering & Institute of Electronics College of Electrical and Computer Engineering
National Chiao Tung University in Partial Fulfillment of the Requirements
for the Degree of Master of science in
Electronics Engineering October 2014
Hsinchu, Taiwan, Republic of China
I
適用於中風復健之即時低複雜度腦機介面開發
研究生: 周宗本 指導教授: 張添烜 博士 國立交通大學 電子工程學系電子研究所碩士班摘 要
透過腦電波（EEG）的腦機介面（BCI）來幫助中風患者復健可以讓患者的 腦波與外界產生連結、溝通，並恢復掌管運動腦區的功能。然而在傳統的方法中， 需要大量的計算複雜度才能得到可信的偵測，並且無法達成即時反應。因此，本 論文提出了適用於中風復健之即時低複雜度腦機介面開發。本論文採用透過多頻帶空間濾波器（Frequency band common spatial pattern, FBCSP）來做為特徵擷取方法，並且為了節省計算複雜度，經由分析將 EEG 通 道的數目由原本的 19 個通道降到只剩 Fz, C3, Cz, C4，僅這 4 個通道來偵測正常 人以及中風病人的運動想像意念。此外，頻帶數量也由原本的 5 個頻帶降到 3 個 頻帶，4~7 赫茲, 8~12 赫茲, 13~30 赫茲以節省計算複雜度。進一步為達成即時線 上腦機介面，我們採用以上方法配合一秒取樣時間長度，並且搭配決定轉換區的 緩衝方法來平滑我們的 BCI 控制。 最後，分析結果顯示我們降低將近 9 成的計算複雜度並在離線分析可以達到 平均 80%以上的正確率，且在線上即時分析的情況下，可以在 1 秒內偵測出受測 者的運動想像意念，而正確率維持在平均 67%。
II
Real Time Low Complexity BCI Interface for Stroke
Rehabilitation
Student: Tsung-Pen Chou Advisor: Tian-Sheuan Chang
Department of Electronics Engineering & Institute of Electronics National Chiao Tung University
Abstract
Stroke rehabilitation with EEG-based brain computer interface enables interaction through brain signals and restoration of motor function of the brain. However, conventional approaches require high complexity for reliable detection and fail to achieve real time response. This thesis proposes a real time low complexity BCI interface for stroke rehabilitation.
The proposed approach is based on the filter bank common spatial pattern (FBCSP) method. To reduce complexity, the EEG channels are reduced from 19 channels to 4 channels, Fz, C3, Cz, C4 to detect the movement intention for normal and stroke people with satisfying accuracy. Furthermore, the filter bank is reduced from five bands to three bands, 4~7Hz, 8~12Hz, 13~30Hz to reduce the complexity. A real time on-line scheme is developed with above method that uses one second time window for EEG analysis and transition region for smooth BCI control. These approaches saves 90% of computational complexity. The simulation results shows over 80% of accuracy for offline analysis, and 67% accuracy for the on-line approach with less than one second response time.
III
誌 謝
首先，要十分感謝我的指導教授，張添烜博士這兩年來的耐心、悉心的指導， 讓我學習到面對研究時，不斷克服挑戰、突破瓶頸的精神，以及許多寶貴的經驗 和知識。也很感謝您體貼學生，多虧有您照顧，我才能夠順利完成本篇論文。此 外，我也要感謝我的兩位口委，交通大學電子工程所楊家驤教授以及資訊工程所 陳永昇教授，感謝兩位於百忙中抽空前來指導，給予寶貴的意見和指教，使本論 文更加完備。 同時我也要感謝林口長庚醫院復健科的陳靖倫醫師，提供我們完善的資源來 收集腦波，以及無私的分享醫學上的研究成果與寶貴的建議和經驗，透過彼此學 術上的交流，讓本論文更豐富。接著我也要感謝實驗室學姊王琬茹，教導我使用 相關工具和分析方法，並不斷給予我信心和勇氣面對研究。感謝實驗室的學長、 同學及學弟妹們，因為你們而讓實驗室每天充滿歡笑，並給予我諸多照顧。我也 要感謝瀞瑩、柏勳、筌安、岡儒，在這兩年研究生涯陪我一起成長，在繁忙的研 究生活中，讓我可以踏實又快樂的過每一天。 感謝我的父母，無條件的包容我，支持我的學業，並給予我加油和打氣，讓 我可以完全無後顧之憂的全力完成碩士學位。最後，我也要感恩我 的師父，教 導我心安、心定，不斷讓許多貴人出現，協助我完成研究。並從許多境中，讓我 有勇氣面對不足的自己、學會交付，從中突破！ 謹以本論文獻給我的家人及所以關心我的師長朋友們IV
Contents
Contents ... IV List of Figures ... VI List of Tables ... VIII
Introduction ... 1
Motivation ... 1 Thesis Organization ... 1Background Overview ... 2
Overview of BCI ... 2 EEG Properties... 3 Signal Processing ... 5 Summary ... 8Materials and Methods ... 9
Materials ... 9
Our Dataset ... 9
Participants ... 9
Experimental Paradigm ... 10
Experimental Setup ... 11
BCI Competition IV Dataset ... 12
Participants ... 12 Experimental Paradigm ... 12 Experimental Setup ... 13 Methods... 13 Preprocessing ... 14 Feature Extraction ... 15
V
Channel Selection ... 18
Frequency Band Selection ... 22
Summary ... 27
Online BCI Implementation ... 30
Motivation ... 30
Methods... 30
Results ... 34
Conclusion and Future Work ... 65
Conclusion ... 65
Future Work ... 66
VI
List of Figures
Figure 2-1: Flow Diagram of BCI ... 2
Figure 2-2: Most used noninvasive techniques. (a) EEG (b)fMRI (c)MEG (d)NIRS [2] ... 3
Figure 2-3: The 19-Channels electrode position ... 4
Figure 2-4: Schematic Diagram of FBCSP ... 8
Figure 3-1: Experiment Paradigm ... 10
Figure 3-2: BCI Competition IV dataset 1 paradigm [19] ... 12
Figure 3-3: BCI flow diagram ... 14
Figure 3-4: Feature extraction accuracy analysis ... 17
Figure 3-5: Normal vs. Patient fMRI plot for right hand grasping & elbow flexion ... 18
Figure 3-6: Channel selection accuracy analysis ... 21
Figure 3-7: Event Related Spectral Power of different subjects ... 23
Figure 3-8: Frequency band selection accuracy analysis ... 25
Figure 3-9: Performance analysis of final result ... 28
Figure 3-10: Tasks summary analysis ... 28
Figure 4-1: Online simulate paradigm ... 31
Figure 4-2: Flow diagram of online BCI ... 31
Figure 4-3: Demonstration of each steps in methods... 33
Figure 4-4: Response time & Accuracy analysis of WH_FH_fast dataset ... 37
Figure 4-5: Comparison of the response time for the subject WH_FH_fast under different parameters ... 38
Figure 4-6: WH_FH_fast, C3CzC4, 0.5~3.5 sec ... 39
VII
Figure 4-8: WH_FH_fast, C3FzC4, 0~3 sec ... 41
Figure 4-9: WH_FH_fast, C3FzC4, 0~1 sec ... 42
Figure 4-10: WH_FH_fast, FzC3CzC4, 0~1 sec ... 43
Figure 4-11: Response time & Accuracy analysis of mrl_FH dataset ... 45
Figure 4-12: mrl_FH, C3CzC4, 0~3 sec ... 46
Figure 4-13: mrl_FH, FzC3CzC4, 0~3 sec ... 47
Figure 4-14: mrl_FH, FzC3CzC4, 0~1 sec ... 48
Figure 4-15: mrl_FH, FzC3CzC4, 0~0.5 sec ... 49
Figure 4-16: Response time & Accuracy analysis of BCI Competition IV dataset 1_b... 51
Figure 4-17: BCI Competition IV dataset 1_b, FzC3CzC4, 0~2 sec ... 52
Figure 4-18: BCI Competition IV dataset 1_b, FzC3CzC4, 0~1 sec ... 53
Figure 4-19: BCI Competition IV dataset 1_b, FzC3CzC4, 0~0.5 sec ... 54
Figure 4-20: BCI Competition IV dataset 1_b FzC3CzC4, 0~0.2 sec ... 55
Figure 4-21: TP_FH_fast, FzC3CzC4, 0~1 sec ... 57
Figure 4-22: WR_FH_fast, FzC3CzC4, 0~1 sec ... 58
Figure 4-23: YC_MI, FzC3CzC4, 0~1 sec ... 59
Figure 4-24: YC_GR, FzC3CzC4, 0~1 sec ... 60
Figure 4-25: mrc_GR, FzC3CzC4, 0~1 sec ... 61
Figure 4-26: BCI Competition IV dataset 1_f, FzC3CzC4, 0~1 sec ... 62
VIII
List of Tables
Table 2-1: EEG Frequency bands [2] ... 3
Table 3-1: Summary of subjects ... 9
Table 3-2: Summary of subjects' task ... 11
Table 3-3: Accuracy analysis respect to different approaches ... 16
Table 3-4: Accuracy analysis respect to different channel numbers ... 20
Table 3-5: Accuracy analysis respect to different frequency band ... 26
Table 3-6: Final result of datasets training & testing performance ... 28
Table 4-1: Number of N for each Datsets ... 35
Table 4-2: WH_FH_fast online parameters setting ... 36
Table 4-3: mrl_FH online parameters setting ... 44
Table 4-4: BCI Competition IV dataset 1_b online parameters setting ... 50
Table 4-5: Online accuracy analysis of all subjects ... 56
1
Introduction
Motivation
Stroke is the loss of brain function due to a disturbance in the blood supply to the brain. This disturbance is due to either blockage of a blood vessel or bleeding of blood vessels of the brain. As a result, the affected area of the brain cannot function normally, and over 80% of patients suffer from the upper limb paralysis, which severely affects the daily life of patients.
The rehabilitation methods in the past decades are by drug or active motor training; however the later one is limited by the lack of mobility of stroke patient in clinical rehabilitation. Hence, this thesis provided a Brain Computer Interface (BCI) based on EEG to classify the subjects’ motor imagery, and detect the intention of hand movement. After the detection of movement, we will trigger the robot arm to perform the rehabilitation action, by doing so, we can enhance the connection of patients’ neural network from brain to limb. This kind of approach raise the efficiency of rehabilitation, also enhance to possibility of cure. The EEG also has the advantages of high time resolution, non-invasive, and low cost, which is suitable for long-term monitor.
Thesis Organization
The rest of the thesis is organized as follow, in Chpater 2, we introduce the background knowledge of BCI system. Then in Chpater 3, we present the experiment materials and the offline analysis methods, results. In Chapter 4, proposes the online methods and analysis results of our design. Last, in Chapter 5, we conclude the thesis.
2
Background Overview
Overview of BCI
Brain Computer Interface (BCI) is a kind of neuro-prosthetic device that directly connects the neural activity of the brain with a machine, also called Brain Machine Interface (BMI).[2] The subjects can communicate the surrounding by producing neural activity that provides enough information to control an artificial device. The common flow of BCI is shown in Figure 2-1, including neuron signal recording, signal processing, and communication to artificial device.
There are two kinds of ways to access the brain’s neural activity: non-invasive and invasive techniques. The most common invasive technique is ECoG; however, it requires the surgery to acquire the signal, which has some clinical risks and is inconvenient for the subjects, but since the electrodes are closer to the brain, the recorded signal has higher amplitudes and smaller spatial scales ranging from a single neuron cell to distributed cell groups.
On the other hand, the non-invasive techniques recompense the drawback of invasive techniques, which is doesn’t need the surgery, and in principle, they have good
3
temporal resolution, but poor spatial resolution. The most common invasive techniques include EEG, fMRI, MEG, NIRS shown in Figure 2-2. In our thesis, we choose EEG to be our approach of neural activity acquisition, for its low cost, and high portability; nonetheless, EEG suffer from the blockage of cranium, cerebrospinal fluids and suffer from the artifacts, such as eye-blink and surrounding noise.
EEG Properties
EEG signal is the sum of the extra-cellular field potential caused by a large group of neurons’ activity. The signal frequency bands range from 0.1 Hz to 100 Hz, and the most common frequency bands are show in Table 2-1.
Table 2-1: EEG Frequency bands [2]
Figure 2-2: Most used noninvasive techniques. (a) EEG (b)fMRI (c)MEG (d)NIRS [2]
4
The most important frequency bands that we concerned about when analyzing the motor imagery or motor activity are β band andμ rhythm. The μ rhythm is also ranging from 8~12 Hz as α band does. However, theα band are seen in the posterior regions of the head on both sides. It emerges with closing of the eyes and with relaxation, and attenuates with eye opening or mental exertion. Theμ rhythm appears in the motion limb’s contralateral side of sensory and motor cortical areas when imaging the movement of the hands or the legs. The motor area’s position are shown in C3 and C4 in Figure 2-3. [15]
The event related desynchronization (ERD) is the power decay within a certain band due to the neuron activity that related to certain event. On the contrary, the event related synchronization (ERS) is the power ascent. μ and β band’s ERD will appear at the motion limb’s contralateral motor cortex. As the completion of motion, the ERD transfers to both side of brain. After ERD vanishes, theβ band’s ERS arises, we can view it as the power re-bounce after the ERD. Motor imagery also produces the similar pattern of ERD/ERS as the motor execution. [15]
5
Signal Processing
Signal processing is the core part of BCI system, it extracts the cognitive state’s message from the brain. However, the EEG signal is very weak after it transfer through the scalp, and it is bury in the background noise. So the main issue of signal processing is to de-noising the signal, raise the signal to noise ratio (SNR) of the EEG, and finally extract the feature for the input of future classification.
The common methods of BCI’s signal processing are, time-frequency domain processing, and spatial filtering. The methods based on time-frequency domain processing are band-pass filtering, Fourier transform, wavelet transform, etc. The methods based on spatial filtering are Laplacian spatial filtering, principle component analysis (PCA), independent component analysis (ICA), common spatial pattern (CSP), etc. [15]
The idea of time-frequency domain processing is that, ERD/ERS of μ and β band are non-phase locking event related response. Hence, the time-domain averaging method can’t extract the feature of power that varies with time. Thus, the band power is the simplest way to analyze the EEG.
The idea of spatial filtering is that by the linear combination of channels’ data, can raise the SNR of EEG signal. The large/small Laplacian is computed by subtracting the linear combination of neighbor’s electrodes from the original electrode’s signal. Large Laplacian refers to the next-nearest-neighbor electrodes, and small Laplacian refers to the nearest-neighbor electrodes.
The CSP method is to use linear transform to project the multi-channels’ EEG data into low dimensional spatial subspace with a projection matrix, of which each row consists of weights for channels. The transform matrix can maximize the variance of one class while minimize for the other.[3][15]
6
The details of CSP algorithm are listed as below, XM and XR denote the EEG signal matrices of “movement” and “rest”, with dimensions N×T, where N is the number of channels and T is the number of samples per channel. The normalized spatial covariance of the EEG can be represented as
𝑹M = 𝑿M𝑿M
′
trace(𝑿M𝑿M′) 𝑹R=
𝑿R𝑿R′
trace(𝑿𝑅𝑿𝑅′) (2-1)
𝑿′is the transpose of X and trace(B) is the sum of the diagonal elements of B. By averaging the normalized covariance over all trials of each group, we get the 𝑹M and 𝑹R . The composite spatial covariance can be factorized as,
𝑹 = 𝑹M+ 𝑹R= 𝑼0𝚺𝑼0′ (2-2) where Σis the diagonal matrix of eigenvalues, and U0 is the matrix of eigenvectors.
The whitening transformation matrix is as below. 𝑷 = 𝚺−1 2⁄ 𝑼
0′ (2-3) We transform the average covariance matrices 𝑹M and 𝑹R into SM and SR by P,
𝑺M = 𝑷𝑹M𝑷′
𝑺R = 𝑷𝑹R𝑷′ (2-4) then we can decompose the SM and SR as.
𝑺M = 𝑼M𝚺M𝑼M′ 𝑺R= 𝑼R𝚺R𝑼R′ (2-5) It can be proven that SM and SR have the same common eigenvectors and the sum of corresponding eigenvalues of these two matrices will be an identity matrix.
𝑼M = 𝑼R= 𝑼 𝚺M+𝚺R= I (2-6) The eigenvectors with the largest eigenvalues for SM have the smallest eigenvalues for
SR and the opposite is true. The transformation of whitened EEG onto the eigenvectors corresponding to the largest eigenvalues in 𝚺M and 𝚺R is optimal for separating variance in two signal matrices. The projection matrix W is denoted as
7
With the projection matrix W, the original EEG can be transformed into uncorrelated components
𝒁 = 𝑾𝑿 (2-8)
Z can be seen as EEG source components including common and specific
components of different tasks. The original EEG X can be reconstructed by
𝑿 = 𝑾−1𝒁 (2-9) Where W-1 is the inverse matrix of W. The columns of W-1 are spatial patterns, which can be considered as EEG source distribution vectors. The first and last columns of
W-1 are the most important spatial patterns that explain the largest variance of one task and the smallest variance of the other.
There are some paper proposed the advanced CSP, called Filter-Bank Common Spatial Pattern (FBCSP), which it pre-filter the EEG data into several bands, and then each band will have their own CSP to enhance the classification rate, but it will also raise the computation complexity [14]. The relation between FBCSP and CSP is shown in Figure 2-4. ICA is a kind of method of blind source separation (BSS) to recover independent sources using multi-channel observations of mixtures of those sources. [8]
8
Summary
The characteristic of EEG varies with subjects, time, equipment, mental condition, etc. This chapter provided an overview and simple background knowledge for BCI system, the main part of BCI system is in the signal processing, and there are still many ways to analyze the EEG raw data.
9
Materials and Methods
Materials
The datasets include our collected datasets from Taoyuan Chang Gung Hospital with 4 normal subjects, 2 stroke subjects, and BCI Competition IV dataset 1_b, 1_f, and 1_g.[4][18]
Our Dataset
Participants
The normal 4 subjects were 2 males, 2 females, with age ranged from 23 to 33. All participants labeled as TP, WR, WH, YC are right-handed, and are the research group members of our own laboratory. All of them are the first time users of BCI.
The first stroke subject is Mr. Cheng, aged 66, with his left brain injured. The second stroke subject is Mr. Lee, aged 46, with his right brain injured. These two patients have been rehabilitating for a while, and are in the late phase of rehabilitation treatment. The subjects are not severely damaged; both can perform easy instructions by their own will, but not fluently. All these subjects are the first time users of BCI.
Table 3-1: Summary of subjects
TP WR WH YC Mrc Mrl Aged 24 27 33 23 66 46 Gender M F M F M M Handedness R R R R R R Injured Parts X X X X left putamenal hemorrhage right ACA infarction
10
Experimental Paradigm
Figure 3-1 shows the timing scheme of the experiment. The 0~4 seconds is the rest period, with blank screen. Our rest period is longer, so that it has the equal length of movement period, for the convenience of analysis tool. Then at the 4s, the screen shows a fixation cross to hint the subjects to concentrate for the coming cue. Finally, the 5~9s is the movement period, the screen will show an arrow pointing left or right with equal chance.
For each subject, we collect 3 to 4 runs, with 80 trials (normal subjects), or 60 trials (stroke subjects) each run. The movement tasks in our test are listed as below. (1) FH_fast: fast finger tapping, about 4 times per second.
(2) FH_comf: finger tapping with the comfortable speed, about 1~2 times per second. (3) GR: grasp, we assume that grasp involves more muscle movement, and larger movement than finger tapping, it might result in stronger EEG waveform.
(4) MI: motion imagery, imagine raising your hand. (5) MIGR: motion imagery, imagine hand grasping.
(6) TMP_fast: thumb press, this is for the stroke subject, because he had difficulty Figure 3-1: Experiment Paradigm
11
tapping his finger, we modify the movement from finger to thumb.
(7) PF: upper limb pushing forward, this is also for the stroke patient, because there is a push forward action in the rehabilitation movement, also the doctor said that pushing movement have stronger EEG waveform, we assume it is easier for the movement detection. The summary of subject’s task are listed in Table 3-2.
The task for each subject is slightly different though the whole collection series, we keep modifying the task movement to get better result, and the collection of one subject is finished within a day.
Table 3-2: Summary of subjects' task
Experimental Setup
The EEG was collected by the Mitsar EEG-201 amplifier (http://www.mitsar-medical.com/eeg-machine/eeg-amplifier-compare/) with 19 channels showed in Figure 2-3. The sampling rate is at 250Hz, and the recorded EEG waveform is passed to the compatible software, called WinEEG. The participants were asked to sit on a chair toward the screen and to fix their body in a room with weak light. The instruction were presented by visual cue from the computer screen with the software, called PSYTASK. Also, the subjects were instructed to minimize eye blinking and to avoid physical movement throughout the EEG recording progress.
TP WR WH YC mrc mrl FH_fast(TMP_fast) V V V V V V FH_comf V V V GR V V V V MI(MIGR) V V V V V V PF V
12
BCI Competition IV Dataset
Participants
The BCI competition IV dataset 1_b, 1_f, and 1_g were provided by Berlin BCI group [18], the reference link about the detail information is at: http://www.bbci.de/competition/iv/. The subjects were all healthy.
Experimental Paradigm
The BCI competition IV datasets, there are two kinds set of data. One is for the calibration (training), and one is for evaluation (testing). For calibration datasets, in the first two runs, arrows pointing left, right, or down were presented as visual cues on a computer screen, with each trial 8 seconds. First, the screen will show 2s fixation cross to inform the subjects to be attention. Then, cues were displayed for a period of 4s during which the subject was instructed to perform the cued motor imagery task. Finally, 2s of blank screen shown in the center of the screen. Then, for the evaluation datasets, there are 4 runs, the motor imagery tasks were cued by soft acoustic stimuli (words left, right, and foot) for periods of varying length between 1.5 and 8 seconds. The end of the
13
motor imagery period was indicated by the word stop. Intermitting periods had also a varying duration of 1.5 to 8s. Like shown in Figure 3-2.[18]
Experimental Setup
The BCI Competition IV datasets are recorded by 64 channels, 1000Hz sampling rate, and 2 classes (+idle state).
Methods
After the EEG collection by the WinEEG software, we transfer the EEG signal to MATLAB platform though the format called EDF plus. The reason that we analyze the waveform on MATLAB platform is that we are familiar with the toolbox called EEGLAB (http://sccn.ucsd.edu/eeglab/)[6] and BCILAB (http://sccn.ucsd.edu/wiki/BCILAB)[7] these two toolbox are based on MATLAB, and we have more transparency and plasticity to modify the code we want, and subtly perform the instruction we gave.
From EDF format to MATLAB, first we cut the data out, then use the EEGLAB to import new data, then we add the labels of each channel, and use the default channel locations of EEGLAB. Last, we load the event types (left / right) and latencies through .txt file exported from WinEEG. After all these steps, we save it as .set file of EEGLAB for the future processing by BCILAB toolbox.
Then we introduce the BCILAB toolbox to do the process. BCILAB is a powerful toolbox based on EEGLAB, and it is also compatible with MATLAB. It comprise all the key components for BCI analyzing, like Figure 3-3. All the components are constructed in module form, also with GUI, and each module has different parameters. We can choose different modules combine together, and define the parameters to fulfill the BCI processing that we want. BCILAB also combines some machine learning module function to classify the prediction, and finally, it has many ways (mean square
14
error, kullback-leibler divergence, negative log-likelihood …) to evaluate the performance.
Preprocessing
The purpose of preprocessing is to make the raw EEG data more clear, and reject the artifacts (eye blink, background noise …). The most common way includes band-pass filtering, independent component analysis (ICA) [1][8][9], principle component analysis (PCA)[8], or the simplest way is to manual removal of fuzzy EEG raw data, which needs some experience, simple but lack of validity.
However, we don’t do any preprocessing in our methods, and there are several reasons:
1. For the band pass filtering, in section 3.2.2, we will mention our approach: FBCSP (filter bank common spatial pattern), it already separates the raw EEG data to different frequency bands, so there is no need to do the preprocessing of band pass filtering. 2. For ICA and PCA, the algorithm transform the raw EEG data into the components (with the same number of channel numbers), and we have to manually reject the “noise” component, or manually select just the C3/C4 component we want, and in some case it
Figure 3-3: BCI flow diagram
Subject Preprocessing selectionFeature Classifier
Preprocessing Selected features Classfier
BCI Training phase
Classifier model Result/Action Action/ MI Evaluation phase EEG EEG features Result
15
is hard and has a lot uncertainty to pick up the right component we want; in addition, although there are some algorithms that can select the component automatically, but they require too much computation resources, which is not suitable for the purpose of this thesis. Also, the EEG data varies with time; hence, the ICA varies with time greatly. It takes quite amount of time to keep calculating the ICA with the updating income of data while in the online application. Due to the above reasons, we don’t do the ICA/PCA in the preprocessing part, and we still get quite satisfactory results.
Feature Extraction
For the training part, I choose the first 40% of the data to become the training data, and the rest 60% are for the testing data. The target marker for training comes from the event (0 represents the rest, 1 represents the movement) that labels on the data. The rest event is at 0sec, and the movement event is at the 5sec as shown in the Figure 3-1, so we extract the epoch from 0.5sec to 3.5sec for each marker event.
Then we choose a feature extraction approach to transform the EEG data to the most suitable feature for further classification by machine learning. There are many approaches in BCILAB, the most common approaches are Log-Bandpower[15], Common Spatial Pattern (CSP)[3][10][11], and Filter-Bank Common Spatial Pattern (FBCSP)[5][12]-[14].
After choosing the approach, we combine the classification to train the model, then we can use the model to test the testing data, and get the accuracy. The following Table 3-3 shows the result. All the three approaches are with 19 channels, LDA classifier, 0.5~3.5 sec epoch extracted. For Log Power and CSP approaches, the algorithm filtered the 7~30Hz, and FBCSP filtered 0.5~3; 4~7; 8~12; 13~30; 31~42Hz by default parameters.
16
Table 3-3: Accuracy analysis respect to different approaches Approach Dataset Log Power CSP FBCSP TP_FH_fast 70.83% 82.29% 83.33% TP_FH_comf 65.48% 70.24% 64.29% TP_MI 53.13% 62.50% 56.25% WR_FH_fast 59.38% 84.38% 96.88% WR_FH_comf 59.38% 60.42% 85.42% WR_MI 66.67% 71.88% 82.29% WH_FH_fast 90.48% 76.19% 92.86% WH_FH_comf 82.29% 77.08% 91.67% WH_MI 72.92% 65.63% 84.38% WH_GR 88.54% 92.71% 87.50% YC_FH_fast 94.79% 96.88% 98.96% YC_MI 92.71% 96.88% 96.88% YC_GR 100.00% 96.88% 98.96% mrc_GR 62.50% 88.89% 91.67% mrc_MIGR 65.28% 81.94% 77.78% mcr_TMP_fast 65.28% 79.17% 79.17% mrl_FH 88.89% 94.44% 93.06% mrl_GR 75.00% 88.89% 84.72% mrl_MIGR 93.06% 98.61% 98.61% mrl_PF 87.50% 91.67% 94.44% Average 76.70% 82.88% 86.95%
17
As we can see from the Table 3-3, FBCSP outperforms the rest algorithms greatly; however, the FBCSP is the 5 more times the computation and calculation time of CSP algorithm because of the 5 times more frequency band to compute. Therefore, in order to cost down the computation time and complexity to reach our purpose and goal of immediate online classification with almost 80% accuracy rate, we try out to modified 1. Channels selection 2. Frequency bands selection parameter to fit our requirement.
From Figure 3-4, we can see that CSP and FBCSP both are good approaches for stroke patients, but FBCSP performs better for normal patients, this is because of that normal subjects have more clear frequency bands than stroke subjects, we will mention this furthermore in 3.2.4.
18
Channel Selection
(a) (b)
(c) (d)
In Figure 3-5, provided by [16], (a) is the normal subject’s right hand grasping fMRI plot, while (b) is the stroke patient subject’s right hand grasping fMRI plot. (c) is the normal subject’s elbow flexion fMRI plot; on the other hand, (d) is the stroke patient subject’s elbow flexion fMRI plot. We can clearly see that for normal subjects,
19
the involved area in brain concentrate around the Cz, C3 (right hand movement, triggered by contra-lateral brain). However, for stroke subjects, the response area spread all over the brain, but still includes C3, Cz, C4, and even Fz. Therefore, according to Figure 3-5, we try to cost down the channel computation in our FBCSP algorithm, but still maintaining the accuracy.
Table 3-4 is the accuracy analysis respect to different channel number, with FBCSP algorithm, training epoch extracted from each event’s 0.5~3.5 sec. 9 channels refer to central 9 channels (F3, Fz, F4, C3, Cz, C4, P3, Pz, P4), and the 6 channels refer to front-central 6 channels (F3, Fz, F4, C3, Cz, C4). The interesting result shows that while we choose only C3, Cz, C4, the accuracy still maintains over 80%, but with great channel number reduction. Also proves the results in Figure 3-5 that the activation area involved while hand motion or MI (motion imagery) are mostly in C3, Cz, C4.
From Figure 3-6, we also tried C3, Fz, C4 these 3 channels to analyze, and we found that comparing to C3, Cz, C4, the normal subjects (TP, WR, WH, YC) showed great progress in the accuracy, but the dataset mrc_GR and mcr_TMP_fast drop dramatically. Due to the above reason, and we will show in the next chapter that normal subject with Fz channel in training will reduce the response time of online classification. Hence, we finally decided to use 4 channels (Fz, C3, Cz, C4) instead of 3 channels (C3, Cz, C4 or C3, Fz, C4) to train the data for the convincing results.
20
Table 3-4: Accuracy analysis respect to different channel numbers Channel Dataset 19 9 6 3(C3,Cz,C4) 3(C3,Fz,C4) 4(Fz,C3,Cz,C4) TP_FH_fast 83.33% 87.50% 78.13% 78.13% 79.17% 78.13% TP_FH_comf 64.29% 66.67% 65.48% 69.05% 67.86% 69.05% TP_MI 56.25% 47.92% 47.92% 50.00% 55.21% 54.17% WR_FH_fast 96.88% 94.79% 88.54% 81.25% 88.54% 85.42% WR_FH_comf 85.42% 83.33% 64.58% 69.79% 70.83% 70.83% WR_MI 82.29% 83.33% 89.58% 84.38% 83.33% 85.42% WH_FH_fast 92.86% 91.67% 92.86% 89.29% 94.05% 84.52% WH_FH_comf 91.67% 86.46% 88.54% 80.21% 85.42% 81.25% WH_MI 84.38% 86.46% 80.21% 76.04% 82.29% 80.21% WH_GR 87.50% 71.88% 85.42% 78.13% 78.13% 72.92% YC_FH_fast 98.96% 97.92% 88.54% 94.79% 92.71% 91.67% YC_MI 96.88% 93.75% 90.63% 85.42% 94.79% 86.46% YC_GR 98.96% 98.96% 98.96% 97.92% 95.83% 96.88% mrc_GR 91.67% 91.67% 90.28% 86.11% 50.00% 58.33% mrc_MIGR 77.78% 83.33% 90.28% 79.17% 80.56% 90.28% mcr_TMP_fast 79.17% 77.78% 75.00% 70.83% 69.44% 75.00% mrl_FH 93.06% 94.44% 87.50% 90.28% 86.11% 86.11% mrl_GR 84.72% 87.50% 79.17% 83.33% 79.17% 83.33% mrl_MIGR 98.61% 98.61% 90.28% 90.28% 88.89% 84.72% mrl_PF 94.44% 97.22% 93.06% 86.11% 84.72% 90.28% Average 86.95% 86.06% 83.25% 81.02% 80.35% 80.25%
21
22
Frequency Band Selection
(a)
23 (c)
(d)
24
Figure 3-7 shows the Event Related Spectral Power (ERSP) of different subjects, the x-axis of plot is time from 0~10,000(ms), the y-axis is the frequency from 0~50(Hz), and the meter aside is the power of that channel. Plot (a), (b) are the representative subjects of normal subjects with different tasks shown in row, and with different hand movement’s C3/C4 channel shown in column. We clearly see that, at time 4~10 sec, there are strong Event Related De-synchronization (ERD, with power decrease) in 8~12Hz (μband) marked by the red circle, which strongly proves that while motion or even motion imagery (MI) there are strong ERD in C3/C4’sμband.
Plot (c), (d) show the two stroke patient subjects’ ERSP. The energy band scatters at all frequencies, not clear as normal subjects’, but still, we can see that there are strongerμband ERD at the un-damaged brain area no matter what hand moves. In plot (c), subject mrc’s damaged area is at left hemisphere of brain, so it shows C4 (right hemisphere of brain) has stronger ERD; likewise, plot (d), subject mrl’s damaged area is at right hemisphere of brain, so it shows C3 (left hemisphere of brain) has stronger ERD.
The default frequency bands of BCILAB’s FBCSP algorithm are 0.5~3Hz; 4~7 Hz; 8~12 Hz; 13~30 Hz; 31~42Hz, these 5 bands, including the μ rhythm(8~12Hz), and β(13~30Hz), which are known for the most related bands while motion &MI. According Figure 3-7 above, we select different bands to analyze, to see whether we can have a reduction of these frequency bands or not. We have several combination of bands listed below:
(A): 0.5~3;4~7;8~12;13~30;31~42Hz, default frequency bands of FBCSP (B): 0.5~3;4~7;8~12;13~30Hz, exclude the 31~42Hz
(C): 4~7;8~12;13~30;31~42Hz, exclude the 0.5~3Hz
(D): 4~7;8~12;13~30Hz , exclude both lowest & highest frequency bands (E): 8~12;13~30;31~42Hz, exclude the lowest 2 bands
25
(F): 0.5~3;8~12;13~30Hz, lowest band &μ, β bands (G): 8~12;13~30Hz , μ, β bands
(H):4~8;8~12;12~16… 36~40Hz, four frequencies bands each, from 4~40Hz
The result of testing these combination of bands are listed in Table 3-5 and Figure 3-8, with FBCSP algorithm, 0.5~3.5 seconds epoch extracted, 4 channels (Fz, C3, Cz, C4) trained for the model, and LDA for the classifier. The result shows that the combination of (D) had best result in normal subjects, and quite satisfying result in stroke subjects. The result was not surprising, because we can see from Figure 3-7 that exclude the 8~12Hz, there are still some ERD in the 4~7 Hz. Thus, finally we decide to choose 4~7Hz, 8~12Hz, 13~30Hz to be our training frequency bands.
26
Table 3-5: Accuracy analysis respect to different frequency band
Frequency Dataset (A) (B) (C) (D) (E) (F) (G) (H) TP_FH_fast 78.13 78.13 77.08 82.29 81.25 81.25 80.21 77.08 TP_FH_comf 69.05 65.48 69.05 71.43 67.86 65.48 70.24 69.05 TP_MI 54.17 52.08 55.21 51.04 52.08 45.83 48.96 53.13 WR_FH_fast 85.42 81.25 85.42 82.29 81.25 79.17 82.29 93.75 WR_FH_comf 70.83 70.83 67.71 73.96 69.79 71.88 68.75 68.75 WR_MI 85.42 81.25 86.46 81.25 76.04 78.13 73.96 87.50 WH_FH_fast 84.52 94.05 83.33 90.48 77.38 95.24 89.29 79.76 WH_FH_comf 81.25 84.38 75.00 77.08 70.83 83.33 73.96 73.96 WH_MI 80.21 76.04 78.13 77.08 79.17 77.08 78.13 78.13 WH_GR 72.92 83.33 77.08 86.46 69.79 82.29 82.29 61.46 YC_FH_fast 91.67 94.79 90.63 93.75 89.58 91.67 93.75 89.58 YC_MI 86.46 87.50 85.42 86.46 88.54 85.42 88.54 85.42 YC_GR 96.88 94.79 97.92 94.79 100.00 93.75 94.79 98.96 mrc_GR 58.33 86.11 62.50 90.28 61.11 91.67 93.06 84.72 mrc_MIGR 90.28 88.89 91.67 91.67 90.28 86.11 91.67 94.44 mcr_TMP_fast 75.00 75.00 72.22 73.61 70.83 76.39 72.22 70.83 mrl_FH 86.11 87.50 86.11 88.89 90.28 88.89 91.67 84.72 mrl_GR 83.33 81.94 81.94 81.94 81.94 84.72 81.94 86.11 mrl_MIGR 84.72 84.72 84.72 81.94 83.33 84.72 81.94 86.11 mrl_PF 90.28 87.50 81.94 81.94 83.33 90.28 84.72 86.11 Average 80.25 81.78 79.48 81.93 78.23 81.66 81.12 80.48
27
Summary
After all the analysis above, we find few parameters listed below. 1. Algorithm: FBCSP
2. Channels selection: Fz, C3, Cz, C4 (4 channls) 3. Frequency band selection: 4~7Hz, 8~12Hz, 13~30Hz
These parameters are to fit our application: fast, less computation, but still convincing accuracy results shown in Figure 3-9(the complexity is compared to 19 channels 5 frequency bands FBCSP), for online instant react BCI system. Furthermore, we hope to develop this BCI system into a portable, wearing device by the means of ASIC IC design. First, the epoch is extracted respect to each event at 0.5~3.5 seconds using the 4 channels (Fz, C3, Cz, C4) EEG data, and the prior 40% data are used for training, the rest 60% are for the testing. Then we select the FBCSP algorithm, with frequency bands cut at 4~7; 8~12; 13~30 Hz to be our feature extraction approach. Finally, the features are fed into the LDA classifier to get the training model. The BCILAB uses the module called bci_train to train the data by the approach we selected. After the training, the model compares the event 0.5~3.5 seconds’ data that labeled by the event marker to classifies this epoch’s type (rest or movement), gives it the percentage to rest event or motion event, and uses 5-fold cross validation and mean square error (MSE) to evaluate the performance. Then, we can use the rest 60% data for the testing, same as evaluating the training data, getting the results of mean square error. The results of training and testing are listed below in Table 3-6.
We see that there are still great variation among the subjects, some tasks have better performance for some subjects, while others don’t, and each kind of task gets different performances too, shown in Figure 3-10 (the task accuracy are the average of all subjects who have done that kind of task). Nonetheless, our total average result had
28
over 80% of accuracy both in normal subjects and stroke subjects. From [17], we can know that accuracy from 43~58% are accuracy by chance, which means our results are not at chance, and convincing.
Figure 3-10: Tasks summary analysis Figure 3-9: Performance analysis of final result
29
Table 3-6: Final result of datasets training & testing performance training testing TP_FH_fast 79.62% 82.29% TP_FH_comf 89.39% 71.43% TP_MI 74.87% 51.04% WR_FH_fast 82.95% 82.29% WR_FH_comf 56.41% 73.96% WR_MI 89.10% 81.25% WH_FH_fast 81.97% 90.48% WH_FH_comf 92.05% 77.08% WH_MI 87.05% 77.08% WH_GR 87.31% 86.46% YC_FH_fast 90.51% 93.75% YC_MI 76.28% 86.46% YC_GR 95.26% 94.79% mrc_GR 87.56% 90.28% mrc_MIGR 75.78% 91.67% mcr_TMP_fast 76.89% 73.61% mrl_FH 89.33% 88.89% mrl_GR 98.00% 81.94% mrl_MIGR 96.00% 81.94% mrl_PF 81.11% 81.94% Average 84.37% 81.93%
30
Online BCI Implementation
Motivation
Our research topic is a project that cooperate with Chang Gung Hospital. We wish to develop a BCI device to help stroke patients’ rehabilitation by the means of the online instant BCI response triggering the robot arm that helps the stroke patient to move their hand, enhancing the neural network communication from brain to limb. Hence, the online BCI classification is the crucial part to fulfill our goal. Different from the previous chapter that does the offline analysis, in this chapter, we want to develop an online analysis method with quick response, high accuracy, based on the parameters and algorithm of the pervious chapter.
Methods
The approach is the same as that in Chapter 3, we choose the following parameters: 1. Algorithm: FBCSP
2. Channels selection: Fz, C3, Cz, C4 (4 channls) 3. Frequency band selection: 4~7Hz, 8~12Hz, 13~30Hz
Then we can train the model by the module, bci_train in BCILAB as described in the Chapter 3. After that we take the rest 60% data and the training model to another BCILAB module called onl_simulate. This module has the parameter “sampling rate”. It will use this sampling rate to update the simulate result with the same length of training model. We choose 20Hz to be our sampling rate in our approach. The training phase paradigm is shown in Figure 3-1, and the online simulated paradigm is shown in Figure 4-1.
31
After the onl_simulate module process, we get a vector of cognitive state at various specified time points, called predictions, p(i), which is the simulated result with respect to time. p(i) is also the percentage of movement intention: 1 refers to movement, and 0 refers to rest, like the prediction line in the top picture of Figure 4-3. However, the prediction results are a continuous, non-smooth line with many glitches, which will result in abrupt movement for the robot arm. To avoid this, we will transform it to a discrete (just 0 and 1) smooth line like the modified classification result showed in the bottom picture of Figure 4-3, to better control the robot arm.
Figure 4-2: Flow diagram of online BCI Figure 4-1: Online simulate paradigm
32
The whole algorithm for smooth movement is listed below, also the flow diagram is shown in Figure 4-2. First, we set a variable, called “threshold(i)”, which is the combination of constant value, 0.4, and the root mean square of previous 10 points (in our 20Hz sampling rate case, 10 points refers to 0.5 seconds data). As we set the threshold(i) to be the combination of previous data and a constant, the threshold(i) can be updated with time, which gains more flexibility and accommodation to conquer the variation of data though time.
threshold(i) = 0.4 × 80% + √∑𝑖𝑖−9𝑝(𝑖)2
10 × 20% (4-1)
Then we define a counter, “C(i)”, with the range from 0~18, that counts how many times the p(i) is larger than threshold(i), or smaller than threshold(i). We accumulate the number that p(i) is larger than threshold(i), in such way, the glitches in p(i) can be eliminated, and therefore, we can get a smoother waveform like the middle plot in Figure 4-3. This flow is shown as below.
{
𝑖𝑓 𝑝(𝑖) ≥ 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑(𝑖), 𝑎𝑛𝑑 𝐶(𝑖 − 1) < 18 𝐶(𝑖) = 𝐶(𝑖 − 1) + 1; 𝑒𝑙𝑠𝑒 𝑖𝑓 𝑝(𝑖) < 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑(𝑖), 𝑎𝑛𝑑 𝐶(𝑖 − 1) > 0 𝐶(𝑖) = 𝐶(𝑖 − 1) − 1; 𝑒𝑙𝑠𝑒 𝐶(𝑖) = 𝐶(𝑖 − 1);
(4-2)
Finally, we can generate the classification result by “C(i)”. If the C(i) value is larger than 10, we set the final modified classification(i) result to be 1 (movement). On the contrary, if the C(i) value is lower than 3, we set the final modified classification(i) result to be 0 (rest). This flow is shown as below.
{
𝑖𝑓 𝐶(𝑖) ≥ 10 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛(𝑖) = 1; 𝑒𝑙𝑠𝑒 𝑖𝑓 𝐶(𝑖) < 3 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛(𝑖) = 0; 𝑒𝑙𝑠𝑒 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛(𝑖) = 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛(𝑖 − 1);
33
The reason why we left a range between 10 to 18, and 0 to 3 of value C(i), shown in Figure 4-3, is that we can avoid the glitches in C(i). We leave it as a buffer, so that the modified classification(i) won’t change too easily. For example, there is a glitch in the third trapezium of the middle plot, if there aren’t any buffer range, the modified classification(i) may drop to 0 due to the glitch. The parameters at each equation are empirical, and are tuned to get the best performance.
34
Results
In this section, we presented the result in two measures, response time and accuracy. The smoothing movement method will result in the delay of the response time. Hence, in order to compensate the delay, we tried shorter training epoch time to be our training model. In Chapter 3, we select 0.5~3.5 seconds, total 3 seconds length of data to be our training epoch. Then it come to us that if we select shorter training epoch length, the higher the percentage of new data occupies the epoch data while the onl_simulate keep updating the data, and this results in the shorter of response time.
We draw the online classification figure from 0 to 200 seconds, in these 200 seconds, there are averagely 10~20 movement events that can correctly classified, the remain 2~3 movement events that can’t be classified are marked as red circle in the plot. The response time is computed by averaging these recognizable movement event’s latencies between true label and the modified classification result, like the equation Response Time =𝑁1∑(𝑡𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛− 𝑡𝑡𝑟𝑢𝑒 𝑙𝑎𝑏𝑒𝑙) (4-4), where tclassification refers to the time point that rest to movement of classification result, and ttrue label refers to the time point that rest to movement of true label. The number of movement events (N) that can be recognizable are listed in Table 4-1. However, the response time showed in Response Time = 1
𝑁∑(𝑡𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛− 𝑡𝑡𝑟𝑢𝑒 𝑙𝑎𝑏𝑒𝑙) (4-4) doesn’t consider the computation time of processing unit. In our MATLAB
analysis, 200 sec data, with 20 Hz sampling rate, it takes about 0.0396 sec to process the online classification, which means it takes 10-5 sec to compute every sample point. The accuracy is computed by sample by sample comparing the true label and modified classification result at the range of 1sec after the event marker till the end of event. This kind of analysis is referred to the BCI competition IV’s accuracy analysis [4][18]. We select three datasets for analysis, WH_FH_fast, mrl_FH, and BCI
35
competition IV dataset 1_b in represent of three kind of subjects, normal, stroke, and un-cued for simplicity.
Response Time =𝑁1∑(𝑡𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛− 𝑡𝑡𝑟𝑢𝑒 𝑙𝑎𝑏𝑒𝑙) (4-4)
Accuracy = (𝑁𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛==𝑡𝑟𝑢𝑒 𝑙𝑎𝑏𝑙𝑒
𝑁𝑡𝑜𝑡𝑎𝑙 )1 s 𝑎𝑓𝑡𝑒𝑟 𝑒𝑣𝑒𝑛𝑡 𝑚𝑎𝑟𝑘𝑒𝑟 ~ 𝑒𝑛𝑑 𝑜𝑓 𝑒𝑣𝑒𝑛𝑡 (4-5)
Table 4-1: Number of N for each Datsets
Dastsets number of N WH_FH_fast, C3CzC4, 0.5~3.5 sec 19 WH_FH_fast, C3FzC4, 0.5~3.5 sec 19 WH_FH_fast, C3FzC4, 0~3 sec 19 WH_FH_fast, C3FzC4, 0~1 sec 19 WH_FH_fast, FzC3CzC4, 0~1 sec 19 mrl_FH, C3CzC4, 0~3 sec 18 mrl_FH, FzC3CzC4, 0~3 sec 19 mrl_FH, FzC3CzC4, 0~1 sec 19 mrl_FH, FzC3CzC4, 0~0.5 sec 17 BCI Competition IV dataset 1_b, FzC3CzC4, 0~2 sec 16 BCI Competition IV dataset 1_b, FzC3CzC4, 0~1 sec 16 BCI Competition IV dataset 1_b, FzC3CzC4, 0~0.5 sec 17 BCI Competition IV dataset 1_b FzC3CzC4, 0~0.2 sec 14
36
First, the WH_FH_fast dataset, we selected different parameters to test the response time and accuracy. The parameters are listed below in Table 4-2.
Table 4-2: WH_FH_fast online parameters setting
Channels Extracted Time Window
para1 C3CzC4 0.5~3.5sec
para2 C3FzC4 0.5~3.5sec
para3 C3FzC4 0~3sec
para4 C3FzC4 0~1sec
37 (a) (b)
38
As we can see from Figure 4-4 that as we change the channels from Cz to Fz, the response time and accuracy enhances. Furthermore, we move the epoch to former part (0 sec started, not 0.5sec started) the performance also increases. Finally, the 1 sec epoch’s performance is better than the 3 sec epoch’s performance, and the reason are stated as before, the shorter the epoch is, the greater percentage the new data occupies, which results in faster updating of classification. We showed the progress of response time in Figure 4-5, and the full waveform are shown in Figure 4-6~Figure 4-10.
39
40
41
42
43
44
Next, we show the analysis of mrl_FH dataset. This subject is a stroke patient; however, the performance is quite satisfying. The parameters that we tried out are listed in Table 4-3, and the response time and accuracy are shown in Figure 4-11.
Table 4-3: mrl_FH online parameters setting
Channels Extracted Time Window
para1 C3CzC4 0~3sec
para2 FzC3CzC4 0~3sec
para3 FzC3CzC4 0~1sec
para4 FzC3CzC4 0~0.5sec
45 (b)
From the result in Figure 4-11, the addition of Fz channel result in worse performance in response time and accuracy for stroke patient, which is in accordance with the result in Figure 3-6. Notwithstanding, we can compensate the performance by picking shorter time window, and the outcome shows it promising. The 1sec window had better performances, but the 0.5 sec window drops the performance in accuracy aspect, the reason may be the over-short period of time window, which we can see the fuzzy prediction result in Figure 4-15.
46
47
48
49
50
Finally, we show the analysis of BCI Competition IV dataset 1_b dataset, which is an un-cued dataset with the evaluation data event period ranging from 1.5sec~8sec. The un-cued dataset has bigger challenge for the online analysis, for the uncertainty at event period; however, the result of BCI Competition IV dataset 1_b are still quite satisfying. The parameters that we tried out are listed in Table 4-4, and the response time and accuracy are shown in Figure 4-16.
Table 4-4: BCI Competition IV dataset 1_b online parameters setting Channels Extracted Time Window
para1 FzC3CzC4 0~2sec
para2 FzC3CzC4 0~1sec
para3 FzC3CzC4 0~0.5sec
para4 FzC3CzC4 0~0.2sec
Since the BCI_1b subject is a normal subject, we didn’t show the effect of channel selection, just directly set the channels to FzC3CzC4, and focus on the effect of extracted epoch’s length. The result in Figure 4-16 shows that the response time generally decreases with the shorter epoch length, but it re-bounce when the epoch is too short. Equally, the accuracy is better when epoch is shorter, but it also drops dramatically when the epoch is too short. In summary, the performance shows best at 1sec epoch length in both response time and accuracy aspect, and the 0.2sec epoch length is too short so that it has too fuzzy prediction result and lacks of the stability shown in Figure 4-20.
51 (a)
(b)
52
53
54
55
56
After the analysis above, we finally set the online prediction parameters as listed below, and we show the accuracies for the rest of the datasets in Table 4-5, waveforms in Figure 4-21~Figure 4-27 (WH_FH_fast, mrl_FH, BCI Competition IV dataset 1_b are shown above in Figure 4-10, Figure 4-14, Figure 4-18). Also mention that we had analyzed one motion imagery dataset, YC_MI, which performance is not so much different from others.
1. Algorithm: FBCSP
2. Channels selection: Fz, C3, Cz, C4 (4 channls) 3. Frequency band selection: 4~7Hz, 8~12Hz, 13~30Hz 4. Extracted time window: 0~1sec
Table 4-5: Online accuracy analysis of all subjects Datasets Accuracy TP_FH_fast 60.53% WR_FH_fast 68.78% WH_FH_fast 73.21% YC_MI 67.95% YC_GR 69.34% mrc_GR 66.79% mrl_FH 75.34% BCI Competition IV 1_b 73.83% BCI Competition IV 1_f 54.22% BCI Competition IV 1_g 57.94%
The online sample by sample performance is generally 10%~15% lower than the offline performance in Chapter 3, because of the response time latency, and the property of online analysis. The results in Table 4-5 show that the accuracy performance generally are above 60%, but few of them are not. We deduced the main reason is the variance among the subjects, we can also see that the variance between the tasks (GR and MI), but still, the motion imagery task still has satisfying performance.
57
58
59
60
61
62
63
64
Finally, we compare our performance with [17] in Table 4-6. However, the accuracy in [17] is not fully well defined, so the number are just for reference. Our algorithm has slightly poor performance in accuracy compared to [17], but the complexity in channels and frequency bands dramatically costs down. Furthermore, we provided the algorithm that can response within averagely, 1 second.
Table 4-6: Performance Comparison with [17]
Proposed [17]
Dataset 2 stroke patients 4 healthy subjects 54 stroke patients 16 healthy subjects Channels 4 27 Method FBCSP FBCSP Frequency Bands 3 9 Offline Accuracy
Stroke Motor Imagery 0.74 0.87
Offline Accuracy
Stroke Finger Tapping 0.87 0.9
Offline Accuracy Normal Motor Imagery 0.78 0.74 Online Stroke Accuracy 0.71 0.82 Online Normal Accuracy 0.68 NA
65
Conclusion and Future Work
Conclusion
In summary, this thesis contributes an online real-time BCI system with low computation complexity.
First, we propose an offline BCI system that can detect the subjects’ motion intension in Chapter 3. A series of analysis have be done though the BCILAB toolbox to find the best accuracy with lowest computation complexity. We choose the feature extraction algorithm to be FBCSP, for the finest accuracy, but the drawback of FBCSP is its computation complexity. In order to compensate the drawback, we choose to use less channel number, from 19 channels down to only Fz, C3, Cz, C4 these 4 channels. Moreover, we further reduce the computation complexity by cost down the frequency bands, from default BCILAB’s 5 bands to only 4~7Hz, 8~12Hz, 13~30Hz, these 3 bands. The final results show that we have averagely 84.33% accuracy for stroke subjects, and 80.64% accuracy for normal subjects.
Second, the online implementation method is introduced in Chapter 4. The method was based on the algorithm in Chapter 3, and we choose short timing window to keep updating the prediction result. After the prediction, we provided an algorithm to smooth the prediction, and discretize the prediction result to classification result by setting a threshold Count with de-glitch buffer range. The analysis shows that with shorter timing window, we can have quicker response time, but if the window becomes too short, the accuracy drops greatly. Finally, choose the time window to be 1second, and we can detect the subjects’ motion intension within 1second, with about 67% accuracy.
66
Future Work
For the future work, the proposed BCI system can be further improved by adding complicated preprocessing unit, like ICA, or modifying the feature extraction algorithm to get better accuracy results. Furthermore, we can realized our design by ASIC hardware approach, which will result in faster response time, and faster computation capability for the more complicated algorithm.
67
Reference
[1] Y. Wang, Y.-T. Wang, and T.-P. Jung, "Translation of EEG spatial filters from resting to motor imagery using independent component analysis," PLoS One, vol. 7, p. e37665, 2012.
[2] J. Becedas, "Brain-Machine Interfaces: Basis and Advances," IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, vol. 42, pp. 825-836, 2012.
[3] Y. Wang, S. Gao, and X. Gao, "Common spatial pattern method for channel selection in motor imagery based brain-computer interface," Engineering in Medicine and Biology Society 27th Annual International Conference, pp. 5392-5395, Shanghai, China, 2005.
[4] http://www.bbci.de/competition/iv/#references
[5] K. K. Ang, Z. Y. Chin, C. Wang, C. Guan, and H. Zhang, "Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b," Front Neurosci, vol. 6, p. 39, 2012.
[6] A. Delorme and S. Makeig, "EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis," Journal of neuroscience methods, vol. 134, pp. 9-21, 2004.
[7] C. A. Kothe and S. Makeig, "BCILAB: a platform for brain–computer interface development," Journal of neural engineering, vol. 10, p. 056014, 2013.
[8] A. Hyvärinen, J. Karhunen, and E. Oja, "Independent component analysis," John Wiley & Sons, vol. 46, 2004.
[9] L.-D. Van, D.-Y. Wu, and C.-S. Chen, "Energy-Efficient FastICA Implementation for Biomedical Signal Separation," IEEE Transactions on Neural Networks, vol. 22, pp. 1809-1822, 2011.
68
[10] K. P. Thomas, G. Cuntai, C. T. Lau, A. P. Vinod, and A. Kai Keng, "A New Discriminative Common Spatial Pattern Method for Motor Imagery Brain-Computer Interfaces," IEEE Transactions on Biomedical Engineering, vol. 56, pp. 2730-2733, 2009.
[11] Q. Novi, C. Guan, T. H. Dat, and P. Xue, "Sub-band common spatial pattern (SBCSP) for brain-computer interface," IEEE/EMBS Conference on Neural Engineering, pp. 204-207, Kohala Coast, Hawaii, USA, 2007.
[12] A. Kai Keng, C. Zheng Yang, Z. Haihong, and G. Cuntai, "Robust filter bank common spatial pattern (RFBCSP) in motor-imagery-based brain-computer interface," 31st Annual International Conference of the IEEE EMBS, Minneapolis, Minnesota, USA, September 2-6, 2009.
[13] K. K. Ang, Z. Y. Chin, H. Zhang, and C. Guan, "Filter Bank Common Spatial Pattern (FBCSP) algorithm using online adaptive and semi-supervised learning," in Proceeding of International Joint Conference on Neural Networks (IJCNN), pp. 392-396, 2011.
[14] K. K. Ang, Z. Y. Chin, H. Zhang, and C. Guan, "Filter bank common spatial pattern (FBCSP) in brain-computer interface," in Proceeding of IEEE International Joint Conference on Neural Networks (IJCNN), pp. 2390-2397, 2008.
[15] Y. J. Wang, "Brain-Computer Interfaces Based on Modulation of Brain Rhythms: Progress from Offline Analysis to Online Application," Dissertation of Doctor of Biomedical Engineering, Tsinghua University, China, April 2007. [16] J. L. Chen, private communication.
[17] K. K. Ang, C. Guan, K. S. G. Chua, B. T. Ang, C. W. K. Kuah, C. Wang, K. S. Phua, Z. Y. Chin, H. Zhang, "A large clinical study on the ability of stroke patients to use an EEG-based motor imagery brain-computer interface," Clinical
69
EEG and Neuroscience, vol. 42, pp. 253-258, 2011.
[18] M. Tangermann, K. R. Muller, A. Aertsen, N. Birbaumer, C. Braun, C. Brunner, R. Leeb, C. Mehring, K. J. Miller, G. R. Müller-Putz, G. Nolte, G. Pfurtscheller, H. Preissl, G. Schalk, A. Schlögl, C. Vidaurre, S. Waldert, B. Blankertz, "Review of the BCI Competition IV," Front Neurosci, vol. 6, p. 55, 2012.
