Chapter 3 Assessment of Motion Sickness Symptoms
3.3 Assessment of Motion Sickness Symptoms with Subjective Evaluation
The MSQ has served as a subjective index of motion sickness in many previous researches. However, it is an arbitrary assessment method which can be too subjective for scientific studies. Both subjective and objective evaluations are used in our study for the systematic motion sickness estimation. Table 3-4 is the performance comparison of different motion sickness indices we investigate in this study.
Table 3-4: Variation performance comparison of different motion sickness indices.
Dominant Frequency Subject
MSQ
Score EKG EGG
GSR
Subject 1 17/50 1.05 (Hz) → 1.3 (Hz) 0.09 (Hz) → 0.15 (Hz) X Subject 3 28/50 1.1 (Hz) → 1.2 (Hz) 0.09 (Hz) → 0.15 (Hz) X Subject 5 11/50 1.3 (Hz) → 1.5 (Hz) 0.09 (Hz) → 0.15 (Hz) X Subject 6 08/50 0.9 (Hz) → 0.9 (Hz) 0.09 (Hz) → 0.12 (Hz) X Subject 7 17/50 1.2 (Hz) → 1.3 (Hz) 0.09 (Hz) → 0.12 (Hz) O Subject 8 05/50 1.4 (Hz) → 1.4 (Hz) 0.09 (Hz) → 0.09 (Hz) X Subject 9 34/50 1.1 (Hz) → 1.25 (Hz) 0.06 (Hz) → 0.12 (Hz) X Subject 10 12/50 1.2 (Hz) → 1.4 (Hz) 0.09 (Hz) → 0.15 (Hz) O
According to Table 3-4, for subject 1, 17 points of MSQ score refers to mild sickness in the subjective assessment. The EKG dominant frequency shifted from 1.05 Hz to 1.3 Hz, and the EGG dominant frequency shifted from 0.09 Hz to 0.15 Hz in “Motion-Sickness” session.
Although there is no able distinguish change in the GSR response, subject 1 was selected for the further EEG signals analysis.
Fig. 3-7. The comparison results of both subjective evaluation and physiological responses.
The MSQ scores of subject 3 and subject 9 are 28 and 34, respectively, which indicate severe sickness in the subjective assessment. By contrast, the MSQ score of subject 6 and subject 8 are only 8 and 5, respectively. The MSQ score lower than 10 ponits in our experiment is considered as indistinct-sickness in the subjective assessment. For objective
indices, EKG, EGG and GSR, if more than two of the three indices have significant changes, the subject is considered as a subject with motion sickness in the experiment. The normalized scores of subjective and objective indices of different subjects are shown in Fig. 3-7. The EKG and EGG scores show the normalized signal differences between the “Baseline session”
and the “Motion-Sickness session” for different subjects.
The three curves in Fig. 3-7 are highly related to each other. It shows strong evidence to the validity of the indices such that we can study the EEG changes related to motion sickness base on these results. More details are discussed in the next chapter.
Chapter 4
Analysis Results of EEG Response during Motion Sickness
The EEG data analysis is based on the cross-demonstrations of subjective evaluation and physiological responses to ensure the objectivity of sickness assessment. The EEG signals analysis procedure is given in section 4.1, and the relationship between the power spectrum of ICA components and motion sickness is given in section 4.2. Finally, the results of spectrum ICA component dynamic changes are given in section 4.3.
4.1 EEG Signal Analysis Procedure
Flowchart of the EEG signal processing procedure is show in Fig. 4-1. The 32-ch EEG data was collected through an electrode cap and amplified by the NuAmps. The sampling rate of the EEG data is 500Hz. It consists of artifacts removal, independent component analysis, useless component rejection, spectral analysis, nausea-related components selection and dynamic spectral analysis.
A 500-pt high pass filter with a cut-off frequency at 1 Hz is used to remove breathing artifacts. The width of the transition band of the high pass filter is 0.2 Hz. A 30-pt low pass filter is then applied to the signal with the cut-off frequency at 50 Hz to remove muscle artifacts and line noise. The transition band width of the low pass filter is 7 Hz. The independent component analysis (ICA) is applied to the filtered EEG signals to obtain the independent components. Some artifacts can be rejected in the process of useless components rejection. The effectiveness of eye blinking and other artifacts removal by using ICA had been
demonstrated in the Jung et al.’s study [17]. The spectral analysis is then applied to the useful independent components to calculate their frequency response. The detailed introduction of the algorithm is given in subsection 4.1.1. The feature of the motion sickness symptoms can be evaluated by the selection of nausea-related components. And finally, the continuous frequency responses of the ICA components are evaluated with dynamic spectral analysis.
Fig. 4-1. Flowchart of the EEG signal analysis procedure.
4.1.1 Independent Component Analysis
The joint problems of EEG source segregation, identification, and localization are very difficult since the EEG data collected from any point on the human scalp includes activity generated within a large brain area, and thus, problem of determining brain electrical sources from potential patterns recorded on the scalp surface is mathematically underdetermined [19].
Although the resistivities between the skull and brain are different, the spatial smearing of EEG data by volume conduction does not involve significant time delay and suggests that the ICA algorithm is suitable for performing blind source separation on EEG data by source identification from that of source localization. We attempt to completely separate the twin problems of source identification and source localization by using a generally applicable ICA.
Thus, the artifacts including the eye-movement (EOG), eye-blinking, heart-beating (EKG), muscle-movement (EMG), and line noises can be successfully separated from EEG activities.
The ICA is a statistical “latent variables” model with generative form:
x(t)=As(t) , (1) where A is a linear transform called a mixing matrix and the S are statistically mutually i independent. The ICA model describes how the observed data are generated by a process of mixing the components S . The independent components i S (often abbreviated as ICs) are i latent variables, meaning that they cannot be directly observed. Also the mixing matrix A are assumed to be unknown. All we observed are the random variablesx , and we must estimate i both the mixing matrix and the ICs S using thei x . i
Therefore, given time series of the observed data x(t)=[x1(t) x2(t) ⋅ ⋅⋅ xN(t)]T in N-dimension, the ICA is to find a linear mapping W such that the unmixed signals u(t) are statically independent.
u(t)=Wx(t), (2) Supposed the probability density function of the observations x can be expressed as:
p(x)= det(W )p(u) (3) the learning algorithm can be derived using the maximum likelihood formulation with the log-likelihood function derived as:
Thus, an effective learning algorithm using natural gradient to maximize the log-likelihood with respect to W gives:
and WTW in Eq. (5) rescales the gradient, simplifies the learning rule and speeds the convergence considerably. It is difficult to know a priori the parametric density functionp(u), which plays an essential role in the learning process. If we choose to approximate the estimated probability density function with an Edgeworth expansion or Gram-Charlier expansion for generalizing the learning rule to sources with either sub- or super-gaussian distributions, the nonlinearity ϕ(u) can be derived as:
⎩⎨
(
(1,1) (-1,1))
before the tanh function and can be determined using a switching criterion as:[ ]
as the elements of N-dimensional diagonal matrix K. After ICA training, we can obtain 32 ICA components u(t) decomposed from the measured 32-channel EEG data x(t).
(11)
Fig. 4-2 shows the scalp topographies of ICA weighting matrix W corresponding to each ICA component by spreading each wi,j into the plane of the scalp, which provides spatial information about the contribution of each ICA component (brain source) to the EEG channels, e.g., eye activity was projected mainly to frontal sites, and the drowsiness-related potential is on the parietal lobe to occipital lobe, etc. We can observe that the most artifacts and channel noises included in EEG recordings are effectively separated into ICA components 1 and 3 in Fig. 4-2.
Fig. 4-2. The scalp topographies of ICA weighting matrix W.
4.1.2 Spectral Analysis
A spectral analysis is applied to the ICA components for the selection of nausea-relation components. Three of 2-min epochs of ICA components are cut according to Fig. 4-3, including Baseline, Motion-Sickness and Rest. The Power Spectrum Density (PSD) of the three epochs of each ICA components is calculated for the further analysis. The results of the spectral analysis are given in subsection 4.2.
Electrode-pop
Eye-blinking Eye-movemen
Fig. 4-3. The three epochs for the spectral analysis.
4.1.3 Dynamic Spectral Analysis
Analysis of changes in spectral power and phase can characterize the perturbations in the oscillatory dynamics of ongoing EEG. Applying such measures to the activity time courses of separated independent component sources avoids the confounds caused by miscancellation of positive and negative potentials from different sources to the recording electrodes, and by misallocation to the recording electrodes activity that originates in various and commonly distant cortical sources.
The dynamic spectral analysis for each ICA component decomposed from 32 channels of the EEG signals is shown in Fig 4-4. The moving-averaged spectral analysis [18] for the ICA data was first accomplished using a 750-point rectangular window with 250-point overlap, i.e., stepping in 1 second at sampling rate 500 Hz. Windowed 750-point epochs were sub-divided
into several 125-point frames using Hamming windows with 25-point step size, each extended to 256 points by zero padding for a 256-point FFT.
Fig. 4-4. Moving-averaged log power spectral analysis for time courses of ICA components.
ICA data spectra were further converted to a logarithmic scale. Then we averaged the bandpower corresponding to each frequency band in all the sub-windows to form a log subband power spectra. The ICA power spectrum time series consisted of 32-channel ICA bandpower estimated at special frequency (such as 10 Hz or 20 Hz) stepping at 1s (500-point, an epoch) time intervals. Finally, a median filtering using a moving averaged 90-s window was used to further minimize the influence of variances.
4.2 Relationship between the Power Spectrum of ICA components and Motion Sickness
Figure 4-5 shows the EEG Power Spectrum Density (PSD) analysis result of subject 10 using the method proposed in subsection 4.1.2. The blue, red, and green line in Fig. 4-5 represents the power spectrums in the “Baseline”, “Motion-Sickness”, and “Rest” sessions, respectively.
Fig. 4-5. Spectral analysis results of subject 10.
According to Fig. 4-5, distinguish able changes magnitude at 12Hz can be obseved in the central parietal lobe area. The suppression of ICA power at 12 Hz during the
“Motion-Sickness” session can be a good demonstration for the nausea-related feature of EEG signals. Power spectrum of ICA components of subjects 3 and 7 also give the same phenomenon.
The spectral analysis result of subject 5 shown in Fig. 4-6. It is different from the result of subject 10. The nausea-related regions of subject 5 are in the right parietal and left parietal lobe. The magnitude of power spectrum at near 13 Hz was suppressed from “Baseline”
session to “Motion-Sickness” session. The 13 Hz peak raise again after a 10-mim rest. Power spectrum of ICA components of subjects 7, 9 and 10 also showed the same phenomenon.
Component in the right parietal lobe Component in the left parietal lobe
(a) (b)
Fig. 4-6. Spectral analysis results of subject 5.
According to our experimental results, power suppression in some specific frequency bands of ICA components are proved to be a common phenomenon when most of subjects in nausea and the suppressions will release when the subjects recovered from nausea after rest.
In addition, EEG sources related to motion sickness are located at the central parietal lobe, right parietal lobe and left parietal lobe.
4.3 Spectral Dynamic Changes of ICA Components
According to the results discussed in subsection 4.2, the ICA power spectrum suppressed at some specific frequency band when the subject feels nausea. The dynamic changes of the ICA spectrum along the time course is discussed in this subsection. The relationship between the spectral ICA components and the transition of the road-type (straight / curve) will also be investigated.
Fig. 4-7. Subject 10’s spectral dynamic variations of ICA component 13 (left parietal lobe) at 23 Hz.
The continuous changes of ICA component spectrum at 23 Hz in the region of the left central parietal lobe of subject 10 is shown in Fig.4-7. The blue, red, and green line in Fig. 4-7
represents the power spectrums in the “Baseline”, “Motion-Sickness”, and “Rest” sessions, respectively. The x-axis here represents the time steps, and the y-axis is the magnitude of ICA spectrum in dB. A significant drop of spectral magnitude of 23 Hz can be found at the transition point between the “Baseline” session and the “Motion-Sickness” session. The spectral magnitude rises again after the subject drived in the “Rest” session. The phenomenons of other subject is given in Fig. 4-8.
Fig. 4-8. Subject 1’s spectral dynamic variations of ICA component 17 (right parietal lobe) at 18 Hz.
The experimental results shown in this subsection demonstrates that the correlation between the feeling of motion sickness and the spectral suppression of ICA components is high. In addition, the EEG signal variations induced by motion sickness can be monitored by using analyzing the spectral dynamic changes of ICA components change immediately with the variations of road-type.
Chapter 5 Discussions
Some specific phenomena of motion sickness in the physiological signals are discovered in chapter 3 and 4. We are going to further emphasize the discussion of cross-subject specific phenomena in this chapter. The discussion of motion sickness influence region on human cortex is given in section 5.1, and the discussion of power spectrum suppression induced by motion sickness is given in section 5.2.
5.1 Motion Sickness Influence Region on Human Cortex
The influence regions of motion sickness on human cortex are discussed in this section.
Three different phenomena are found in this study including the 20Hz suppression in left and right parietal region, the 10 Hz suppression in left and right parietal region and of 10/20 Hz power suppression in the central parietal region.
5.1.1 The 20 Hz power suppression in left and right parietal region
The spatial distributions on scalp topographies of weighting matrices for dominant ICA components of subject 1 are shown in Fig. 5-1. It indicates the 20 Hz power suppression components occurred with symmetry. In the other word, the 20-Hz suppression phenomena occur in both left and right parietal region at the same time. The curve of 20 Hz power activity suppression is given in Fig. 5-2.
The peak near 20 Hz in “Baseline” session was suppressed in “Motion-Sickness” session obviously. The phenomenon is obviously, and is considered as an important discovery in our study.
(a) Component in the left parietal lobe. (b) Component in the right parietal lobe.
Fig. 5-1. The 20 Hz power activity suppression of subject 1 in two different ICA components.
Fig. 5-2. The 20 Hz power suppressions in the side parietal lobe.
5.1.2 The 10 Hz power suppression in left and right parietal region
The spatial distributions on scalp topographies of weighting matrices for dominant ICA components of subject 5 are shown in Fig. 5-3. It indicates the 10 Hz power suppression components occurred with symmetry. The 10 Hz power suppression is the most common indication for the motion sickness symptoms, since 4 of the 6 subjects have the similar phenomenon in this area.
(a) Component in the left parietal lobe. (b) Component in the right parietal lobe.
Fig. 5-3. The 10 Hz power activity suppression of subject 5 in two different ICA components.
The 10-Hz suppression phenomena occur in both left and right parietal region at the same time. The curve of 10 Hz power activity suppression corresponding to Fig. 5-3 is shown in Fig. 5-4.
Fig. 5-4. The 10 Hz power suppression in the side parietal lobe.
5.1.3 The 10/20 Hz power suppression in the central parietal region
The central parietal region is also considered as an important nausea-related area according to our results. The spatial distributions on scalp topographies of weighting matrices for dominant ICA components of subjects 3, 7, and 10 are shown in Fig. 5-5. All the components are at the central parietal region and they have both 10 Hz and 20 Hz power activity suppressions. The curves of 10/20 Hz power activity suppression corresponding to Fig. 5-5 is shown in Fig. 5-6.
(a) Subject-3. (b) Subject-7. (c) Subject-10.
Fig. 5-5. The nausea-related area in the central parietal region.
Fig. 5-6. The 10 Hz and 20 Hz power suppression in the central parietal lobe.
5.1.4 The nausea-related regions on human cortex
Table 5-1 is a comparison of the nausea-related regions in different subjects. All the subjects have the power-suppressions near 10 Hz or 20 Hz in the parietal lobe, as shown in Fig. 5-7.
Table 5-1: Comparison of nausea-related regions.
Component
In general, the parietal lobe plays important roles in integrating sensory information from various senses and in the manipulation of objects. This area of the cortex is responsible for somatosensation. This cortical region receives inputs from the somatosensory relays of the thalamus.
Fig. 5-7. The region of parietal lobe.
5.2 Power Spectrum Suppression Induced by Motion Sickness
We have concluded that all the subjects have the power-suppressions phenomena near 10 Hz or 20 Hz in the parietal lobe in section 5.1. The frequency band of power-suppression differs from subjects, but inside the range between 8 and 22 Hz. The range of the power-suppression frequency band is sometimes wide for subject 1 and 5, which is about 10 to 20 Hz. And the range for subject 3, 7, and 10 may be narrow with the range about 18 to 19 Hz and 10 to 13 Hz in the central parietal lobe. All the subjects have the power-suppressions in the range from 10 to 13 Hz and more than 80 % of the subjects have the power-suppressions in the range from 18 to 19 Hz.
5.3 Reliability of Different Physiological Responses
The reliabilities of different physiological responses are going to be discussed in this section. The three different objective indices for the assessment of nausea-symptoms are EKG, EGG and GSR. We are going to define some useful assessing parameters for the reliability of each signal. The EKG variation ratio which is corresponding to the EKG signal is defined as:
%
where fMotion−Sickness is the EKG dominant frequency in “Motion-Sickness” session, fBaseLine is the EKG dominant frequency in “Baseline” session. The EGG and GSR variation ratios can also be defined as the same way.
%
Table 5-2 is the comparison of the motion sickness indices we used in this research including the subjective MSQ score and the objective indices.
There is no significant difference in the GSR signals between “Baseline” and
“Motion-Sickness” sessions, except subject 7 and subject 10. A small questionnaire was proposed to the subject before every experiment, which included a question: “Do you feel sweating during motion sickness?” The answer of both subject 7 and subject 10 is “Yes”, while the others answer no. From this point of view, although the GSR signal is useless for most of subjects, but it is a good physiological index for the subjects who feel sweating during motion sickness with significant changes and fast response time.
Table 5-2: Comparisons of indices.
Subject MSQ score
Subject 1 24 % 67 % 4 % * 17/50
Subject 3 9 % 67 % 3 % * 28/50
Subject 5 15 % 67 % 5 % * 11/50
Subject 6 0 % * 33 % 6 % * 8/50
Subject 7 8 % 33 % 175 % 17/50
Subject 8 0 % * 0 % * 1 % * 5/50
Subject 9 14 % 100% 8 % * 34/50
Subject 10 17 % 67 % 54 % 12/50
* No significant difference
The influence factors to the EKG signal is varied. And also, the variation rates of EKG are not as significant as GSR or EGG. It was found from the result that the EGG signal is the most efficient physiological signal, which is suitable for most of subjects and provides excellent response time.
REKG REGG RGSR
Chapter 6
Conclusions and Future Work
The nausea-related EEG dynamics corresponding to motion sickness inclining tasks is studied in this thesis with the virtual-reality based dynamic driving environment. The VR-based dynamic driving environment provides the advantages of safety, low cost, and the realistic stimuli to the subjects. The MSQ is designed and the physiological responses (including EKG, EGG and galvanic skin response) are recorded to assess the motion sickness.
The EEG data analysis is based on the cross-demonstrations of subjective evaluation and physiological responses to ensure the objectivity of sickness assessment. In other words, the EEG changes correlated to motion sickness will not only refer to the MSQ score, but the objective indices should also be involved for systematic evaluation. It was found from the result that the EGG signal is an efficient index, which is suitable for most of the subjects with excellent response time.
Using ICA and PSD analysis technology, the power suppression in some specific frequency bands (such as 10 Hz or 20 Hz) of ICA components are proved to be a common phenomenon when most of subjects during motion sickness and the suppressions will release when the subjects recovered from motion sickness after rest. All of subjects indicate that the
Using ICA and PSD analysis technology, the power suppression in some specific frequency bands (such as 10 Hz or 20 Hz) of ICA components are proved to be a common phenomenon when most of subjects during motion sickness and the suppressions will release when the subjects recovered from motion sickness after rest. All of subjects indicate that the