4 Experimental Results
4.2 Main Finding
There are two notable findings in this study. First, the drum loop with faster tempo and lower complexity reduces the LF/HF measure most after drum loop listening. The reduction of the LF/HF after listening to the drum loop is shown in Fig. 4.1.
From Fig. 4.1, it can be observed that the value of LF/HF in 13-15 minutes (resting after drum loop listening) is lower than in 8-10 minutes (baseline resting). This phenomenon is particularly obvious in the L3 loop. According to the Section 2.1.5, the LF measure stands for sympathetic activity, the HF measure stands for parasympathetic activity and the LF/HF ratio is used to observe the balance between the sympathetic and parasympathetic systems.
Fig. 4.1: The C2 comparison of LF/HF measure
Observing the C2 comparison of LF/HF shown in Fig. 4.1, it is inferred that the subjects feel even more relaxing after drum loop listening than in the baseline resting state.
The similar result is also found in the previous study that a randomly inserted short pause during the continued music listening experiment decrease blood pressure, heart rate, and the LF/HF in the subjects. The relaxation effect is even greater than the quiet relaxation at baseline [9]. Being different from the previous study, the drum loop music is adopted rather than the general music and two musical rhythmic features are proposed to be the key component of music-regulating human autonomic nerve system in the thesis. By the systematic method, the experimental results are not only the observed phenomena, but also can be used to construct a model for predicting the physiological responses after music listening. The new finding in this study is that the relaxation aftereffect is stronger in the drum loop with faster tempo and lower complexity.
Second, the entrainment phenomena can be observed in Fig. 4.2. The L3 loop with the lowest complexity synchronizes the heart rhythm and results in the reducing SDNN (HRV) in the resting period rather than the listening condition. Recalling the principle of entrainment mentioned in the Section 3.2, it could be expected that the lowest complex loop (L3) can synchronize the listener’s heart rhythm to a simpler structure than the other loops can do. As shown in Fig. 4.3, it is interesting that the synchronization effect is more significant in the resting period after the drum loop listening than in the listening period. The detailed numerical expression of experimental results is listed in Table 4.1. The C1 and C2 comparisons of all HRV measures obtained in the thesis are shown in Fig. 4.4 to Fig. 4.10.
Fig. 4.2: The C2 comparison of SDNN measure
Fig. 4.3: The C1 comparison of SDNN measure
Table 4.1: The numerical expression of two notable findings in the experimental results
C1 (The comparison between resting and listening
C1 (The comparison between resting and listening drumloop)
C2 (The comparison of resting between baseline and after listening drumloop)
C2 (The comparison of resting between baseline and after listening drumloop)
Fig. 4.4: The C1 and C2 comparisons of LF/HF measure
C2
Fig. 4.5: The C1 and C2 comparisons of SDNN measure
C1
Fig. 4.6: The C1 and C2 comparisons of THB measure
C1
Fig. 4.7: The C1 and C2 comparisons of MRR measure
C1
Fig. 4.8: The C1 and C2 comparisons of RMSSD measure
C1
Fig. 4.9: The C1 and C2 comparisons of LF measure
C2
Fig. 4.10: The C1 and C2 comparisons of HF measure
Chapter 5
Implementation
5.1 Motivation of HRV Chip
Measurement of HRV provides a non-invasive method to obtain reliable information on autonomic modulation of heart rate and has become an important tool for risk assessment to millions of patients who suffer from chronic diseases. A compact, high accuracy, real-time HRV assessment system could provide a valuable feature for implantable and portable cardiac monitoring and intervention devices. The reliable QRS detection is crucial for HRV analysis.
Reviewing the previous System on Chip implementation of QRS detection, there are little information about accuracy and complete standard database testing results for verification [40-41]. Therefore, the implementation in this work focuses on the high accuracy QRS complex detector.
Fig. 5.1: The bit-width of each processing block
0
Fig. 5.2: The deviation of detected R peak between the software QRS detector and the hardware QRS detector
5.2 Accuracy Simulation
For achieving the high accuracy, the bit-width of each processing stage needs to be decided carefully. The bit-width of each processing block is shown in Fig. 5.1 where (x.y) means the bit-width is composed of x bit integer and y bit decimal fraction. The deviation of each processing stage between the software QRS detector and the hardware QRS detector is simulated through all the MIT-BIH Arrhythmia Database. The deviation of detected R peak between the software QRS detector and the hardware QRS detector is shown in Fig. 5.2. It can be seen that the detection results is very close between them. The maximum deviation is 0.00304 samples, it is just 8.45μs (0.00304/360 = 8.444, 360 Hz sampling rate) differences.
So the accuracy of the hardware QRS detector is almost the same as the software QRS detector. The detailed deviation of each record is listed in Table C.1 of Appendix C.
5.3 Hardware Architecture
As mention in section 3.1.2, the QRS detection can be divided into two stages. The
preprocessing stage emphasizes the desired components in order to maximize the signal-to-noise ratio. The peak detection stage decides if an incoming peak is a true QRS complex based on a user-specified threshold. It can be seen that the preprocessing stage of the QRS detection algorithm adopted in this study is composed of several digital filters.
The systolic array architecture for these digital filters is adopted in this work [42]. For computing one-dimensional recursive convolution characterized by the transfer function shown as (5.1) where ai (for i = 0 to N) and bi (for i = 1 to N) are real coefficients, the array structure shown in Fig. 5.3 can be used to achieve an appropriate trade-off between throughput and the amount of hardware required. Because these digital filters mentioned in (3.1) and (3.2) all can be represented as (5.1), they can be implemented by continuously connecting these array structures shown in Fig. 5.4 where the purple blocks represents the registers.
For reducing the amount of hardware required in the chip, observing the arrangement of the registers shown in Fig. 5.3, it can be found that the area closed by the red rectangular can be used as a basic processing element (PE) in the array. The basic PE can be reused continuously to update the different registers and the same result will be obtained. Observing the difference equations listed in (3.1) and (3.2), there are only five possibilities of coefficients. They are 0, 1, -1, 2 and -2. So the four multiplication operation in the PE can be
( )
1 0Fig. 5.3: The systolic array architecture for digital filters
Fig. 5.4 The connecting array architecture of QRS detection preprocessing stage
Fig. 5.5: The proposed PE reusing architecture
simplified to four shifting operation. The proposed PE reusing architecture is shown in Fig.
5.5.
5.3 The Specs
The final implementation result is shown in Table 5.1 and layout is shown in Fig. 5.6.
There are two HRV analysis systems in the previous work. The first one measures RR intervals from ECG signals, then categorizes and stores HRV measures in an internal memory [40]. The second one presents the design of an ECG-processing System-on-Chip (SoC), which incorporates an ARM922T hard macrocell as its processor core. This SoC takes the ECG signals as inputs, and detects the positions of the QRS complexes [41]. The comparison between the previous works and our design is listed in Table. 5.2. According to the comparison of the chip specification, the proposed chip is a cost effective solution needing only 5.1% chip area of the previous work [41] and it can be easily embedded into the biomedical platform solution.
Table 5.1: Summary of the high accuracy QRS detector SoC
Input 13 bit digitized ECG Raw Data
Output 16 bit RR interval
Technology tsmc 0.18 μm
Die Size 1288.6 x 1314.7 μm2 Core Size 812.9 x 835.6 μm2
Gate Count 35630
Max Frequency 50 MHz
Power 25 mW
Accuracy ±6ms
Table 5.2: Comparison of HRV analysis SoC
[40] [41] Proposed
Tech 0.5μm UMC 0.18μm tsmc 0.18μm
Area 3x3 mm2 4095x3202 μm2 812.9x835.6 μm2
Freq. 1 kHz 112.23 MHz 500 Hz
Power 1.5 μW N/A 2.21 μW
Accuracy ± 7 ms N/A ± 6 ms
Database Verification N/A N/A MIT-BIH Arrhythmia Database
Fig. 5.6: The layout of the high accuracy QRS detector chip
Chapter 6
Conclusion
6.1 Discussion
Based on the notable findings mentioned in section 4.2, some inference is constructed and detailed as follows. First, C1 and C2 comparison of the LF/HF measure are discussed and shown in Fig 6.1. The changes of the LF/HF measure during drum loop listening is observed in C1 comparison shown in Fig. 6.1(a). I infer that the main factor contributing to the changes of the LF/HF measure during drum loop listening is the ability of the rhythm pattern to attract the subject’s attention, because the LF/HF measure can be used to reflect the degree of arousal.
When people pay more attention to something or they are aroused by something, their LF/HF measure will show a higher value. In other words, the drum loop which makes the subjects
Fig. 6.1: (a) The C1 comparison of LF/HF measure (b) The C2 comparison of LF/HF measure
feel most surprising will result in the most increasing in the LF/HF measure.
From Fig. 6.1(a), it can be inferred that the L1 makes the subjects feel most surprising or attentive. If the surprising factor of a rhythm pattern can be attributed to two musical rhythmic features proposed in the thesis, there should be some relationship between them. Observing Fig. 6.1(a), it seems that the rhythm pattern with slower tempo will increase the LF/HF measure more. About complexity, the publication by Berlyne (1971) states that an individual’s preference for certain piece of music is related to the amount of activity it produces in the listener’s brain, to which he refers as the arousal potential [39]. According to this theory, which is backed up by a large variety of experimental studies, there is an optimal arousal potential that causes the maximum liking, while a too low as well as a too high arousal potential results in a decrease of liking. He illustrates this behavior by an inverted U-shaped curve (shown in Fig. 6.2) which was originally introduced in the 19th century already by Wundt (1874) to display the interrelation between pleasure and stimulus intensity [39].
Berlyne identifies three different categories of variables affecting arousal. As the most significant he regards the collative variables, containing among others complexity, novelty/familiarity, and surprise effect of the stimulus.
Mapping to the experimental result shown in Fig. 6.1(a), we can also find an inverted U-shaped curve which is shown in Fig. 6.3 if the drum loops are ordered from low to high complexity. It makes sense because the LF/HF measure reflects the arousal potential in some
Fig. 6.2: The Wundt curve for the relation between music complexity and preference
Fig. 6.3: The inverted U-shaped curve for the relation between the surprising factor and rhythmic complexity
degree. So it is concluded that there is a chance to use two rhythmic characteristics, tempo and complexity, to parameterize the subjects’ attention response during drum loop listening.
The responses of the LF/HF after drum loop listening is observed in C2 comparison shown in Fig. 6.1(b). I infer that the main factor contributing to the responses of the LF/HF measure after drum loop listening is the ability of the rhythm pattern to entrain the subjects and consume their energy. When people consume more energy after drum loop listening, they will be calmer in the immediate rest. Observing Fig. 6.1(b), it can be found that the L3 loop with faster tempo and lowest complexity results in most decrease of LF/HF measure after drum loop listening. In other words, the L3 loop is easier to entrain the human heart rhythm and cause most energy consumption. It could be speculated that that’s why some people relax by listening to the electronic dancing music, which is typically featured in faster tempo and
Fig. 6.4: (a) The C1 comparison of SDNN measure (b) The C2 comparison of SDNN measure
lower complexity.
Another interesting phenomenon is the synchronized SDNN measure (heart rhythm complexity) is observed significantly in the resting state after drum loop listening rather than drum loop listening state. It is shown in Fig. 6.4(a) that the SDNN measure is not changed obviously in the drum loop listening state, but it can be found in Fig. 6.4(b) that the L3 loop with the lowest complexity reduces the SDNN measure most and the L4 loop with the most complexity increases the SDNN measure most in the resting state after loop listening.
6.2 Conclusion
There are many literatures discussing the interaction between music and human physiological or psychological responses, but a systematic model is still not constructed completely. This work uses a systematic method to study the complex problem. The problem is scaled down to the simplified and definite topic first. For more accurate experiment control, the simpler auditory stimuli, drum loop pattern which is more suitable to exclude the effect of other music features, is choused as the experimental stimuli. This work represents the first try to use a systematic method to explore the relationship between music perception and its physiological modulation effect.
In this study, the concept of two musical rhythmic features, tempo and complexity, modulating human autonomic nervous system is proposed and the entrainment phenomenon is observed. Two important experiment results explain that the rhythm pattern with faster tempo and lower complexity is easier to entrain human heart rhythm and result in a more relaxing physical state after drum loop listening. Both findings are significant in the resting state after drum loop listening rather than the baseline resting state. In other words, the music aftereffect is even more influential. Although the physiological responses among the subjects sometimes differ largely, the observed results are worthy to study further. The reliability of the results will be assessed in the future.
Besides, the complete software environments for HRV signal processing and musical rhythmic characteristics analysis are constructed. In hardware implementation, a high accuracy and low cost QRS detection chip is realized. This chip represents the first step to construct a single chip solution for a complete HRV analysis.
Fig. 6.5: A systematic model which links music perception and relating physiological responses
6.3 Future Work
Review the initial motivation of this study. Our goal is to construct a systematic model which links music perception and relating physiological responses. The model can be shown in Fig. 6.5. There are many features in music. Each physiological or psychological response (Result1, Result2, etc…) detected by all kinds of biosensors may be resulted from one main music feature or the combination of them (Feature1, Feature2, etc…). The final descriptive emotional or physical state may be identified by integrating these physiological or psychological responses (Result1, Result2, etc…).
Either music perception or physiological modulation is not straightforward. So this work starts from a simplified problem. The musical feature choused is rhythm. The physiological and psychological responses are observed by HRV. For completing the physiology based intelligent music playing system proposed in Fig. 1.2 further, some future works are suggested as follows:
Music Perception Analysis
Two musical rhythmic characteristics, tempo and complexity, are proposed to be two
main features in modulating the autonomic nervous system. The complexity is judged by each subject in this study. Some measures of complexity that corresponds to a high degree with a human’s subjective notion of complexity have been discussed [38-39]. The automatic algorithm for extracting the complexity of simple rhythm pattern (drum loops) should be developed in the future.
Drum loops are widely used in computer music composition and production as a means to generate high-quality music tracks in a quick and easy manner. Most pop music use the drum loop music as the background rhythm base. So the drum loop extraction algorithm is helpful for the automatic music analysis system [43-44]. These algorithms will be integrated to the system for fully automatic musical rhythm analysis in the future.
Bio-signals analysis
Besides the time and frequency domain analysis method, the nonlinear method is also important in biomedical signal processing. Nonlinear phenomena are certainly involved in the genesis of HRV. They are determined by complex interactions of haemodynamic, electrophysiological and humoral variables, as well as by autonomic and central nervous regulations. Therefore, the nonlinear method may be another suitable observation window to explore the physiological modulation induced by music perception [45-46].
For more comprehensive physiological signal analysis, more biomedical signal (pulse, photoplethysmograph abbreviated as PPG, etc…) will be captured for cross analysis in the future. On the other hand, a non-contact optical measurement system for acquiring the HRV signal is under developing [47-48]. The HRV signal will be easier to be captured and the physiology based intelligent music playing system is more portable in the future.
Emotion recognized by physiological responses
As describing in the section 1.2, the physiology based intelligent music playing system chooses suitable music for user to make them achieve desired physiological or emotional state (ex. powerful, active, calm, etc…). I think emotion recognition by physiological responses is important and interesting [49-50]. If the music induced emotion can be measured and recognized correctly by the physiological signals, the emotional responses for various music
can be recorded continuously when user is listening. It is helpful for the system to understand the individual preference or emotion response for the specific music type by long term machine learning. The experience of music listening will be improved through the interactive system. So the emotion recognition should be integrated into the system in the future.
Bibliography
[1] Hon EH; Lee ST, “Electronic Evaluations of the Fetal Heart Rate Patterns Preceding Fetal Death: Further Observations,” Am J Obstet Gynecol, vol. 87, no. 6, pp. 814-826, 1965.
[2] Akselrod S, Gordon D, Ubel FA, Shannon DC, Barger CA, Cohen RJ, “Power Spectrum Analysis of Heart Rate Fluctuation: A Quantitative Probe of Beat-to-Beat Cardiovascular Control,” Science, vol. 213, no. 4504, pp. 220-222, 1981.
[3] Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology, “Heart Rate Variability: Standards of Measurement, Physiological Interpretation, and Clinical Use,” Circulation, vol. 93, no. 5, pp. 1043-1065, 1996.
[4] P. T. BASON; B. G. CELLER, “Control of the Heart Rate by External Stimuli,” Nature, vol. 238, pp. 279-280, Aug. 1972.
[5] Levy; Matthew N., “Sympathetic-Parasympathetic Interactions in the Heart,” Circ. Res., vol. 29, no. 437, 1971.
[6] Leon Glass, “Synchronization and Rhythmic Processes in Physiology,” Nature, vol. 410, pp. 277-284, March 2001.
[7] M Clayton; R Sager; U Will, “In Time with the Music: The Concept of Entrainment and its Significance for Ethnomusicology,” ESEM CounterPoint, vol. 1, 2004.
[8] Cook Perry, Music, Cognition, and Computerized Sound, Cambridge: The MIT Press, 2001.
[9] L Bernardi; C Porta1; P Sleight, “Cardiovascular, Cerebrovascular, and Respiratory Changes Induced by Different Types of Music in Musicians and Non-Musicians: the Importance of Silence,” Heart, vol. 92, pp. 445-452, 2006.
[10] Johnson JE, “The Use of Music to Promote Sleep in Older Women,” Journal of Community Health Nursing, vol. 20, no. 1, pp. 27-35, 2003.
[11] Cooke, Marie; Chaboyer, Wendy; Schluter, Philip; Hiratos, Maryanne, “The Effect of Music on Preoperative Anxiety in Day Surgery,” Journal of Advanced Nursing, vol. 52, no. 1, pp. 47-55(9), October 2005.
[12] Gertjan Wijnalda; Steffen Pauws; Fabio Vignoli; Heiner Stuckenschmidt, “A Personalized Music System for Motivation in Sport Performance,” IEEE Pervasive Computing, vol. 4, no. 3, pp. 26-32, July-September 2005.
[13] Christopher F. Chabris, “Prelude or Requiem for the 'Mozart Effect'?,” Nature, vol. 400, pp. 826-827, August 1999.
[14] N., Matthew. Vagal Control of the Heart: Experimental Basis and Clinical Implications.
City: Futura Publishing Company, 1994.
[15] G. Berntson; J. T. Bigger; Jr.; D. Eckberg; P. Grossman; P. Kaufmann; M.Malik; H.
Nagaraja; S. Porges; J. Saul; P. Stone; W. V. D.Molen, “Heart Rate Variability: Origins, Methods and Interpretive Caveats,” Psychophysiology, vol. 34, pp. 623–648, 1997.
[16] I A O'Brien; P O'Hare; R J Corrall, “Heart Rate Variability in Healthy Subjects: Effect of age and the derivation of normal ranges for tests of autonomic function,” British Heart
[16] I A O'Brien; P O'Hare; R J Corrall, “Heart Rate Variability in Healthy Subjects: Effect of age and the derivation of normal ranges for tests of autonomic function,” British Heart