doi:10.6342/NTU201801179
國立臺灣大學工學院土木工程學系 碩士論文
National Taiwan University Master Thesis
環境教育應用：數位繪本的配樂如何引發情感連結 Emotional Induction Effects of Soundtracks in the
Educational Multimedia: A Case Study from Environmental Education
張沄真 Yun-Chen Chang
指導教授：康仕仲 教授 共同指導：曾敬梅 助理教授 Advisor: Prof. Shih-Chung Kang Co-advisor: Prof. Ching-Mei Tseng
中華民國 107 年 2 月
Feb. 2018
doi:10.6342/NTU201801179
doi:10.6342/NTU201801179
中文摘要
透過情緒的觸發能增進個體對環境議題的認知與行為改變；因此，如何提升人 與自然之間的情感連結是環境教育的重要課題。多媒體教材，例如：影片、遊戲與數 位故事…等被認為是能夠有效引發情感連結的教學媒介，近年來也被應用在環境教育 的領域。研究顯示配樂很可能是多媒體教材中引發情感反應的強力因素之一，然而，
這些配樂普遍未經專業譜曲或設計，因此也未能有效與影像結合並增強觀看者的情緒 反應，達致教育目標。本研究延續同一研究團隊的系統性與縱貫性研究，以水環境教 育教學影片<<水寶>>為素材，並依據歸納出的作曲理論為其專門設計配樂，並研究配 樂如何配合教學媒材引發情緒反應。本研究的目標是比較「設計配樂」與「現成配樂」
所引發之情緒反應是否符合預期與假設。36 位大學生經過第一階的篩選，根據其對音 樂的感受與環境的感受被分配成背景均等的兩組，其中一組觀看配有設計配樂的<<水 寶>>，另一組則觀看使用現成音樂作為配樂的<<水寶>>。觀看過程中，受測者必須以 研究者所特別設計的紀錄工具同步紀錄其情緒反應（正向、負向，或其他），情緒資 料則使用視覺化工具進行紀錄、轉化與分析。看完影片後，受測者被邀請立即進行關 於影片觀看過程情緒感受的追蹤訪談。研究結果顯示配有設計配樂的<<水寶>>能引發 觀看者較精準(亦即符合設計)的情緒反應，現成音樂則容易使觀看者對影片產生與設 計意圖不符之解讀。從訪談中，受測者指出現今教育媒材的配樂確實需要改進，並提 出影片長度、旁白字詞與情緒的多寡、影片知識的難度、劇情的可預測性，以及影片 的教學場合皆是影響配樂設計準則的因素。
關鍵字：配樂、教育多媒體、情緒反應、數位故事、環境教育
doi:10.6342/NTU201801179
Abstract
Inducting learners’ emotional connection with nature is crucial in environmental education. Educational multimedia, like videos and games, have been recognized and gradually applied to influence students’ emotions, awareness, attitude and behavioral changes.
Soundtracks, an effective emotion stimulus, however, has rarely been professionally dealt with or integrated to enhance educational multimedia. The present study fills a void by linking the emotional induce effects with composed soundtracks and continues to explore the affective aspects of educational multimedia. In the present study, soundtracks were composed by the researcher to integrate with WaterBob, a digital storybook created by the same research team.
The purpose of the present research is to investigate effects of soundtracks (composed or compiled canned soundtracks) and emotional induction on environmentally responsible behavioral intentions. The sample consisted of 36 college students. Participants were randomly assigned into two groups. While watching the videos, one group listened to composed soundtracks, whereas another group listened to compiled soundtracks downloaded from online musical service platform. Participants were simultaneously asked to provide emotional reactions (positive, negative, or mixture). A visualization tool was developed to document and further analyze the data. The result indicated a significant difference in emotional induction between the two groups. The follow-up interview showed a significant relation between emotional induce and environmentally responsible behavioral intentions. In addition, the duration, the complexity of the content, the narration, the predictability of the plot, and the educational occasion are considered important factors for soundtrack design, and were further discussed.
Keywords: soundtrack, educational multimedia, emotional induce, digital story, environmental education
doi:10.6342/NTU201801179
TABLE OF CONTENTS
中文摘要………....i
ABSTRACT………..…ii
TABLE OF CONTENTS………...iii
LIST OF FIGURES………...v
LIST OF TABLES………....vi
Chapter 1. Introduction………..1
1.1 Background………..1
1.2 Research Goal………..3
1.3 Research Questions………..3
1.4 Definitions of Terms………4
Chapter 2. Literature Review……….5
2.1 Educational multimedia in environmental education………..5
2.2 The Functions of Soundtracks in the Context of Multimedia………..9
2.3 The Difference between Composed and Compiled Soundtracks………11
2.4 Summary……….13
Chapter 3. Soundtrack Design………..14
3.1 Soundtrack Elements Inducing Emotion……….14
3.2 The Audio-Visual Counterpoint………..15
3.3 Leitmotiv……….17
3.4 The Analysis of Mode……….17
3.5 Practices of Soundtrack Design………...20
3.6 Soundtrack Design of WaterBob………..21
doi:10.6342/NTU201801179
Chapter 4. Methodology………28
4.1 The Educational Multimedia – WaterBob………...28
4.2 Participants………...30
4.3 Data Collection……….32
4.4 Experiment Design and Data Analysis……….34
4.5 Procedure………..34
Chapter 5. Results and Discussion……….37
5.1 The Heat Map of Emotional Induction………37
5.2 Soundtrack Design Issues and Principles………44
Chapter 6. Conclusion………...47
Reference………...49
Appendix 1. Full Content of WaterBob………52
Appendix 2. Sheet of WaterBob………79
doi:10.6342/NTU201801179
LIST OF FIGURES
Figure 1 The audio-visual counterpoint…….……….16
Figure 2 The audio-visual counterpoint of WaterBob…………..………...22
Figure 3 WaterBob and the leitmotiv……….………...23
Figure 4 The Mouse and the leitmotiv……….………...24
Figure 5 The Cat and the leitmotiv……….25
Figure 6 The Rabbit and the leitmotiv………26
Figure 7 The emotion categories of WaterBob………...33
Figure 8 The documenting tool………...33
Figure 9 The heat map of emotional induction………...38
Figure 10 The emotional flow………39
Figure 11 The unexpected emotion responses………....40
doi:10.6342/NTU201801179
LIST OF TABLES
Table 1 The designed rubric of soundtrack (adapted from the research of digital storytelling
by Barrett (2006)) ………7
Table 2 The function of composed and compiled soundtracks………..11
Table 3 Sections of WaterBob………....21
Table 4 The design of leitmotiv of WaterBob………....23
Table 5 The design of leitmotiv of the Mouse………....24
Table 6 The design of leitmotiv of the Cat………...25
Table 7 The design of leitmotiv of the Rabbit………....26
Table 8 The design of leitmotiv of the main theme………....27
Table 9 The emotion expression of WaterBob………...29
Table 10 Grouping Result………...31
Table 11 The count and percentage of unexpected emotion (t-test)………...42
Table 12 Elements of soundtrack design………...46
doi:10.6342/NTU201801179
doi:10.6342/NTU201801179
doi:10.6342/NTU201801179
Chapter 1. Introduction
Environmental educational have been facing difficulties. For a closer relationship between human and nature, effective education strategies thus have been studied to induce emotion. Among all the strategies, our research team focused on digital education media. This research focused on how soundtracks, the audio aspect of the media, induce viewers’ emotion and enhance the transfer of education. Composing theories were applied to a digital storybook, WaterBob, to examine the effect of the soundtracks was examined and to study important factors for soundtrack design.
1.1 Background
The rise of human awareness as well as the acquisition of knowledge of the concept of environmental education, e.g., water conservation, are considered a civic responsibility (Slattery & Rapp, 2003). The government and industry have sought to promote water resource conservation activities, and students and the general populace are expected to learn about water conservation. However, environmental education nowadays is facing difficulties to some extent; i.e. the fragile relationship between nature and industrialized society makes it hard for educators to raise public environmental awareness. One’s environmental awareness and attitude have a personal and social component, and exist within all individuals who care about the environment that leads to pro-environmental behaviors. Among various of studies strive to solve the above problem, more and more recent studies have been investigating factors that help establish a closer relationship between human and nature. Their research has shown that affective factors such as emotion, meaning, and affection are critical to nature connectedness (Berenguer, 2007; Lumber et al., 2017). Specifically, emotional connection with nature is considered beneficial to pro-environmental behaviors. One of the effective and meaningful method is to educate people via both formal and informal learning activities with a focus on
doi:10.6342/NTU201801179 emotional connection. As a result, teaching methods and materials stressing on emotion
induction have not only been received more attention but also developed and applied to environmental education practice.
There is still a significant void in the literature regarding emotional responses to information, awareness and intentional behaviors about the environmental issues. Although previous research has considered the importance of affection aspects in environmental education, the role of emotional induce has been overlooked (Perrin, 2011). It seems appropriate to begin to explore the influence of emotional induce of individuals’ behavioral intentions and curiosity regarding environmental information, particularly via educational multimedia.
In the past decades, affection education strategies thus have been developed to raise individual’s awareness in the field of environmental education, including field trips (Farmer et al., 2007), digital game (Cheng et al., 2013), and videos (Hill and Nelson, 2011). In addition to knowledge-, lecture-based learning, the above alternative and informal ways of learning have shown positive effects on the connection between the human-nature relationship. Among these strategies, our research team has focused on the development and validation of a series of educational multimedia, including computer games (Kuo, 2013), virtual reality (VR) games (Hsu, 2017), and digital storybook (Yang, 2017). These educational multimedia was developed in hope of increasing learners’ awareness and the behavior intention of water conservation. Empirical study studies were conducted to examine the effects of each experimental intervention. Most consistently, the results showed significant changes in cognition and behavior intention.
doi:10.6342/NTU201801179 1.2 Research Goal
The present study strives to link the emotional induce effects with composed soundtracks and continuing to explore the affective aspects of educational multimedia, specifically, to exam the emotional impact of soundtracks in educational multimedia. The researcher predicted that composed soundtracks would trigger significant differences in expected emotion (positive or negative) that exhibits emotional induction, compared to compiled soundtrack design.
Applying composing theories, the soundtracks were designed and added to a digital storybook, WaterBob (Yang, 2017). Yang’s study focused on the development and validation of effects of digital stories in the environmental education. The results from her research indicate that digital stories may trigger signifcant emotional responses. Through the practice of soundtrack design, we expected to achieve the following objectives: (1) to exam whether composing theories could be meaningfully applied in educational soundtracks to induce anticipated emotional responses, (2) to develop more specific soundtrack design rubrics as a reference for future application in the educational field.
1.3 Research Questions
Based on the objectives of our soundtrack design, the following research questions are set:
Q1: Could composed soundtrack of educational multimedia trigger expected emotional reaction?
Q2: Follow upon the first question, what are the essential elements for soundtrack design?
doi:10.6342/NTU201801179 1.4 Definitions of Terms
Educational Multimedia
Research in different fields uses different words to refer to digital teaching material.
While educational experts (Robin, 2005; Barret, 2006) use “digital storytelling” to stress the importance of story, musicologists (Bullerjahn, C. and Güldenring, M., 1994; Cohen, 2001;
Boltz, 2004) use various words including multimedia, film, video, clip, and audiobook according to the characteristic of the experiment design and materials. In this paper, the researcher uses the term “educational multimedia” since it could represent most of the words used in musicology, and could as well stress the educational context.
Soundtrack and Music
Both soundtrack and music is used to indicate audio materials in this paper. Whereas
“soundtrack” refers to practical use and the entity of the audio material, “music” refers to theoretical implication. In addition, the word “music” are used in this paper while the connection between audio and visual is not the primary concern.
Composed and Compiled
In the following chapter 2 types of soundtracks would be discussed: the composed and the compiled. The former refers to soundtracks explicitly composed for specific multimedia, while the latter refers to music with other design purpose but is used as a second-hand or canned soundtrack in multimedia.
doi:10.6342/NTU201801179
Chapter 2. Literature Review
In this chapter, the researcher first introduces various of educational multimedia in environmental education, with a focus on emotional induction. After pinpointing the issues in soundtrack design of educational multimedia, functions of soundtracks in the context of educational multimedia were discussed, such as how does soundtrack as revenue to induce emotion that better interpret digital stories. Following a discussion of needs and problems of composed soundtracks, instead of compiled ones.
2.1 Educational multimedia in environmental education
In recent years, schools and the academics have been emphasizing on multimedia technology. Films, videos, and short clips are used as teaching materials in and out of the classroom for environmental education. These multimedia are considered beneficial for both the teachers and the students in a variety of aspects. Most of these materials are knowledge- based instructions (Hill and Nelson, 2011); however, emotive videos have also been proved to be even more useful (Dorian and Litchfield, 2011). As researchers begin to explore human’s emotional connection to nature, it is important to investigate their self-awareness and behavioral implications of this connection by the revenue of educational multimedia (Perrin, 2011). That is, emotion induction is important in environmental education. Digital storytelling, an emotive instruct method (Robin, 2005), thus could be served as an effective strategy in environmental education nowadays.
The role of emotion in educational media
Emotion is a core component of human experiences, which also may lead to changes in decision-making, memory and behaviors. Among various research approaches, most of the researchers agree upon that all emotional consist of feeling states involving positive or negative valence (Schutz et al., 2006). Emotions can assist in the way information is received, retained,
doi:10.6342/NTU201801179 interpreted, and used (Perrin, 2011). There is a reciprocal relationship between thoughts and
emotions.
Elements of educational multimedia that induce emotion
Emotions are necessary for learning purpose. Learning materials generating emotions can be applied to transfer knowledge(Hinton et al., 2008). One of the material, educational story videos, was termed as “digital storytelling”.
According to Center of Digital Storytelling (2005), there are seven elements of a multimedia: perspective, a critical question, emotional content, audio narration, supportive soundtracks, economy, and pacing. These elements could further be categorized into four aspects: story (elements 1~4), audio narration (element 4), music (elements 5), and production (elements 6~7). Among these aspects, story and music have been used to induce emotion.
Storybooks have been used to instruct environmental issues. Yang (2017) also demonstrates how people connect to natural objects through an object view story in digital story form. Music, on the other hand, has also been proved to be an effective emotion stimulus no matter in or out of the context of visual presentation (Kreutz et al., 2008; Cohen, 2001). However, while most of the multimedia in environmental education are with carefully designed plots that full of originality, the music aspect has rarely been professionally dealt with. Often, free-download music was used and could not support the story to induce emotion.
Issues in soundtrack design of educational multimedia
According to Robin (2005), a well-composed soundtrack for educational multimedia should be original and could induce emotion responses (Table 1). This rubric, the only literature mentioning soundtrack design in educational multimedia, provided an overall design principle that seemed apparent. However, the researcher of this paper indicates 2 critical fallacies hidden in the rubric.
doi:10.6342/NTU201801179 The first fallacy is that a soundtrack should be judged within the context of the story.
Music that induces rich emotional responses does not necessarily trigger the same reaction when put in the visual context; on the other hand, music that is not evident could sometimes induce rich emotion if combined with appropriate visual components. A piece of soundtrack could only be useful when integrated with suitable film (Lipscomb and Kendall, 1994; Boltz et al., 1991). Viewers are not going to feel happy hearing something in major Key if the visual scene is horrible. A good example would be the movie A Clockwork Orange. The lead character, Alex, is a big fan of Beethoven and also a cruel scoundrel. When Beethoven’s symphony No.9, a passionate piece of music, was combined with horrible montage scenes in the movie, the audience feels horrified instead of excited.
The second fallacy is that the rubric did not consider the application on different multimedia. There are many ways to present storytelling multimedia. Robin (2006) categorized storytelling multimedia into 3 groups: personal narratives, historical documentaries, and stories that instruct a particular concept. Each of the group should have its soundtrack design rubric since the content, story structure, narration, and target emotion all differ from each other.
Table 1 The designed rubric of soundtracks (adapted from the research of digital storytelling by Barrett (2006))
Index Excellent Good Satisfactory Needs Improvement
Soundtrack – Emotion
Music stirs a rich emotional response.
Music stirs an emotional response.
Music is evident.
Inappropriate choice of music.
The Power of Soundtrack – Originality
All of the music is original.
Over half of the music is
original.
Some, but less than half of the music is original.
None of the music is original.
doi:10.6342/NTU201801179 Problems and needs of soundtrack design in educational multimedia
Multimedia in the educational field tends to uses free-download or compiled music. These soundtracks are thus lack of originality, and their ability to induce emotion is in doubt. One possible reason might be that for traditional knowledge-based multimedia, emotion induction is not as important, soundtrack design is thus neglected. However, when we now focus more on affection factors in educational materials, music, an effective emotion stimulus, should receive more attention and be seriously dealt with. The lack of a good design guideline might be another reason why the soundtracks nowadays are of low quality. With a better rubric, the soundtracks in educational multimedia could be more supportive for the story.
Follow upon a series of studies from a granted longitudinal project of the same research team, the present thesis extends Yang’s (2017) research on the development the effect of environmental educational multimedia and further examine the emotional impact by adding soundtrack on top of the digital storybook.
2.2 The Functions of Soundtracks in the Context of Multimedia
Although there has been little research about soundtracks in educational context, musicologists have been studied on people’s emotional reaction toward soundtracks in multimedia. According to Cohen (1999, 2001), music is one of the strongest sources of emotional expression in film, and there are 8 functions of music in multimedia. First, music masks unwanted environmental noises; second, music signals connection separation between different shots or events; third, music directs viewer’s attention to specific visual aspects or objects, and thus could induce emotion towards the focus objects; fourth, music induces mood;
fifth, music communicates meanings; sixth, music serves as a memory “hook”; seventh, music enhances the immersive experience in a film; finally, music provides another aesthetic aspect for film. Six out of the eight functions has been regarded contributive to emotion induction (Cohen, 2001). Many studies have also validated the functions mentioned above.
doi:10.6342/NTU201801179 Although research on this topic usually requires more conditions since researchers need
to specify whether emotion reaction is induced by music alone, film alone, music in the context of film, or film in the context of music, experiments have successfully proved that music plays a significant role of emotion stimulus, many validates Cohen’s statement. Although there are also many studies about how pure music influences emotion, researchers only focus on music in multimedia context in this paper.
Soundtrack as revenue to induce emotion
As Kalinak (1992, p.92) has strongly claimed, soundtracks are “the most efficient code”
that expresses emotion in film. Through pitch, timing, and loudness characteristics in music can induce different emotion that can be incorporated into the visual information (Hevner, 1936;
Rigg, 1964; Scherer, 1978). An experiment conducted by Baumgartner et al. (2006) demonstrated the above statements. When fearful and sad pictures either alone or combined with congruent emotional musical excerpts (classical pieces) were presented to the participants, the result clearly showed that music increased the emotional experience of the subjects.
Additional emotion processing brain activity was also found when music was combined with visual information.
Soundtracks as revenue for a better understanding of a story
When combined with visual information, music’s ability of emotion induction could even influence how the visual information is interpreted. The emotion triggered by music would hook themselves to the visual objects or the implied meaning of the narrative (Cohen, 2001) and could thus control, or bias the interpretation of visual events. In a study conducted by Bullerjahn and Güldenring (1994), a 10-minute film segment was presented to the subjects with one of 5 different soundtracks. The result showed that the soundtrack is indeed a factor influencing viewer’s judgements. The difference between the judgements included the appropriateness of emotional categories (sad, thrilling, sentimental, vivid), the genre of the film
doi:10.6342/NTU201801179 (horror, comedy, thriller, crime), the intention for the character’s actions, and the expected
ending of the film.
Another experiment was done by Thompson et al. (1994) further demonstrated that specific music rules did influence how the closure of a film was interpreted. In their research, a soundtrack ended with a dominant chord to tonic chord (the closed soundtrack) was proved to significantly increase the judgement of the closure of a film compared to an unclosed soundtrack, which differed only in the final bar.
How soundtracks guide the emotion and interpretation in films is especially important when the meaning of a film is ambiguous. Boltz (2001) asked subjects to watch 3 ambiguous film segments with either positive music, negative music, or no music at all. The result showed that compared to the no-music group, positive and negative music not only bias subjects’
interpretations but also their mood-congruent memory of the film. Participants in the experiment showed that music influences their judgement of characters’ relationships to one another, the intention of the characters, and even which elements are best-remembered.
Soundtracks as revenue for remembering a story
The fact that music could enhance emotional expression and lead the interpretation of a film makes film information be encoding as an integrated memory and could be remembered better. A study by Boltz (2004) demonstrated that musical soundtracks influence the remembering of film information. In the experiment, subjects viewed a set of music/film clips that were either emotional congruent or incongruent. The participants were then asked to recall the music, the film, or both in tandem. The results showed that the viewers encoded the emotion congruent music as integrated information while incongruent music encodes the film and the music independently. This study provided a possible explanation for why mood-congruent music helps remember film information (Boltz, 1991). When the audio and visual dimension in multimedia are encoded as a whole, it is easier for the viewers to remember rather than
doi:10.6342/NTU201801179 remember the music and film independently. This could also explain why the movie industry
pays serious attention to soundtrack production. Since audio information is easier to be podcasting on different media, a hit theme song could provide a better opportunity for the audience to recall information and emotional expression in a film.
2.3 The Difference between Composed and Compiled Soundtracks
Despite the negligence of soundtrack design in educational multimedia, the value of composed soundtracks should not be underestimated. A study by Lipscomb and Kendall (1994) proved that the professionally designed soundtracks do match the film better. The participants viewed 5 different segments in a feature film, Star Trek IV: The Voyage Home and asked them to choose the best fitting soundtrack for each segment. In all the segments, the soundtrack that most frequently selected was the score originally composed for the excerpt. This experiment showed that a soundtrack specifically designed for a film differs from any other music.
However, what element in music makes the composed soundtrack unique remains implicit. We may try to analyze the difference based on the 6 functions of music that induce emotion reaction in a film proposed by Cohen (2001).
Function of Soundtracks Composed Compiled 1. Provision of Continuity v △
2. Direction of Attention v △
3. Mood Induction v △
4. Communication of Meaning v △
5. A Cue of Memory v △
6. Immersion v △
v: function fulfilled △: function fulfilled incompletely Table 2 The function of composed and compiled soundtracks
doi:10.6342/NTU201801179 As shown in Table 2, for the provision of continuity, compiled music could only signals
separation but not continuity. Since all the canned music in one multimedia are with different orchestration, style, mixing, it is hard for canned music to make connections in 2 different scenes or events. For the guidance of attention, a music excerpt must either match the dynamic aspect or the implied meaning of an object. It is difficult for a piece of compiled music to meet this requirement while at the same time fits the plot and emotion of the visual part. For mood induction, Baumgartner et al. (2006) have proved that emotional congruence is a critical factor for music excerpts to support the emotional expression in a film. For communication of meaning, Bullerjahn and Güldenring (1994), Thompson et al. (1994), Cohen (2001), and Boltz (2001) have shown that emotional congruent soundtracks had the power to lead viewers’
interpretation of a film. For memory enforcement, as Boltz (2004) indicated, when the audio and visual dimension in multimedia is integrated, the viewer would remember the film better.
For immersion, since music contributes to the reality of the narrative, the more the music fits the film’s emotion, plot, historical setting, and pacing, the more the viewer can immerse in a film.
One may argue that for function 3, 4, and 5, emotion congruence plays a critical role on whether a piece of compiled music could function could function as well as composed music. This is indeed a question need to be further investigated. Although many of the educational multimedia uses emotional incongruent music as backgrounds, there are still some multimedia in which the musical emotion matches the visual information. In what way the emotion congruence compiled music differs from emotion congruence composed music thus become an issue worth investigating.
doi:10.6342/NTU201801179 2.4 Summary
To conclude, the soundtrack serves as a potentially substantial source of emotional expression in multimedia context. When combined with visual information, music could influence the emotional reaction, the interpretation, and the memory of visual events. However, it seems not all soundtracks can achieve these functions. Composed soundtracks have been proved to match film segments better and could be recognized by the viewer. However, what element in composed music that makes it unique remains implicit; in addition, if the compiled music is emotional congruence, what is the difference between composed and compiled music could be a question need further investigation. The present project strived to contribute to these unknown aspects of soundtrack design in the following chapters.
doi:10.6342/NTU201801179
Chapter 3. Soundtrack Design
In this chapter, the researcher first introduces the soundtrack elements that induce emotions. The audio-visual counterpoint, the relationship between characters and music (Leitmotiv), and the mode of music excerpts that are considered important in soundtrack design, and were further discussed. Following a practice of soundtrack design, including the story characters, theme, as well as the soundtrack design of WaterBob.
3.1 Soundtrack Elements Inducing Emotion
As Lalo Schifrin (2011), a professional score composer stated, the ultimate objective of the soundtrack is to psychologically support the film, no matter through what composing method.
“All the spectrum of emotions and mental states can be expressed by music.”, said Schifrin (2011, page). Many other composers and musicologists have also proposed numerous composing theories to validate how music induce emotion. Many score writters stressed the audio-visual relationship. The symbolic connection between characters and music (Leitmotiv) is as well emphasized. Other subtle elements including the orchestration and the harmony also need to be carefully designed through the composers’ keen intuition, rtistic techniques, and a wealth of experience.
Among all the subtle details, composer Aaron Copland (1945) concluded four elements to listen for in music: rhythm, melody, harmony, and tone color. Other musicologists categorized the “music cues” with a more complex manner (Juslin and Lindström, 2010) (e.g., pitch, mode, melodic progression, rhythm, tempo, sound level, articulation, and timbre). How the music elements associated with the emotion has also been studied. Since all the elements are intercorrelated, it is challenging to determine the emotional induction of an individual music element (Eerola et al., 2013). Nonetheless, most of the studies agree with that mode and tempo
doi:10.6342/NTU201801179 are the most critical factors for emotional reaction (Juslin and Lindström, 2010; Eerola et al.,
2013), although the influence varies among different discrete emotions. As a result, in this paper researcher focus on analyzing the mode, particularly the melody and harmonic structure, in soundtrack design. As for the design of tempo and timbre, although were also carefully dealt with, are not the central issues in this paper due to the limited context.
3.2 The Audio-Visual Counterpoint
As stated by Schifrin (2011), the music composing counterpoint theory could be applied on the audio-visual relationship. The audio-visual relationship could be either parallel or contrary (Figure 1). Parallel counterpoint relationship is waterproof and more conservative, while contrary counterpoint relationship is rather excited and is often used in an ironic or reflective situation. Since most educational media propose critical questions iming to raise environmental awareness, the latter matches more in the case.
doi:10.6342/NTU201801179 Figure 1 The audio-visual counterpoint
doi:10.6342/NTU201801179 3.3 Leitmotiv
Leitmotiv (German, meaning leading motive) is a musical theme that accompanies a specific element in which it appears. A character, a location, or a unique situation in the plot could be accompanied by a leitmotiv to signal its unique meaning in a story. Richard Wargner is typically be regarded as the first one to use this idea. In his opera Ring Cycle, hundreds of leitmotivs were identified. Among all the themes, the most essential leitmotiv that represents the critical concept Rhinegold, repeats in the opera many times throughout the 4 mammoths.
The theme appears in many different variations since the Rhinegold concept changes in the opera. In film industry, Star Wars is considered an indicator for applying leitmotiv. Each character, situation, and location all has unique themes. The theme for the lead role, Luke Skywalker, transformed from noble to sorrow, helps the audience to undergo emotion changes with Luke.
3.4 The Analysis of Mode
There are different ways to analyze the mode in music since the aesthetic values changes all the time. However, the mathematical analysis method used in the 20th century could be easily used to explain most of the music pieces existed. In the following section, Allen Forte’s analyzing method for atonal music (1976) would be applied to elaborate the soundtrack design in the educational digital storytelling the researcher and her team created.
The color of Intervals
All melodies and harmonic structures in a mode, even those most complex ones, could be destructed to intervals. The distance between every 2 notes forms an interval. Since there are 12 different pitches in a sale, there could be 12 different intervals, from 0 semitone to 12 semitones. Trough transformation, the 12 intervals could then be reduced to 7 interval classes (0=0, 1=11, 2=10, 3=9, 4=8, 5=7, 6=6). The class 0 is omitted for practical purpose, leaving 6
doi:10.6342/NTU201801179 intervals classes. As listed below, each interval class has its unique feature and emotional
influences:
Class 1 (1 semitone), the minor second: It is the most disconsonant interval.
Harmonically, it provides a distinctive characteristic and could induce a terrifying feeling. One classic example was the theme song of Jaws composed by John Williams. The melody hovers around the E and F note, providing a feeling that "grinding away at you, just as a shark would do, instinctual, relentless, unstoppable." as described by Williams. In another example, “The Crisis” from The Legend of 1900, Ennio Morricone put a semitone below every mediant in triad. In this case, the minor second provides an anguished feeling symbolizing the tragic romance.
Class 2 (2 semitones), the major second: It is a slightly dissonant, emotionally neutral interval. If serving as the transformation of the minor seventh, it conveys a sense of hope and longing (Schiffrin, 2011). In Leonard Bernstein’s “Somewhere” from West Side Story, the repeating major second expresses the desire for a better future for the lovers.
Class 3 (3 semitones), the minor third: All the third intervals are imperfect consonant.
With a flowing quality, the thirds are often used as the motive power in music. The minor trird is the foundation interval in the minor triad and could express sadness, nostalgia, or the feeling of something lost (Schiffrin, 2011). A great number of examples from Beethoven’s Symphony No.5 to “Summertime” from Porgy and Bess composed by George Gershwin would illustrate the emotion effect brought out minor third. The interval was also used in most of the musicology experiments as an association to sadness.
Class 4 (4 semitones), the major third: It is imperfect consonant, following the unison,
the octave, and the perfect fifth in the order from consonant to disconsonant. The major third is the foundation interval of the major triad and is commonly used to express positive emotions, from delightfulness to ecstasy. Examples including the major third (for example, Johann
doi:10.6342/NTU201801179 Strauss II’s Op.314 “The Blue Danube”) were commonly used in musicology experiments as
an association with happiness.
Class 5 (5 semitones), the perfect fourth: It is once considered consonant in the
medieval polyphony stage since it formed the basis for music composition. Nonetheless, when the major and minor triads become the core element in music composition, the perfect fourth is then be considered disconsonant. Nowadays, if the perfect fourth serves as the transformation of the perfect fifth, it is considered consonant; otherwise, it is disconsonant. It provides a neutral, steady feeling with sometimes a crystal clear texture. It was used at the beginning of the "Bridal Chorus" from Wagner's Lohengrin.
Class 6 (6 semitones), the augmented fourth: Containing a tritone, the augmented fourth
is the most intensive dissonant interval. Called “diabolus in musica” (the devil in music) in the Middle Ages, this interval contains overwhelming emotion that cannot be suppressed. It also has a strong tendency that leads to the perfect fifth. “Maria” from West Side Story (Leonard Bernstein) illustrated the burning desire provided by the augmented fourth.
The “Interval Vectors”
Introduced by Martino in 1961, the description of all the intervals in one combination of pitches is called an interval vector. Any pitch combination could be denoted as [ic1ic2ic3ic4ic5ic6] (ic is the abbreviation for the total number of a specific interval class). For example, the interval vector for the major triad is [001110] since there is 1 minor third (class 3), 1 major third (class 4), and 1 perfect (class 5).
The interval vector is like the color palette for a musician. With the interval vector, the color and emotion of a pitch combination could be further analyzed. For example, if the proportion of dissonant intervals is relatively more substantial, the emotion expression could also be more intensive.
doi:10.6342/NTU201801179 3.5 Practices of Soundtrack Design
Our team joined On-line to On-site Water Education (O2O), a project of Ministry of Education. In O2O, audio books and games were created to deliver environmental concept about water resource preservation. Among them, the audiobook WaterBob was chosen to be used as a practice in this paper.
The theme of WaterBob is about limited water resource (full content in Appendix). The lead character, WaterBob, is the representation of either physical water, water storage facility, or water resource agency. Three other supporting characters relatively represent different water consumption parties: the Mouse (civil livelihood water), the Cat (industrial water), and the Rabbit (agricultural water).
WaterBob’s duty was to collect water resource and distribute water resource to the 3 water consumers. In return, WaterBob got energy candy from the consumers. However, water demand increased. The priority of water distribution among different consumer parties became an issue. Moreover, water supply decreased due to droughts and pollution. WaterBob tried to negotiate with water consumers, yet none of them accepted getting less water or paying more energy candy. WaterBob was exhausted and could not supply water anymore. Not until then did the water consumers felt regret, but it was too late. WaterBob would never come back again.
As shown in Table 3, the structure of the story could be divided into 4 main sections and additional opening and ending section. In section 1, WaterBob distributed water to the consumers; in section 2, water demand increased and water supply decreased; in section 3, WaterBob tried to negotiate but was refused; In section 4, the consumers were regretful, but WaterBob would never come back. There is no plot in the opening and ending section. Only the soundtrack and a few pictures leads the viewers into the story in the beginning and last longer after the end of the story.
doi:10.6342/NTU201801179 Opening Section 1 Section 2 Section 3 Section 4 Ending
/ WaterBob
distributed water to the consumers
Water demand increased and water supply decreased
WaterBob tried to negotiate but was refused
WaterBob would never come back
/
Table 3 Sections of WaterBob
3.6 Soundtrack Design of WaterBob
Music theories were applied to design the soundtracks of WaterBob. Although there were many aspects about soundtrack design, in this research we focus on the audio-visual relationship of the video and the main theme of each character.
The audio-visual counterpoint
The full audio-visual counterpoint relationship is shown in Figure 2. In the opening section, there is no plot but a picture of WaterBob bathing in the water. Music is the dominant element leading the viewers to immerse into the story, so the music is expressive and with an unclosed ending chord that arouses curiosity. Section 1 is the introduction of the story and the main characters, so there is not much tension in the music. Section 2 is where the main crisis arises, so the music twist dramatically to alert the audience. In section 3, the crisis keeps worsening. The tension in visual presentation and the audio narrative is high enough, so the sense of the presence of music decreases, leaving space for the visual and narrative information.
In section 4, a critical issue is proposed. There is no music at all so the viewers will not be distracted. In the ending section, again there is no visual information. Nonetheless, that is when the viewer reflects what they have learned from the story. So the expressive music in the opening is used again to make the audience last longer in the immersion and think deeper.
doi:10.6342/NTU201801179 Figure 2 The audio-visual counterpoint of WaterBob
The leitmotivs and mode analysis
Leitmotiv could be applied to various elements: the characters, the environment, and even the meaning of the story. In WaterBob, however, Objective View Story (OVS) was applied, and personified objects are created to describe environmental issues from the object’s view. In this case, the characters are the most essential element that should be emphasized.
Therefore, the leitmotivs for each character were thus designed.
The instrument is designed according to the characteristic of each character, and the melody and harmony vary according to the plot and emotion expression in each section (full score in the appendix).
doi:10.6342/NTU201801179 WaterBob
The lead character WaterBoB is a tender figure with curvy and crystal appearance. The soft and round timbre of the vibraphone is thus used to represent WaterBob. The leitmotiv for WaterBob is a 4-note descending scale, illustrating the flow of the river. The semitone between G and F sharp provides a little swaying feeling. As the story develops, the mode changes from major to minor and the number of interval classes 1 and 6 (minor second and augmented fourth) increases to reflect the stressful situation faced by WaterBob.
Figure 3 WaterBob and the leitmotiv
Characteristic Instrument Section Plot Emotion Expression
Interval Vectors
Soft Tender
Vibraphone Electric piano
Water replenish Peaceful [101220](major form)
Insufficient
water Worried [101220](minor
form) [122230]
[212221](minor form)
WaterBob dying Pitiful / Table 4 The design of leitmotiv of WaterBob
doi:10.6342/NTU201801179 The Mouse
The Mouse is a tiny bathing lover. A melody played by flute is just suitable to show the agility of the mouse and the association with water. The number of interval classes 1 and 6 (minor second and augmented fourth) increases to show the anger of the Mouse.
Figure 4 The Mouse and the leitmotiv
Characteristic Instrument Section Plot Emotion Expression Interval Vector Small
Bathing lover Flute solo Clarinet Strings
Water delivery Delightful [001110](major form)
Water delivery Delightful [001110](major form) Refusing more
payment Helpless [202222]
[001110](minor form)
Table 5 The design of leitmotiv of the Mouse
doi:10.6342/NTU201801179 The Cat
Being the first customer asking for more water supply, the Cat plays the role of antagonist in the story. The bassoon is thus used to reflect the unctuous, conning characteristic of the Cat. The leitmotiv of the Cat has more disconsonant intervals than the other characters at the beginning of the story. The most dramatic harmony (with the most disconsonant intervals) appears in the second section where it is the turning point of the story.
Figure 5 The Cat and the leitmotiv
Characteristic Instrument Section Plot Emotion Expression
Interval Vector
Conning Profit- oriented
Bassoon solo Factory sound effects
Cashier sound effects
Water delivery Delightful [114112]
[111111]
Demanding more water
Queer Alerted
[445332]
[201021]
[011110](minor form) Refusing more
payment
Helpless [114112]
[323331]
[111111](minor form)
Table 6 The design of leitmotiv of the Cat
doi:10.6342/NTU201801179 The Rabbit
The guitar was chosen to show the pastoral feature of the Rabbit. The six-eight time rhythm illustrate the bouncy motion of the Rabbit. The most intensive plot happens in the second section where the Rabbit takes the lead to shout at WaterBob, so the interval vector contains more disconsonant intervals.
Figure 6 The Rabbit and the leitmotiv
Characteristic Instrument Section Plot Emotion Expression
Interval Vector
Pastoral Bouncy
Guitar Strings Vibraphone
Water delivery Delightful [001110](major form) [011110](major form) Refusing less
water Anger [001110](minor form)
[211132]
[012120]
[402122]
Refusing more
payment Helpless [111120](minor form)
[010101]
Table 7 The design of leitmotiv of the Rabbit
doi:10.6342/NTU201801179 The Main Theme
The video addressed a critical question: “How can we reach a balance between water supply and different demand customers?”. This question was not pointed out by any character, scene, nor plot, yet the audience could sense it within the story. A theme song was composed to emphasize the question and was placed at the beginning and the ending sections of the video.
The theme song provided a sad atmosphere, with an open chord that gives the story a reflective feeling.
Instrument Section Plot Emotion Expression
Interval Vector
Electric piano The Beginning The Ending
Reflective Sad
[001110] (minor form) [001110] (major form) [001110] (major form) [122230]
Table 8 The design of leitmotiv of the main theme
doi:10.6342/NTU201801179
Chapter 4. Methodology
The present research investigates the effects of soundtracks inducing emotion on environmentally responsible behavioral intentions. Thirty-six college students were randomly assigned into two groups. While watching the video, one group listened to composed soundtracks, whereas another group listened to compiled soundtracks downloaded from online musical service platform, and simultaneously they were asked to provide their emotional reactions (positive, negative, or mixture) by pressing buttons. The data were collected by a researcher self-developed documenting tool and further analyzed by visualization tool.
4.1 The Educational Multimedia - WaterBob
Two sets of soundtracks, one is researcher self-composed while another is compiled one downloaded from online musical service platform, were added onto WaterBob, a digital storybook previous created by Yang (2017), from the same research team. WaterBob, a 10- minute digital storybook, was created in hopes of increasing learners’ awareness of water conservation. In the composed group, the accompanying soundtracks were composed by the researcher following composing theories as described in chapter three; in the other group, the soundtracks were downloaded from online Creative Common music resources (music credit to Purple Plant
http://www.purple-planet.com and Youtube Audio Library https://www.youtube.com/audiolibrary/music).
Since emotional congruence has been demonstrated to be an influential factor, the emotional expression of each download music excerpt was close to the corresponding composed soundtracks. Although there was plenty music in the CC resources, it is still challenging to find music excerpts expressing exactly the same emotion. Nonetheless, the
doi:10.6342/NTU201801179 instance, two composed music segment for Cat triggered complex emotion (shown in Table 9),
so the researcher could only use download music that was as close to the composed music as possible.
Section Subsection Composed music Compiled music
Opening / Reflective Reflective
Section 1 Mouse 1 Delightful Delightful
Cat 1 Delightful
Queer
Delightful Magical
Rabbit 1 Delightful Delightful
WaterBob 1 Peaceful Peaceful
Mouse 2 Delightful Delightful
Section 2 Cat 2 Queer Alerted
Mysterious Alerted
Rabbit 2 Anger Anger
Tense
WaterBob2 Worried Worried
Section 3 Rabbit 3 Helpless Helpless
Cat 3 Helpless Helpless
Mouse 3 Helpless Helpless
WaterBob 3 / /
Section 4 Reflection / /
Ending / Reflective Reflective
Table 9 The emotion expression of WaterBob
doi:10.6342/NTU201801179 Having dealt with the emotion congruence issue, the remaining differences between the
two music groups were the music style, the orchestration, and the space of mixing, and whether leitmotivs were designed. These difference further influenced the cohesion of music segments and the coherence of the story between the two groups.
4.2 Participants
In the first stage of the experiment, recruiting advertisements was posted on PTT, indicating the purpose, procedure and potential participants of the study with compensation of NTD 250 per test. The screening process was then conducted to select two similar groups to examine the intervention of composed soundtracks vs. compiled ones. In doing so, a self- developed online questionnaire using Google Form was released to select proper participants.
The questionnaire includes 2 parts corresponding to the purpose of the present study, that is, to examine participants’ emotional induction of soundtracks in educational multimedia toward water conservation issues. The first part of the questionnaire contains 4 questions.
Every 2 questions measured Environmental Awareness and Music Awareness with 4 points grading scale (From 1 point, extremely disagree, to 4 points, extremely agree).
The Questions are listed below:
Environment Awareness
1. In general, I am aware of environmental issues.
2. I practice water conservation in daily life.
Music Awareness
1. Music influences my mood and feeling.
2. I am aware of the soundtracks while watching a video.
The average grade of questions 1 and 2 is then be defined as “Environment awareness”, while the average of the grade of questions 3 and 4 be defined as “Music awareness”.
doi:10.6342/NTU201801179 The second part of the questionnaire asked the participants to provide emotional
reactions towards video and music materials. The video material was The Beans and the Water Fairies, a three-minute water environment educational video developed by our team with
composed music. The music material was an audio file composed of four music section download online (similar to the compiled soundtracks used in quantitative data collection) (music credit to Purple Plant
http://www.purple-planet.com and Youtube Audio Library https://www.youtube.com/audiolibrary/music).
From the result of the screening questions, 36 subjects out of 100 respondents were selected and randomly divided into two groups, and the grade of Environment Awareness, Music Awareness, their emotions, and the sex ratio were balanced in both groups (Table 10).
Group Group size
Environment Awareness
Music Awareness
Emotion response (film)
Emotion response (music)
Female:
Male
Composed 18 3.00 4.00 29.16 times 24.66 times 12:6
Compiled 18 3.02 3.61 30.33 times 23.88 times 12:6 Table 10 Grouping Result
doi:10.6342/NTU201801179 4.3 Data Collection
To serve the purpose of research, documenting tool as well as the following measurement tools were adopted to collect data. A visualization tool build up with Tableau was also utilized to collect and further analyze data.
Interactive Response
Since there are many sections in WaterBob inducing different emotional reactions, it is necessary to document the interactive responses from the viewers while watching the digital story.
Emotion Categoriess
Within literature, emotions are often classified into two categories: positive or negative emotions (Hinton et al., 2008; Schutz et al., 2006; Boltz, 2004). Although the emotional expressions in the segments of WaterBob are rich, in the experiment the researcher conclude all the expressions into 3 emotion types: positive, negative, and mixture (Figure 7). Although many musicology experiments use measurements that categorizes emotion to more than 3 categories (for example, the Geneva Emotion Wheel contains 20 emotion types), they are too complicated for this research. Since the interactive response is asked to be provided while watching the film simultaneously, the viewer may be distracted from the film when responding their emotion if there are too many emotion types to choose. With just three categoriess to choose, the distraction would not be slight and could be neglected.
doi:10.6342/NTU201801179 Figure 7 The emotion categories of WaterBob
Documenting Tool
A website (Figure 8) produced by HTML and JavaScript was designed for the respondents to record their emotional reactions while watching the material simultaneously.
As shown below, 3 buttons (positive, negative, or mixture) were placed below a video player screen. Users could easily record their emotional reactions and the triggered times.
Figure 8 The documenting tool
doi:10.6342/NTU201801179 Visualization Tool
An Excel file was specifically designed for the experiment. With the format prepared in advance, the data recorded from the documentary tool could be presented in figure immediately. The researcher could recognize the unexpected emotional responses right after the participants finishing watching the material, so the indication of those unexpected emotional responses could then be clarified during the interview.
For further analysis of the data, Tableau Public was used as the visualization tool. Tableau Public allows the researcher to analyze multidimensional data with various calculation functions and to present the data.
4.4 Experiment Design and Data Analysis
The purpose of the study is to exam whether composed soundtrack could significantly induce expected emotional responses. To answer the research question one, the simple t-test, also known as the independent samples t-test, was used for comparisons with a continuous dependent variable. In our case, participants were invited to watch two videos, composed vs.
compiled ones. Simple t-test in SPSS 23 was used to provide full details of the t-test calculation, including sample mean, the sum of squares, and standard deviation.
4.5 Procedure
Upon arriving at the lab, the participants watched WaterBob with either composed or compiled soundtracks. Participants’ emotional responses to soundtracks triggered by educational multimedia, the WaterBob, were collected. That is, interactive emotional reactions (positive, negative, or mixture) toward the material were asked to be provided simultaneously.
After watching WaterBob, follow-up interviews were held to understand the details behind their responses. A set of questions were asked to know what the participants feel about the
doi:10.6342/NTU201801179 visual and audio information. The meaning of the emotional responses gathered in the
qualitative investigation was also specified in the interview.
Interactive Emotional Responses
The participants viewed the material individually. A Macbook and a mac earphone were used as the media. Before the investigation, the participants received the following instructions explaining task requirement:
Please press the buttons (Positive, Negative, or Mixture) which represent your emotional reaction while watching the video.
Positive: including but not limited to exciting, entertaining, delightful, passionate, encouraging...
Negative: including but not limited to nervous, angry, sad, fearful, guilty...
Mixture: all feelings that cannot be categorized into Positive nor Negative Follow-up interviews
Since the emotion responses could only be categorized into 3 types, the exact meaning is vague. Thus, an interview is needed for the researchers to further explain the quantitative data. A self-designed question set was used in the interview. The question set can be divided into 3 parts: questions about the video and music, questions about the specific meaning of particular emotion reactions, and general questions. The first and second part of the questions were used to explain the quantitative result. The general questions were designed to get a clear view about what people think about the educational multimedia nowadays. Since in the experiment only one material was used, the researcher was interested in the participants’
opinion and feelings toward other educational multimedia.
doi:10.6342/NTU201801179 Question set
Questions about WaterBob
Do you remember any particular plot or music?
Is there any particular plot or music that you like/ dislike? Why?
What do you think about the plot design?
What do you think about the music design?
Questions about emotion reactions
Unexpected emotion reactions will be identified. The participants would be asked to explain what they feel that emotion at that time.
General questions
Have you ever watched education multimedia before?
If yes, could you briefly describe the theme and presentation of the multimedia?
How many detail could you still remember?
What’s your feeling and reflection after watching that multimedia?
Is the multimedia knowledge driven or emotion driven? with real person or in animation?
Do you remember the background music in the multimedia?
Do you think the visual and audio matched each other?
doi:10.6342/NTU201801179
Chapter 5. Results and Discussion
The researchers analyzed the emotion induction between the two groups and found a significant difference. Through a simple t-test, a significantly higher environmentally responsible behavioral intent was suggested in the intervention group than in the control group.
The follow-up interview also showed a positive relation between emotional induce and environmentally responsible behavioral intentions.
5.1 The Heat Map of Emotional Induction
The emotional responses from individual participants are shown in Figure 9. Each strip represents one button hit from the participants. All the button hits were further gathered and summarized in Figure 10, which illustrates a more explicit emotion flow while watching the video.
Since the emotional settings for music in both groups, composed vs. compiled soundtracks, are the same, we can see that the overall emotional response pattern from the 2 groups is similar. However, with closer examination, it is found that there are some regions in which the participants answered unexpected emotional responses.
To further explain the details and specific phenomenon, the participants were asked to explain what is the meaning of those unexpected emotional. These follow-up interview data were used to further analyze the click data exhibit in the heat map. First, researcher recognized some irrelevant data; for example, two participants pressed “negative” when Mouse, one of the main characters in WaterBob, first came out. They reported that the reason they did that was because they were afraid of mice. This emotional response is not related to music design nor storyline. Some other participants reported negative responses about the narration or the breaking time were also identified and removed.
doi:10.6342/NTU201801179 Composed Soundtrack Group
Compiled Soundtrack Group
Figure 9 The heat map of emotional induction
doi:10.6342/NTU201801179 Composed Soundtrack Group
Compiled Soundtrack Group
Figure 10 The emotional flow
doi:10.6342/NTU201801179 After removing all the irrelevant data, all the unexpected emotional responses were
summarized. Three particular sections in which the experiment group hits were identified as unexpected emotion (Figure 11). To be specific, the plot of the three sections are: (1) the Cat asked for more water supplement, (2) WaterBob cannot find water, and (3) the consumers felt regret about what they had done.
Figure 11 The unexpected emotion responses
doi:10.6342/NTU201801179
41
From Figure 11 we can see that the hitting frequency varies in participants. To eliminate the influence caused by this difference, we use the below formula to normalize the data.
C = ∑ (&'× )*
+ × )') (Equation 1) P = .
)* (Equation 2) Where:
C = count of unexpected emotion P = Percentage of unexpected emotion
ui = unexpected emotion response in the region (individual) et = emotion response in the region (total)
ei = emotion response in the region (individual) N = group size
After eliminating the influence of differences on the count of emotion, the result is shown in Table 11. In the composed music group, the count of unexpected emotion in the three sections are 4.82, 0.00, and 0.00, all lower than those in the compiled music group which are 14.08, 9.95, and 7.30, with t=3.02, 2.89.2.53, respectively, all of p<.001. For the percentage of unexpected emotion, in the composed music group are 3.30%, 0.00%, and 0.00%, all lower than those in the compiled music group who are 9.58%, 5.38%, and 5.29%. Both the count and percentage showed that the accuracy of emotional reaction is higher in the composed music group.
doi:10.6342/NTU201801179 Section Plot
Count of Unexpected Emotion
(C)
Percentage of Unexpected Emotion
(P) t-test
Composed music (n = 18)
Compiled music (n = 18)
Composed music (n = 18)
Compiled music (n = 18)
1 The Cat asked for more water supplement
4.82 (1.22) 14.08(5.43) 3.30% 9.58% t= 3.02(p<.001)
2 WaterBob cannot find water
0.00 9.95(2.65) 0.00% 5.38% t=2.89(p<.001)
3 Reflection of the consumers
0.00 7.30(1.09) 0.00% 5.29% t=2.53 (p<.001)
Table 11 The count and percentage of unexpected emotion (t-test)
Discussion
There are three sections in which the experiment group hits more unexpected emotion.
From the interview, we found that in these sections the participants interpreted the story falsely;
for example, in section one, participant no.4 reported “I felt happy for the Cat since it got more orders”; in section two, participant no.89 reported “I thought the polluted river is a hope for WaterBob”; in section three, participant no.19 and no.89 reported “I thought the animals were self-reflecting so maybe good things could happen”.
