Colloquium
Analyzing the latent emotional transfer pattern (LETP) of a learning
community in an online peer-assessment activity
_1301113..116Huei-Tse Hou and Kun-Hung Cheng
Address for correspondence: Dr Huei-Tse Hou, Graduate Institute of Applied Science and Technology, National Taiwan University of Science and Technology, 43 Keelung Road, Section 4, Taipei 106, Taiwan. Email: hthou@ mail.ntust.edu.tw. Kun-Hung Cheng, Digital Content Production Center, National Chiao Tung University, Hsinchu.
Introduction
Peer assessment is utilized extensively in digital learning, and many studies have examined the positive influences of online peer assessment on learning (eg, Tseng & Tsai, 2007). However, other studies indicate that this type of activity has negative influences. For example, students may question the fairness of an assessment or disagree with others’ assessments (eg, Kaufman & Schunn, 2010). These effects may lead to changes in the collective emotional state of the learning community and affect the organizational climate and pattern of interactions. Because a commu-nity’s knowledge sharing is correlated with its organizational climate (Bock, Zmud, Kim & Lee, 2005), negative comments may lead to negative emotions, especially in online discussions that incorporate peer assessment (Falchikov, 2005). The latent emotions in students’ interactions may be dynamically affected by their peers’ comments and criticism.
This study focuses on the overall pattern of latent emotional changes, which we call the “latent emotional transfer pattern” or LETP. Teachers who are able to successfully monitor a learning community’s LETP are better able to provide timely guidance, control the spread of negative emotions among students and prevent negative influences on learning. Analysis of LETP may also benefit studies in social psychology and online learning.
Research on the overall pattern of learners’ emotional transitions in online peer-assessment activities is extremely rare. The question of how to effectively identify and evaluate a learning community’s emotional transfer pattern has not been discussed. The Self-Assessment Manikin, or SAM (Lang, 1985), is a self-report scale that can quickly, easily and effectively measure the emotional responses of users in different circumstances (Morris, 1995). The SAM is a pictorial scale that uses the dimension of “Valence” to assess whether an emotional response is positive or negative. The SAM includes pictures of facial expressions on a scale from 1 to 9 (from negative to positive emotional responses). Respondents assess their emotional state by selecting a picture that best reflects how they feel. Appropriate analytical techniques are needed to analyze the transi-tional pattern of emotions in online interactions. Lag sequential analysis (Bakeman & Gottman, 1997; Hou, Chang & Sung, 2010) facilitates the analysis of the sequential correlation between two or more behaviors or phenomena in a learning community and has been used in the analysis of visual behavioral transfer patterns (eg, Hou et al, 2010).
The purpose of this study is to utilize the lag sequential analysis technique to examine students’ emotions toward online peer assessment, as reported on the SAM. This paper represents an unprecedented attempt to explore the effectiveness and practical use of LETP in online learning. Method
The participants in this study were 65 college students in a multimedia production course in northern Taiwan. The students were required to produce a film on a certain topic as a group British Journal of Educational Technology Vol 43 No 4 2012 E113–E116 doi:10.1111/j.1467-8535.2012.01301.x
© 2012 The Authors. British Journal of Educational Technology © 2012 BERA. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA.
project. The students were divided into 21 groups with two or three students per group. An online forum with basic posting and response features was established for this study. Each group was asked to provide a URL that linked to their film and a description of their work on the forum. Each group was then asked to view the other groups’ films and provide peer-assessment comments and responses. In addition to commenting on other groups’ work, the participants could respond to comments from other groups. To assess the emotional state of participants who posted com-ments, a column was added to the interface that asked the students to describe how they felt. Students could only post messages after this column was completed. The column was based on the “Valence” dimension of the SAM, in which participants select a picture (on a 1–9 scale) that reflects their emotional state (the selected picture is automatically coded as “e1” to “e9” in the database). These emotional responses were not displayed in the forum and were only recorded in the database for follow-up analysis. During the activity, the teacher provided no guidance or feedback to ensure an accurate observation of the LETP during peer assessment. The emotional scores were organized in chronological order based on the posting time for sequential analysis (Bakeman & Gottman, 1997).
Results and discussion
We acquired 665 codes from the 21 groups. Sequential analysis yielded the Adjusted Residuals Table, as shown in Table 1. Each row in the table represents a starting emotional state, and each column represents the immediate follow-up emotional state. A sequence with a value of 1.96 or greater indicates its statistical significance (p< .05). The emotion-transition diagram, as shown in Figure 1, was constructed based on the significant sequences. Each circle represents the emo-tional status code, and arrows indicate the direction of sequences. The value on the arrowhead is the z-score, and its thickness represents the significance of the sequence. The nine positive and negative emotional codes were put into three blocks: negative emotions (e1~e3), neutral emotions (e4~e6) and positive emotions (e7~e9). The figure provides an initial understanding of the LETP of the learning community during the peer assessment.
We found that most positive emotions remained stable (ie, e8→e8, e9→e9). Most of the neutral emotions also remained stable (e5→e5, e6→e6), but some emotions in the neutral-emotion block moved downward (eg, e5→e4) or to the negative-emotion block (eg, e4→e2). Some emotions in the negative-emotion block tended to gradually become more positive (eg, e2→e3, e1→e6), but most of the transitions were of a small magnitude (e2→e3, z = 6.84). Occasional transitions of negative emotions into the neutral-emotion block were also observed (e1→e6), but their level of significance was lower (z= 2.01). It is worth noting that none of the sequences of negative emotions moving to the positive-emotion block reached the level of significance.
Table 1: Adjusted residuals table for sequential analysis (lag= 1)
Z e1 e2 e3 e4 e5 e6 e7 e8 e9 e1 2.63* -0.15 -0.47 -0.08 1.37 2.01* -0.61 -2 -1.27 e2 1.83 -0.7 6.84* -0.68 -1.12 -0.39 0.67 -0.41 -0.89 e3 -0.45 -0.28 -0.12 -0.27 -0.45 0.94 0.51 0.55 -1.02 e4 1.13 2.67* -0.26 -0.62 0.05 0.64 -1.41 -0.05 -0.52 e5 0.18 -1.1 -0.46 2.02* 3.42* -0.07 0.31 -1.1 -1.47 e6 -1.3 0.46 -0.7 -0.16 -0.38 2.19* 0.91 -1.63 -0.18 e7 -1.07 -0.32 0.54 -0.83 0.89 0.07 1.28 -0.54 -0.34 e8 -0.59 -1.49 -0.87 -0.14 -1.78 -1.43 0.32 2.53* 0.55 e9 -0.05 1.39 -1.04 0.44 -1.13 -1.8 -1.85 1.3 2.15* *p< .05. e, emotional state.
E114 British Journal of Educational Technology Vol 43 No 4 2012
We discovered that in online peer assessment, most of the students’ positive emotions did not move to the neutral- or negative-emotion block. However, neutral and negative emotions were unlikely to move into the positive-emotion block. Although emotional transitions may move to a more positive state (e2→e3, e1→e6), negative emotions still failed to move to the positive-emotion level, and neutral emotions often moved to the negative-emotion block (e5→e4, e4→e2). Based on the results of this analysis, we were able to create a visual presentation of the learning group’s LETP as an important reference for researchers and teachers to provide guidance for this type of learning activity.
Conclusion and suggestions
In this study, we utilized the pictorial emotion scale of the SAM and sequential analysis as an innovative way to examine the LETP of online peer assessment. Our initial findings indicate that in online peer assessment, students’ positive emotions may occur continuously and do not tend to shift to neutral or negative emotions. However, neutral and negative emotions are unlikely to change to positive emotions, but negative emotions are also unlikely to shift to neutral emotions. This study demonstrates the LETP of peer assessment and its potential limitations in establishing a positive organization’s atmosphere. Because knowledge sharing is correlated with organiza-tional climate (Bock et al, 2005), we advise teachers to intervene in peer assessment in a timely fashion with an online SAM, especially when “negative emotions occur frequently” or “neutral emotions shift to negative ones”, to promote positive interactions for knowledge construction. Researchers may utilize our innovative analytical technique to analyze the content of learners’ online interactions and discussions as a way to determine the timing of encouragement and mediation and to design optimal guidance strategies. Future researchers may consider implementing SAM and sequential analysis in various online-learning systems. Automated and
Figure 1: Emotion-transition diagram
Colloquium E115
visualized real-time analysis would provide teachers and researchers of online-learning commu-nities with an instant LETP as a reference for providing educational guidance or qualitative analysis.
Acknowledgements
This research was supported by the projects from the National Science Council, Republic of China, under contract number NSC-100-2628-S-011-001-MY4, NSC-100-3113-S-011-001, and NSC -99-2511-S-011-007-MY3.
References
Bakeman, R. & Gottman, J. M. (1997). Observing interaction: an introduction to sequential analysis (2nd ed.). Cambridge, UK: Cambridge University Press.
Bock, G. W., Zmud, R. W., Kim, Y. & Lee, J. (2005). Behavioral intention formation knowledge sharing: examining roles of extrinsic motivators, social-psychological forces, and organizational climate. MIS
Quarterly, 29, 1, 87–111.
Falchikov, N. (2005). Improving assessment through student involvement. London: Routledge-Falmer. Hou, H. T., Chang, K. E. & Sung, Y. T. (2010). Applying lag sequential analysis to detect visual behavioral
patterns of online learning activities. British Journal of Educational Technology, 41, 2, e25–e27.
Kaufman, J. H. & Schunn, C. D. (2010). Students’ perceptions about peer assessment for writing: their origin and impact on revision work. Instructional Science, 39, 3, 387–406.
Lang, P. J. (1985). The cognitive psychophysiology of emotion: fear and anxiety. In A. H. Tuma & J. D. Maser (Eds), Anxiety and the anxiety disorder (pp. 131–170). Hillsdale, NJ: Lawrence Erlbaum.
Morris, J. D. (1995). Observations: SAM: the self-assessment manikin—an efficient cross-cultural measure-ment of emotional response. Journal of Advertising Research, 6, 63–68.
Tseng, S. C. & Tsai, C. C. (2007). On-line peer assessment and the role of the peer feedback: a study of high school computer course. Computers & Education, 49, 1161–1174.
E116 British Journal of Educational Technology Vol 43 No 4 2012
