ORIGINAL ARTICLE
Validation of the Impact of Event Scale-Revised for
adolescents experiencing the floods and mudslides
Cheng-Sheng Chen
a,b,c, Chung-Ping Cheng
d, Cheng-Fang Yen
b,c,*, Tze-Chun Tang
c,
Pinchen Yang
a,c, Rei-Cheng Yang
e,f, Ming-Shyan Huang
g,h, Yuh-Jyh Jong
b,f,i,
Hsin-Su Yu
j,ka
Department of Psychiatry, Faculty of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
b
Graduate Institute of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
c
Department of Psychiatry, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
d
Department of Psychology, National Cheng Kung University, Tainan, Taiwan
e
Department of Pediatrics, Faculty of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
fDepartment of Pediatrics, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan g
Department of Internal Medicine, Faculty of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
h
Department of Internal Medicine, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
i
Department of Laboratory Medicine, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
j
Department of Dermatology, Faculty of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
k
Department of Dermatology, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
Received 9 February 2011; accepted 11 May 2011 Available online 23 November 2011
KEYWORDS Adolescent; Impact of Event Scale-Revised; Posttraumatic stress disorder; Reliability; Validity
Abstract The purpose of this study was to validate the Impact of Event Scale-Revised (IES-R) for adolescents who had experienced the floods and mudslides caused by Typhoon Morakot in Taiwan. The internal consistency, construct validity, and criteria validity of the instrument were examined. Principal component analysis followed by an oblique rotation was used to derive a three-factor solution. These factors were labeled intrusion, hyperarousal, and avoid-ance; all three factors together accounted for 58.1% of the variance. The total Cronbach’s alpha of 0.94 reflected the good internal consistency of the instrument. With reference to diagnosis of posttraumatic stress disorder, the IES-R cutoff point for posttraumatic stress disorder was 19 of 20 with a sensitivity of 85.7% and specificity of 84.1%. In conclusion, the
* Corresponding author. Department of Psychiatry, Kaohsiung Medical University Hospital, No. 100, Tzyou 1stRoad, Kaohsiung 807, Taiwan.
E-mail address:chfaye@cc.kmu.edu.tw(C.-F. Yen).
Available online atwww.sciencedirect.com
journal homepage : http://www. kjms-online. com
1607-551X/$36 Copyrightª 2011, Elsevier Taiwan LLC. All rights reserved. doi:10.1016/j.kjms.2011.06.033
IES-R can be used as a reliable and valid instrument when evaluating psychological distress among adolescents who have experienced a natural disaster, such as flooding and mudslides. Copyrightª 2011, Elsevier Taiwan LLC. All rights reserved.
Introduction
Typhoon Morakot brought unprecedented amounts of rain-fall in Taiwan peaking at 2,777 mm (109.3 in.) and caused severe flooding and enormous mudslides on August 8, 2009. The result was catastrophic damage in southern Taiwan, which left more than 700 people dead and caused roughly $3.3 billion USD in damages. Most of deaths in this disaster were as a result of terrifying mudslides, and nearly all of the inhabitants living in the affected mountainous areas experienced the threat of mudslides. Typhoon Morakot and the associated floods and mudslides had a great psycho-logical impact on these inhabitants (http://en.wikipedia. org/wiki/Typhoon_Morakot).
Adolescents as a population are vulnerable to post-traumatic stress disorder (PTSD) when they are exposed to life-threatening stressors. Epidemiological studies for PTSD in Taiwan have shown higher prevalence of PTSD among the people experiencing the earthquake disasters and infection outbreak [1,2], including adolescents [3,4]. PTSD symptomatology in adolescents resembles that in adults but also includes some aspects of the childhood presentation with modifications [5]. Adolescents might present more frequently with repetitive and intrusive thoughts and flashbacks, sleep disturbance, anger, sepa-ration anxiety, and cognitive problems[6]. Therefore, it is not always suitable to detect PTSD in adolescents using an instrument that has not been specifically validated for adolescents.
There has been several instruments developed for screening of PTSD, such as Clinician-Administrated PTSD Scale [7], PTSD Checklist [8], Post-traumatic Stress Diag-nostic Scale[9], and the Impact of Event Scale (IES). The latter three instruments were self-administered with similar numbers of items. Each of these assessment tools have their own merits. As our study team has used IES to screen stress reaction among the hospital adult workers during the severe acute respiratory syndrome outbreak[2], we would like to examine its use for the adolescents in the natural disaster. IES was developed by Horowitz et al.[10]
as a measure of psychological stress reactions after trauma. IES is a short, easily administered, self-report questionnaire of 15 questions. It has been widely adopted and used in related research for more than 30 years. The revised version of the IES (IES-R) contains seven additional items related to the hyperarousal symptoms of PTSD [11]. The IES-R has been used to measure mental reactions in response to stressors, such as injury [12,13] and illness
[14,15]. Despite its wide use, this tool has not been specifically and formerly validated for posttraumatic responses to natural disaster among adolescents or children.
The aim of this study was, therefore, to examine the internal consistency, factor structure, and criteria validity of the IES-R in a sample of adolescents who had
experienced the threat of floods and mudslides caused by Typhoon Morakot. Additionally, the implications of these findings, in terms of the psychometric properties of the IES-R for use with natural disasters, are discussed.
Methods
This study was conducted in the heavy damaged area 3 months following the floods and mudslides caused by Typhoon Morakot. A total of 277 adolescents who were in Grade 7e9 from three junior high schools were recruited into this study. They had been separated from their families and allocated to three boarding areas where they could continue to receive formal education in a group setting. Most of them were living in group dormitories and visits involving family members were only possible at weekends. The Institutional Review Board of Kaohsiung Medical University approved the study, which allowed the use of passive consent from the parents of 277 displaced adoles-cents. Before conducting the study, we prepared a leaflet explaining the purpose and procedure of this study. Students were told to take the leaflet with them at a weekend for their parents or main caretakers to inspect. If parents disagreed about taking part in the study, they could tell their children to refuse to participate. Written informed consent was also obtained from each student themselves before the study started.
The participants were invited to complete the self-administered 22-item Chinese version of IES-R [11,16]. Participants were asked to report the frequency of symp-toms over the past week on a 5-point scale ranging from 0 (not at all) to 4 (extremely). The IES-R yields total scores that range from 0 to 88. The sum of all individual scores was taken to indicate the severity of the psychological reactions to the previous stress.
Blind to the results of the IES-R, diagnostic interviews were conducted using Mini-International Neuropsychiatric Interview for Children and Adolescents (M.I.N.I.-Kid) schedule [17]. Based on the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (American Psychi-atric Association, 1994), the M.I.N.I.-Kid assesses 22 common psychiatric disorders and suicide risk in children and adolescents. It has been used previously among early adolescents in Taiwan[18]. Because of time limitations and specific purpose of this study, only the PTSD module of the M.I.N.I.-Kid was used for the interview. Fifteen psychiatric staff, made up of two senior board-certified child psychi-atrists, five psychipsychi-atrists, five clinical psychologists, and three psychiatric nurses participated as interviewers. All of them had many years of clinical experience in both adolescent and adult psychiatry. Before conducting the diagnostic interviews with the adolescents who had expe-rienced Typhoon Morakot, all interviewers received comprehensive training in the use of the M.I.N.I.-Kid. First, all interviewers attended a workshop on the M.I.N.I. held by
the Taiwanese Society of Psychiatry. The workshop intro-duced the M.I.N.I.’s concept, its content, and the principle of use; it also included a video-recorded case interview using the M.I.N.I. that allowed the attendees to rate a diagnosis. Second, two senior board-certified child psychiatrists (P. Y. and C. F. Y.) delivered a seminar to the other interviewers to introduce the M.I.N.I.-Kid modules. Third, each interviewer conducted a diagnostic interview with an adolescent using the PTSD module of the M.I.N.I.-Kid under the supervision of a child psychiatrist (C. F. Y.) and then received feedback about the interview. Lastly, the rater diagnostic reliability among the 15 inter-viewers was examined using six tape-recorded M.I.N.I.-Kid case interviews. The kappa for the PTSD ratings ranged from 0.67 to 1, which indicated good inter-rater diagnostic reliability.
Cronbach’s alpha coefficients were calculated to determine the internal consistency of the IES-R. Explor-atory factor analysis, using principle component analysis followed by oblique rotation, was used. Factor numbers were determined using the scree test and needed to explain at least 10% of the variance. In the scree test, the eigenvalues of all the factors were examined after each factor was extracted until a large jump or discontinuity was observed, after which the factors that remained were retained. Items with loadings greater than 0.3 were entered into a factor. If the factor loading was greater than 0.4 for more than two items, the author would judge the appropriate factor for the item. The criteria validity of IES-R against the diagnosis of PTSD as generated by the M.I.N.I.-Kid was determined via relative operating characteristic (ROC) analysis [19]. Accordingly, the optimal cutoff point was estimated with respect to sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
Results
Six students were excluded because of absence on two consecutive visits. There was no difference in the gender ratio of these six students (nZ 6, 4 boys and 2 girls) with that of the groups who participated. However, these six students were a little older than their counterparts (Z of Mann Whitney U testZ 2.645, p < 0.01). In total, 271 students (97.8%) completed all questionnaires and the interview. Among them, 124 (45.8%) were boys and 147 (54.2%) were girls. Their mean age was 13.4 years (standard deviationZ 1.0 years, range: 12e15 years). Of the 271 students, 70 (25.8%) met the PTSD diagnostic criteria in relation to Typhoon Morakot. The mean of the total scores of the IES-R was 16.9 (standard deviationZ 16.2) with a range from 0 to 82.
Factor analysis yielded three factors with eigenvalues exceeding unity, and these accounted for 58.1% of the total variance. Six items (Item 1, 3, 6, 9, 14, and 20) with load-ings greater than 0.3 made up the Factor I subscale accounting for 20.1% of the explained variance. Factor II consisted of nine items (Item 5, 7, 8, 11, 12, 13, 14, 17, 21, and 22) and accounted for 18.4% of explained variance. Factor III consisted of seven items (Item 2, 4, 10, 15, 16, 18, and 19) and accounted for 19.6% of the explained variance.
According to the meanings of the items making up each factor, the three factors were labeled as intrusion, hyper-arousal, and avoidance, respectively (Table 1).
Based on the items making up the newly extracted factors, the analyses of internal consistency revealed Cronbach’s alpha values of 0.89, 0.85, and 0.88 for the intrusion, avoidance, and hyperarousal subscales, respec-tively. The overall Cronbach’s alpha for the total IES-R was 0.94.
The criteria validity of the IES-R for PTSD was examined using ROC analyses (Fig. 1). The area under the ROC curve was 0.92 (95% confidence intervalZ 0.88e0.96), which indicated excellent criteria validity for the IES-R in relation to a diagnosis of PTSD.Table 2shows the criteria validity of the IES-R in terms of sensitivity, specificity, PPV, NPV, and overall misclassification rate (OMR) at various different cutoff points. As the IES-R is treated to be used as a screening tool, which suggests a requirement for greater sensitivity, we suggest that the optimal cutoff point for the IES-R should be 19 of 20, which gives a sensitivity of 85.7%, a specificity of 84.1%, a PPV of 65.2%, a NPV of 94.4%, and an OMR of 15.5%.
Discussion
In this study, we found that the IES-R as an instrument for evaluation of adolescent psychological stress associated with exposure to nature disasters, such as flooding and mudslides, has a three-factor structure. These factors assess intrusion, hyperarousal, and avoidance symptom-atology. The high Cronbach’s alpha values for all subscales as well as the total scale indicates very good internal consistency. The optimal cutoff point for the IES-R in this study, when detecting PTSD, was determined to be 19 of 20, which gives both good sensitivity and good specificity.
The IES-R factor structure in this sample was identical to that in the original studies at the component level with minor differences at the item level[20], although a valida-tion study did not support three factors model for a sample of accident participants or emergency room patients in Hong Kong[16]. As the latter article explained, difference in validation of IES-R may come from different clinical samples and age groups studied. Similarities in the IES-R factor structure when the present study and the original study are compared imply that the same psychological distress psychopathologies occurred with the present flooding stressor. Compared with the original factor analysis
[20], four items in our study did not load on their originally proposed factors. These were Item 2 (I had trouble staying asleep) and Item 16 (I had waves of strong feelings about it), which was reclassified as hyperarousal rather than intrusion; Item 14 (I found myself acting or feeling like I was back at that time), which was reclassified as intrusion rather than hyperarousal; and Item 21 (I felt watchful and on guard) from hyperarousal to avoidance. These inconsis-tencies at item level may be attributed to differences in the presentation of PTSD between adolescents and adults. In addition, it may also reflect that these items are more ambiguous because they each incorporated aspects of all three factors. In these circumstances, it is not unexpected
that these loading may not occur consistently on the same factor during a different study.
We found good internal consistency coefficients for all three extracted factors and as a whole the reliabilities were also good; this implies that each of the measures is a relatively homogeneous construct. The criteria validity of the IES-R was demonstrated by the high validity index against the diagnosis of PTSD using a standardized interview schedule. The cutoff point suggested here is 19 of 20, although greater specificity, PPV, and OMR can be had by a cutoff of 20 of 21. However, the difference between the two cutoff points is small. There is a need for greater sensitivity when the IES-R is used as a screening tool and therefore it is suggested that adolescents with scores of 20 and more from the IES-R should be selected as possibly suffering from PTSD.
At this cutoff point, the PPV was 65.2%, indicating that 34.8% of the adolescents scoring 20 or more would not meet the PTSD diagnosis. This relatively higher false-positive rate
Table 1 Factor loading of the Impact of Event Scale-Revised items
Original factor and items[20] Factor I Intrusion Factor II Avoidance Factor III Hyperarousal Intrusion factor
1 Any reminder brought back feelings about it 0.883 0.087 0.023
2 I had trouble staying asleep 0.143 0.212 0.884
3 Other things kept making me think about it 0.946 0.218 0.103 6 I thought about it when I did not mean to 0.492 0.414 0.101 9 Pictures about it popped into my mind 0.494 0.370 0.023 16 I had waves of strong feelings about it 0.278 0.251 0.379
20 I had dreams about it 0.341 0.238 0.182
Avoidance factor
5 I avoided letting myself get upset when I thought about it or was reminded of it
0.153 0.347 0.151 7 I felt as if it had not happened or was not real 0.113 0.598 0.176 8 I stayed away from reminders about it 0.004 0.781 0.038
11 I tried not to think about it 0.126 0.457 0.389
12 I was aware that I still had a lot of feelings about it, but I did not deal with them
0.057 0.731 0.055 13 My feelings about it were kind of numb 0.070 0.508 0.212 17 I tried to remove it from my memory 0.142 0.646 0.112
22 I tried not to talk about it 0.303 0.466 0.032
Hyperarousal factor
4 I felt irritable and angry 0.060 0.113 0.512
10 I was jumpy and easily startled 0.150 0.201 0.444
14 I found myself acting or feeling like I was back at that time 0.502 0.334 0.024
15 I had trouble falling asleep 0.026 0.121 0.917
18 I had trouble concentrating 0.122 0.208 0.608
19 Reminders of it caused me to have physical reactions,
such as sweating, trouble breathing, nausea, or a pounding heart
0.195 0.086 0.457
21 I felt watchful and on guard 0.045 0.570 0.127
0.883 0.087 0.023
% Of explained variance 20.1 18.4 19.6
Reliability (Cronbach’s alpha) 0.90 0.85 0.88
1.0 0.8 0.6 0.4 0.2 0.0 1 - Specificity 1.0 0.8 0.6 0.4 0.2 0.0 Sensitivity
Figure 1. The relative operating characteristic curve of the revised version of the Impact of Event Scale at different cutoff points.
may reflect that anxiety syndromes other than PTSD are present in the adolescent. In addition to PTSD, three other distinct types of posttraumatic response, namely depres-sion, anxiety disorders, and behavioral disorders, have been proposed[21]. It was possible that those with anxiety disorder but not PTSD also show up with higher IES-R scores. Previous studies have found that anxiety disorder may include social phobia, agoraphobia, and separation anxiety disorder [22,23]. Our findings seem to suggest that, in addition to the IES-R’s use as a screening tool, other anxiety symptomatology assessment tools are needed for adoles-cents in these circumstances. Nonetheless, overall, this study demonstrates that the IES-R performs well as a screening instrument for PTSD among adolescents and is very adequate in this role.
Study caveats need to be addressed. This study was conducted about 3 months after Typhoon Morakot had occurred. Thus, this result cannot be generalized to the very acute postdisaster stage. At the acute stress stage, it is possible that the posttraumatic reactions are more salient and show more variability. Resilience can help some people to return to a normal mental state from an acute stress reaction quickly, whereas others suffer from persistent mental dysfunction. They are the candidates for additional mental help during the reconstruction or rehabilitative stage after a disaster. In this context, we suggest that the IES-R needs to be testified for identifying this group to provide a higher range of application. In addition, conver-gence and diverconver-gence validity testing needs to be testified to strengthen the validation in the further studies.
In conclusion, our study suggests that the IES-R appears to be a promising and readily applicable instrument for the quick and early detection of psychological distress among adolescents who have experienced a natural disaster, such as a severe threat of flooding and mudslides.
Acknowledgments
This study was supported by grants (NSC98-2321-B-037-063 and NSC98-2410-H-037-005-MY3) from the National Science Council, Taiwan.
References
[1] Su CY, Tsai KY, Chou FH, Ho WW, Liu R, Lin WK. A three-year follow-up study of the psychosocial predictors of delayed and unresolved post-traumatic stress disorder in Taiwan Chi-Chi earthquake survivors. Psychiatry Clin Neurosci 2010;64:239e48.
[2] Chen CS, Wu HY, Yang P, Yen CF. Psychological distress of nurses in Taiwan who worked during the outbreak of SARS. Psychiatr Serv 2005;56:76e9.
[3] Hsu CC, Chong MY, Yang P, Yen CF. Posttraumatic stress disorder among adolescent earthquake victims in Taiwan. J Am Acad Child Adolesc Psychiatry 2002;41:875e81.
[4] Yang P, Yen CF, Tang TC, Chen CS, Yang RC, Huang MS, et al. Posttraumatic stress disorder in adolescents after Typhoon Morakot-associated mudslides. J Anxiety Disord 2011;25:362e8. [5] Salmon K, Bryant RA. Posttraumatic stress disorder in chil-dren. The influence of developmental factors. Clin Psychol Rev 2002;22:163e88.
[6] Yule W. Posttraumatic stress disorder in the general pop-ulation and in children. J Clin Psychiatry 2001;62(Suppl 17): 23e8.
[7] Blake DD, Weathers FW, Nagy LM, Kaloupek DG, Gusman FD, Charney DS, et al. The development of a Clinician-Administered PTSD Scale. J Trauma Stress 1995;8:75e90.
[8] Blanchard EB, Jones-Alexander J, Buckley TC, Forneris CA. Psychometric properties of the PTSD Checklist (PCL). Behav Res Ther 1996;34:669e73.
[9] Foa EB, Johnson KM, Feeny NC, Treadwell KR. The child PTSD Symptom Scale: a preliminary examination of its psychometric properties. J Clin Child Psychol 2001;30:376e84.
[10] Horowitz M, Wilner N, Alvarez W. Impact of Event Scale: a measure of subjective stress. Psychosom Med 1979;41: 209e18.
[11] Weiss DS, Marmar CR. The Impact of Event Scale-Revised. In: Wilson JP, Keane TM, editors. Assessing psychological trauma and PTSD. New York: Guilford Press; 1996. p. 399e411. [12] Gillies ML, Barton J, Di Gallo A. Follow-up of young road
accident victims. J Trauma Stress 2003;16:523e6.
[13] Schafer I, Barkmann C, Riedesser P, Schulte-Markwort M. Posttraumatic syndromes in children and adolescents after road traffic accidentsea prospective cohort study. Psychopa-thology 2006;39:159e64.
[14] Kean EM, Kelsay K, Wamboldt F, Wamboldt MZ. Posttraumatic stress in adolescents with asthma and their parents. J Am Acad Child Adolesc Psychiatry 2006;45:78e86.
[15] Yonemoto T, Kamibeppu K, Ishii T, Iwata S, Hagiwara Y, Tatezaki S. Psychosocial outcomes in long-term survivors of high-grade osteosarcoma: a Japanese single-center experi-ence. Anticancer Res 2009;29:4287e90.
[16] Wu KK, Chan KS. The development of the Chinese version of Impact of Event ScaleeRevised (CIES-R). Soc Psychiatry Psy-chiatr Epidemiol 2003;38:94e8.
[17] Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59(Suppl 20):22e33. quiz 34e57. [18] Wang SJ, Juang KD, Fuh JL, Lu SR. Psychiatric comorbidity and suicide risk in adolescents with chronic daily headache. Neurology 2007;68:1468e73.
Table 2 Validity indices (%) of the Impact of Event Scale-Revised at different cutoff scores using the diagnosis of post-traumatic stress disorder as the criterion for “case” definition
Cutoff point Sensitivity (%) Specificity (%) PPV (%) NPV (%) OMR (%)
18/19 85.7 83.1 63.8 94.4 16.2
a19/20 85.7 84.1 65.2 94.4 15.5
20/21 82.9 86.6 68.2 93.5 14.4
21/22 78.6 87.6 68.8 92.1 14.7
NPVZ negative predictive value; OMR Z overall misclassification rate; PPV Z positive predictive value.
[19] Swets JA. ROC analysis applied to the evaluation of medical imaging techniques. Invest Radiol 1979;14:109e21.
[20] Weiss DS. The Impact of Event Scale-Revised. In: Wilson JP, Keane TM, editors. Assessing psychological trauma and PTSD: a practitioner’s handbook. New York: Guilford Press; 2004. p. 168e89.
[21] Balaban V. Psychological assessment of children in disasters and emergencies. Disasters 2006;30:178e98.
[22] Hoven CW, Duarte CS, Lucas CP, Wu P, Mandell DJ, Goodwin RD, et al. Psychopathology among New York city public school children 6 months after September 11. Arch Gen Psychiatry 2005;62:545e52.
[23] Giaconia RM, Reinherz HZ, Silverman AB, Pakiz B, Frost AK, Cohen E. Traumas and posttraumatic stress disorder in a community population of older adolescents. J Am Acad Child Adolesc Psychiatry 1995;34:1369e80.
