3.3.1 Pilot Study
In this section, the procedures of the pilot study are elaborated in detail. The pilot study was completed first to ensure the quality and validity of the task design which would be implemented later in the formal study. The pilot study was conducted in November of 2015 and experimental groups were recruited from Wan-Fang Public Elementary School in Taipei.
The participants in the pilot study ranged from Grade 1 to Grade 6.
To prevent the possibility of carry-over effects, all participants were asked to complete the Isolation task (see in Appendix A) first and then to continue to finish the Word-in-Context task (see in Appendix B). Before the first task began, full instructions were given to the participants and they were not allowed to discuss the answers with their classmates during the trials. Also, the instructor accompanied the participants while they were completing the trials. When the instructor noticed that the participants had difficulty understanding the descriptions of certain questions, she would ask the participants whether it was necessary to replay the slides again. During the experiments, the instructor made sure that all the participants in a single group had clearly watched and listened to the description in each slide before moving on to the next slide.
The findings of the pilot study can be summarized as follows: for the first research question concerning the performance in each phase, Phase 1 is composed of Grade 1 and Grade
2, while Grade 3 and Grade 4 belong to Phase 2. Grade 5 and Grade 6 then make up Phase 3.
In the pilot study, Phase 2 and Phase 3 outperformed Phase 1 in both two tasks. Phase 1 had the lowest mean score (Mean=0.45) in idiomatic interpretation. The mean percentage (Mean=0.70) of the Word-in-Isolation trial in Phase 2 was very close to the mean percentage (Mean=0.72) of the Word-in-Isolation trial in Phase 3. This outcome supports the conditions proposed by the GEM within Phase 2 and Phase 3 as the participants in both phases displayed a sensitivity towards idiomatic expressions. They had the ability to distinguish between an idiomatic expression's literal and metaphorical meaning.
In addition, signs of the age effect can be found through a comparison of performance between Phase 1 and Phase 3. Phase 3 (Mean in the Word-in-Isolation task= 0.71; Mean in the Word-in-Context task= 0.91) outperformed Phase 1 (Mean in the Word-in-Isolation trial= 0.45;
Mean in the Word-in-Context trial= 0.75) on both tasks. This indicated that there was indeed development evident in primary school children’s idiom interpretation. Based on the between-group comparison of the participants’ performance, these results supported the predicted
developmental phases in the GEM (Levorato & Cacciari 2002).
Regarding the familiarity of idioms, it can be seen that the mean percentage of familiar idioms was higher than unfamiliar idioms. For example, the mean score in the Grade 3 group regarding the familiar type was 0.88. However, the mean score in the unfamiliar type was 0.50.
The same phenomenon can be found in other groups other than the Grade 3 group. This result
supports claims made in prior literature that familiarity affects children’s idiom interpretation
(Nippold & Rudzinski 1993, Levorato & Cacciari 1992, Nippold & Taylor 1995, Nippold et al.
1996, Hsieh & Hsu 2010). The familiarity of idiomatic expressions is a crucial factor for the children. In the pilot study, the participants did indeed have difficulty in interpreting unfamiliar idioms. The most difficult type of idiom in the task for the participants were those that were of both the unfamiliar type and the quadra-syllabic type. The Phase 1 group in particular had the lowest mean score when encountering quadra-syllabic idioms.
For the final research question, comparing the mean percentages in the Word-in-Isolation task with those in the Word-in-Context task provides evidence for the hypothesis which postulates that context aids the elementary students in every grade. As can be seen, each group performed better on the Word-in-Context task. Furthermore, the Phase 1 group displayed the most improvement when moving from the Word-in-Isolation task to the Word-in-Context task (Mean in the Word-in-Isolation task= 0.45; Mean in the Word-in-Context task= 0.75). Out of all of the groups, the Phase 3 group displayed the best performance in the Word-in-Context task, with their percentage reaching over 0.90.
After analyzing the data from the pilot study, several inadequacies were found. First of all, the description of the options in the task misled participants’ interpretations of the idioms. For example, Mr. Krab was described as a mean boss running a hamburger shop in the storyline.
The correct answer for question 13 chou-xian-wei in the Word-in-Context task should be the
third option mei-you-li-mao-de-you-qian-ren, a rich man who is impolite. However, the subjects are affected by the description Mr. Krab’s personality so they prefer the first option
ai-sheng-qi-de-ren, ‘one who easily gets mad.’ In addition, more subjects for each of the six
groups in the pilot study should be recruited. This would yield more solid results explaining the performance of the participants since the gap of the mean percentage between Phase 2 and Phase 3 was not found to be obvious.
3.3.2 Scoring Policy
Both tasks in the present study are composed of interpretation questions. The purpose of these two tasks is to compare the interpretation of idiomatic expressions of students in each grade. In both tasks, the participants' responses are classified into four categories including the literal, metaphorical, related and other types by three raters. The answers of the filler questions are excluded in the analysis. The following Table 3-6 provides examples of representative answers for each type of interpretation:
Table 3-6 Four Types of Interpretations of Chou-Lian in the Formal Study
Type Interpretation
Literal “An odorous face”
Metaphorical “An unhappy face”
Related “An angry face”
Other “I don't know”
One response in each category is considered as one frequency count. The frequency count within the four different categories is calculated for further statistical analysis. In addition to this, the frequency count of each type of idiom in the experimental design represents the participants' interpretational preferences for chou-related idioms. Based on the frequency counts, we can study various factors that affect their interpretation, including the type of idiom, familiarity, context, and the age effect.