In order to control the quality and validity of the test items, a pilot study was conducted prior to the formal study. For the purpose of this study, children‟s comprehension of figurative expression lao was examined; thus, a pilot study was conducted to make sure the given instruction, test materials, and the tasks presented by the PowerPoint slide were fully comprehensible for all age groups of children.
The pilot study was composed of two comprehension tasks, the Word-in-Isolation task (WII) and the Word-in-Context task (WIC). Each task consisted of 20 questions, including 16 target items and 4 fillers (See Appendix C).
The subjects were recruited from one Elementary School in Taichung. A total of 15 students from G1 to G5 (3 people for each grade) were asked to participate in the study. They were given two trials prior to each formal task and asked to answer the questions. The two tasks were conducted one-on-one in an unoccupied classroom and the total time was around 20 minutes.
The results showed that the fifth graders (10 years old; Mean= 0.81 in the WII task, Mean = 0.88 in the WIC task) performed better on both tasks than the first graders (6 years old; Mean 0.44 in the WII task, Mean= 0.53 in the WIC task). The correct response rate for both tasks grew as the subjects‟ age increased. The results
53
also indicated that the subjects performed better on literal meanings than on metaphorical meanings (For more information, please refer to Table I in the Appendix D).
It was found that the subjects performed better on the WIC task (Mean=0.81) than the WII task (Mean= 0.68), indicating that the linguistic context is actually a useful aid in comprehending metaphorical expressions. In addition, the results showed that the mean score for lao with an animate NP (Mean= 0. 78) was higher than lao with an inanimate NP (Mean=0.71). Finally, it was found that Type 2 (i.e., affix) (Mean= 0.71) was more difficult than Type 1 (i.e., non-affix) (Mean= 0.77) for the subjects.
However, there were some inadequacies of the design of the pilot study. First, some test materials needed to be modified because it was found that the linguistic context of certain items was misleading (for example, Q8: lao jingcha in the Appendix C), and resulting in the third graders‟ inability to comprehend some items in the WIC task. Second, it was found that two of the second graders‟ significantly outperform other subjects, showing that average language ability should be taken into consideration in the subject recruitment in the formal study.
3.3.2 Formal Study
In the formal study, the research procedure was basically similar to that of the pilot study. Since the subjects consisted of young children, a consent form (See Appendix E) was given to their parents in advance. The consent form contained important information of the present study and what to expect if the children took part in the study. In addition, parents were assured that their children‟s personal
54
information is kept confidential. Once the consent forms were collected, the tasks were given to the subject.
First, clear instructions were given prior to the tasks. The subjects were introduced to meet the main character in the tasks, Brother Crab. The instruction was described as follows: Wo shi pangxie didi. Wo zhuzai haidi wangguo li, wo xiangqu ludi shang maoxian haiyou jiao pengyou. Danshi wo ke neng ting bu dong xin pengyou shou de hua. Ke yi bangbang wo ma? „I am Brother Crab and I live in Undersea Kingdom. I‟m going to go on an adventure to the land and make new friends there. But I may not understand what my new friend says. Can you kindly help me?‟ In addition, there were two trial questions to make sure the subjects know how to complete the tasks. The trial is illustrated in Table 3-5.
Table 3-5 An Example of a Trial Question The subject saw:
The subject heard:
Dongdong nao, shi shi kan
“Diandengpao” shi shenme yisi ne?
„What does “a light bulb” mean?‟
55
The subjects were asked to answer the questions orally, and the answers were recorded. After the trial, the interpretation task started. As mentioned before, the questions were presented with the pre-recorded voice by the experimenter. The subjects were asked to answer each question according to their intuitions. They had twenty seconds to answer each question. The Interpretation Task took them approximately 15 minutes to complete.
In the multiple-choice task, since the designed story and questions were the same and options were provided, subjects took approximately 10 minutes to complete.
3.3.3 Scoring
In this section, we briefly explain how answers were scored in the two tasks. In the interpretation task, the subjects‟ responses were rated according to their correctness and in the multiple-choice task, there was only one answer. If the subject chooses the correct answer to the question, he or she was given one point. If the subject does not choose any answer or choose the wrong answer, no point was given.
The analysis gave a clear picture of whether our children do not comprehend the questions or whether they interpret the target item literally. The data were scored by two raters. All disagreements in scoring were discussed and resolved so that 100%
agreement was attained. All the data were entered into the computer and processed by SPSS as soon as the tasks were completed.