4. Results
4.3 Accuracy
types by seven tokens. However, the validity of type-token ratio was usually claimed to be affected by variation of productive length, hence the MSTTR was suggested by Richards and Malvern to calculate the average TTR for segments of forty words.
Participants’ narratives were first divided into segments of 40 words. Then, the TTR of each segment was calculated. After adding the TTR scores for each segment and dividing the total scores by the number of segments, we can come up with the MSTTR. The measure of lexical complexity employed in this study is in the form of the following equation, where 1～N are segments 1~N, and N is the total number of segments:
3.4.3 Accuracy Measures
To answer research question three, two accuracy measures were adopted on the grounds that they investigated two crucial features of language accuracy, including a general examination of oral accuracy (i.e. error-free clauses) and a specific focus on verb forms (i.e. correct verb forms), which was one focal point of the form-focused guided worksheet in this study.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
48
1. Clausal accuracy: the proportion of error-free clauses to the total number of clauses.
Error-free clauses were defined as clauses where no errors occurred with regard to syntax, morphology, lexical appropriateness or collocation.
2. Verb accuracy: the ratio of correctly used verbs to the total number of verbs was calculated. Errors pertaining to tense, aspect, modality, and subject-verb agreement were counted.
3.4.4 Interrater Reliability
An inter-rater agreement, calculated by means of percentage agreement, was performed to ensure interrater consistency in analyzing learners’ performances. From each group, 20% of the oral transcription in the main study was randomly selected to be analyzed by a second rater (Cohen, 1960). There were two raters in this study, one was the researcher and the other was a Canadian native English speaker majoring in linguistics. Both raters were familiar with the purpose and research design of this study. Based on Cohen’s conclusion, a percentage agreement of at least 80% was considered acceptable. The interrater consistency was 97% for the segmentation of the narrative data into T-units, and the interrater agreement was 93% for the calculation of the three measures.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
49
Chapter Four Results
This chapter focuses on the results of the quantitative analysis of the speech data. The descriptive statistics of the linguistic outcomes from the three planning conditions are compared first. Then, these data are submitted to one-way ANOVA to evaluate the effects of different planning conditions. The following content is organized as follows. Part one is related to research question one—what are the effects of guided planning and paired planning on the fluency of young EFL learners’
oral discourse during a narrative story-retelling task? Part two and three focus on research questions two and three respectively—what are the effects of guided
planning and paired planning on the complexity of young EFL learners’ oral discourse during a narrative story-retelling task? What are the effects of guided planning and paired planning on the accuracy of young EFL learners’ oral discourse? A more detailed report on these results is offered below.
4.1 Fluency
The first section examines the outcomes of the fluency aspect of participants’
oral performances through two kinds of fluency measures, one is rate A (i.e. number of syllables per minute) and the other is rate B (i.e. number of meaningful syllables).
‧
production under three planning conditions. Overall, the results were of similar patterns for both fluency measures. Individual guided planning (IGP) had the highest fluency mean for both rate A (109.28) and rate B (102.97), and individual unguided planning (IUP) had the lowest fluency mean scores, 96.25 and 91.79 for rate A and rate B respectively. As shown in Table 4.1, however, the one way ANOVA yielded no significant difference among the three planning groups for both fluency measures. It is concluded that neither guided worksheets nor paired planning can conspicuously help participants in performing tasks more fluently.Table 4.1 Means of Fluency Note. IUP=individual unguided planning group,
IGP=individual guided planning group, PGP=paired guided planning group SD=standard deviation, ns=no significant difference at p >.05
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
51
4.2 Complexity
The second section considers the results for complexity. Two measures were used to assess the complexity of participants’ oral narrative production: syntactical
complexity and lexical variety. Results of the complexity measures are presented in Table 4.2. In terms of both complexity measures, there was a similar ranking pattern for complexity scores among the three planning groups. With respect to syntactic complexity, the paired guided planning group (PGP) obtained the highest mean (1.42), the IGP group the intermediate (1.37), and the IUP group achieved the lowest mean (1.31) (PGP>IGP>IUP). With respect to lexical variety, the PGP group also obtained higher mean scores than the IGP group (0.72>0.70), with the IUP group having the lowest mean scores (0.69) (PGP>IGP>IUP). Nevertheless, the results of ANOVA shown in Table 4.2 suggested that the three planning groups were not noticeably different in using complex sentences and words to narrate the story.
‧
Table 4.2 Means of Complexity
Note. IUP=individual unguided planning group, IGP=individual guided planning
group, PGP=paired guided planning group
SD=standard deviation, ns=no significant difference at p >.05
4.3 Accuracy
Lastly, the effects on accuracy were evaluated via two measures: error-free clauses and error-free verbs. For both accuracy measures, there was a consistent ranking pattern for the effect of various planning conditions on participants’ oral accuracy, as shown in Table 4.3. Accuracy in the use of correct clauses and verbs was the highest in the PGP group (73.12% accurate clauses and 80.61% accurate verbs), while the IUP group tended to be the least accurate(45.69% accurate clauses and 58.77% accurate verbs). Further analysis of the ANOVA showed that the three groups yielded significant differences in clausal and verb accuracy (p= .000), demonstrating that the participants accomplished story-retelling tasks with manifestly different
‧
amount of accurate clauses and verbs under three kinds of planning conditions.
Table 4.3 Means of Accuracy (% correct) Means(SD)
45.69(17.69) 67.08(13.88) 73.12(13.11) 27.10 .000***
Correct Verbs
58.77(15.21) 76.93(13.10) 80.61(10.06) 24.51 .000***
Note. IUP=individual unguided planning group, IGP=individual guided planning group, PGP=paired guided planning group
SD=standard deviation, ns=no significant difference at p >.05, *** p <.001
A post hoc test was carried out for both accuracy measures (see Table 4.4). In both cases, the post hoc test indicated that the significant difference was between IUP and IGP groups (p= .000). In contrast, for both accuracy measures, the post hoc test failed to show substantial differences between the IGP and PGP groups. These data summarized above revealed that guided planning notably assisted participants in using accurate clauses and verbs to tell the story, while paired planning did not benefit participants to perform tasks with more grammatically accurate language.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
54
Table 4.4 Locations of Significance Related to Accuracy Locations of Significance
IUP-IGP IGP- PGP
Correct Clauses .000*** .211 Correct Verbs .000*** .480
* p < .05, ** p < .01, *** p <.001
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
55
Chapter Five Discussion
The aim of this study is to examine the effects of guided planning and paired planning on the fluency, complexity and accuracy of young learners’ oral production.
The main focus will be laid on the speech performances of individual guided planning (IGP) group and paired guided planning (PGP) group; the individual unguided
planning group (IUP) serves as a control group to be compared with the IGP group.
This chapter will address findings in the current study versus findings in the previous studies as well as possible explanations for the findings.
5.1 Fluency
The purpose of this section is to evaluate the effects of guided worksheets and paired planning on participants’ oral narratives from the language facet of fluency.
The findings reported here partly support those in former planning studies.
5.1.1 The Effects of Guided Planning on Oral Fluency
The results of this empirical study indicated that for both rate A and rate B, participants in the IGP group did not tell the story more fluently than those in the IUP group. Apparently, the use of guidance did not lead participants to speak more fluently.
Such results are inconsistent with those in the previous literature (Foster & Skehan,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
56
1996; Kawauchi, 2005; Mehnert, 1998; Sangarun, 2005; Wendel, 1997), which reported positive effects of guided planning on fluency. Possible explanations for this discrepancy are as follows.
The offer of a recorded version of the story in L1 might have minimized task difficulty and resulted in the inconspicuous effect of guidance on fluency. Foster and Skehan’s (1996) study noted that the effects of guidance on enhancing fluent
production might be more prominent for cognitively challenging tasks, such as the narrative tasks in this study, than for ―ready-encoded‖ tasks, such as personal information tasks. In the present study, although more fluent speech was assumed to be stimulated by narrative tasks, the provision of L1 story translation prior to the narrative task might have decreased the difficulty level of tasks. Consequently,
learners could easily perform tasks fluently without the need for guided planning. And the fluency difference between guided and unguided groups was reduced. In other words, without the aid of L1 translation, narrative tasks could be more challenging.
Therefore, the effects of guidance might be more notable in stimulating guided planners to narrate a story more fluently.
In addition, as Mochizuki and Ortega (2008) mentioned in their study, the unguided planners’ avoidance of certain tough content and their focus on rehearsal might contribute to the unnoticeable difference in fluency between guided and
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
57
unguided groups. A similar situation was also found in this study. The IUP
participants, with no guided handouts as linguistic resources to rely on, might have skipped the tough part of the story, selectively drafted what they were able to write, and produced the shortest output (by almost 15 words fewer on average than the guided planners’ output). Consequently, the IUP planners had more practicing time, concentrated on practicing the shorter story, and their fluency difference from guided speakers was narrowed.
Moreover, Ellis (2009) suggested that the direction of learners’ attention to a single aspect, such as grammatical structure, might result in the unnoticeable
influence of guided planning on productive fluency. Numerous studies also observed that, under the situation of guidance towards both form and meaning, the speech manifested significantly higher fluency than the speech under unguided planning (Crookes, 1989; Foster & Skehan, 1996; Mehnert, 1998; Sangarun, 2005 ; Wendel, 1997). Sangarun (2005) explored deeper into this issue by comparing the effects of various guided planning (i.e. a focus on form, on meaning, and on both meaning and form) on the quality of learners’ utterance. Sangarun found that the provision of a guided worksheet which aimed only at grammar is less effective than guided planning that is focused jointly on meaning and form. It was hypothesized that a balanced orientation of learners’ attention to both form and meaning could reduce learners’
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
58
cognitive burden to comprehend grammatical concepts and formulate ideas. Learners, with lighter processing load, could have more attentional resources to monitor the grammatical structures in speech production as well as narrate the story more fluently.
In this study, students only received form-focused guidance, which might alleviate participants’ processing load during grammar conceptualization. However, it was still effort-consuming for them to formulate the story content. Therefore, guided planning did not significantly lead to greater speech fluency. Correspondingly, such an
unremarkable impact of guidance on oral fluency was shown in other form-focused guided planning studies (Mochizuki & Ortega, 2008; Skehan & Foster, 1999).
5.1.2 The Effects of Paired Planning on Oral Fluency
The results of the statistical analysis revealed that there was not more fluent speech in paired planners’ story retelling. The results for fluency corroborate those found in Foster and Skehan’s (1999) and Tuan and Neomy’s (2007) studies. This finding is congruent with Levelt’s (1989) report that fluency cannot be considerably boosted unless learners can use the planning time efficiently. Such planning efficiency is highly related to an efficient cooperation through working with a fixed talking partner (Pinter, 2007) and sufficient warm-up preparation (Luoma, 2004). On the one hand, Pinter indicated the importance for young participants to repeatedly practice
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
59
tasks with the same partner, so that they could understand mutual needs more and assist each other more efficiently. In this study, it was probably laborious for participants to communicate efficiently with randomly assigned peers, leading to influent speech. On the other hand, Luoma reported that paired speaking test-takers required appropriate pre-task training to discuss with partners efficiently. The
implementation of only one familiarization task in this study might be insufficient to promote discussing efficiency. For these two reasons noted above, paired planners may take too much planning time on discussion at the expense of rehearsal time.
More specifically, it was necessary to acquaint young participants with the model of paired discussion task. Such judicious interactive training with a fixed interlocutor could enable young learners to discuss more efficiently, utilize more time for rehearsal, and thus tell a story more fluently.
Furthermore, paired planners have made consistent comments on paired planning.
Many of the PGP planners (67%) disapprove the valuable role of paired planning in oral fluency, while a small portion (33%) expressed high opinions. Among the
negative comments on the effects of paired planning on fluency, some paired planners felt that individual planning was more efficient than paired planning. Others noted that they did not have the chance to rehearse since their speaking partners kept interrupting. Therefore, a less fluent articulation was produced as a consequence of
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
60
candidates’ spending too much time on discussion instead of rehearsal. Such results are also in tune with Chiu’s (2008) study where the majority of participants showed preference for self-preparation over paired discussion due to the inefficiency of paired planning.
5.2 Complexity
We turn next to examine the impact of the guided and paired variable on productive complexity. Overall, only half of the results obtained in this study fit into findings in earlier research, as illustrated below.
5.2.1 The Effects of Guided Planning on Oral Complexity
According to the quantitative complexity measures, there were no noticeable differences in paired planners’ use of complex sentences and words for story narration.
Whereas previous studies of strategic planning obtained conflicting findings, the current results for complexity verify those found by Mehnert (1998), Tajima (2003), Elder and Iwashita (2005) and Gilabert (2007) that strategic planning had no
significant effects on stimulating discourse complexity.
First, this finding is compatible with Elder and Iwashita’s (2005) argument that narrative tasks might not be challenging enough to induce participants’ complexity performances since the participants would pay most attention to organizing the story
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
61
content instead of taking the risk of using greater language variety. The argument noted above might explain the null effects of guidance on complexity in the present study. The tasks in this study were composed of a series of pictures, with the tellers having the pictures in order and the audience having unscrambled ones. The
sequencing nature of the task might persuade participants to focus on conveying main ideas in an organized and understandable way, so that the audience could reorder the pictures more easily. Participants’ reflection from the questionnaire mirrored the suggestion mentioned above that they thought their ability to address the audience was improved after attending this story-retelling activity. That is, the participants’
main focus was on the audience to whom they endeavored to convey the meaning completely and clearly. In short, participants maintained concentration on the story content when accomplishing narrative tasks. As a result, the oral output of
sophisticated language was hindered.
Also, this finding confirmed Foster and Skehan’s (1996) conclusions that the L1 story translation might have weakened the effects of guidance on language complexity.
Foster and Skehan reported that complicated and difficult tasks would enhance the impact of strategic planning. According to their research, the most linguistically complex production shown as a function of guidance was for the narrative task, while the least complex discourse was for the easiest personal task. In this empirical study,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
62
although narrative tasks were supposed to promote the function of strategic planning in this study, the offer of L1 story translation in precedent of the task might decrease the task difficulty as well as weakening the effects of guidance on productive
complexity. Such finding of the insignificant influence of guidance on output complexity is in complete agreement with Mochizuki and Ortega’s (2008)
investigation, in which the L1 story content was delivered to participants prior to the task.
Furthermore, this finding is congruent with Skehan’s (1996) argument of memory limitation. He noted that the human psychological space is limited.
Accordingly, when they focus on one aspect of performance, other aspects are paid less attention to. In other words, various dimensions of the production compete for finite mental resources. In the current study, significant outcome was gained for guidance on accuracy, while unnoticeable difference was found in complexity. It appears that it was difficult for participants to give consideration to both complexity and accuracy at the same time. Such results also echo Crookes’ (1989) finding that whereas strategic planning promoted learners’ oral accuracy, there was weak improvement in complexity. To sum up, if the impact of guidance on accuracy increased, its’ effects on complexity would decrease, and vice versa.
Finally, the inconspicuous influence of guidance on complexity may be related to
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
63
participants’ low proficiency. Several studies (Ortega, 1999; Mochizuki & Ortega, 2008) suggested that if learners’ proficiency was only at an incipient stage, they may not be equipped with the ability to complexify their speech. That is to say, a
proficiency ceiling effect has restrained learners’ performance from becoming more linguistically complex. In this study, despite the high-beginning proficiency of
participants, they were still of low proficiency, compared with those in other strategic planning research, such as Kawauchi’s (2005) intermediate senior high and
Sangarun’s (2005) intermediate to advanced attendants.Accordingly, even if given planning time, participants still did not know how to conduct the tasks with more complex utterances. In this study, participants’ output transcription exhibited their initial level of proficiency. From the speech data, the researcher found that
participants preferred to use syntactically simple sentences (e.g., He was so surprised.) than complex ones (e.g., He was so surprised to see his mother that he started crying.) in story-retelling. To conclude, for low proficiency students, it could be an arduous task to narrate a story with complex sentences during a short period of time.
5.2.2 The Effects of Paired Planning on Oral Complexity
From the results of the quantitative analysis, there were evidently no differences between PGP and IGP groups. It may reasonably be suggested that the paired element
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
64
did not aid participants in performing richer and more variant language. This result does not support the findings in previous research (Brooks, 2009; Chiu, 2008; Taylor, 2000). In trying to account for such discrepancy, several potential reasons are offered below.
First, the lack of task repetition (i.e. exercising the same kind of task repeatedly) might impede learners from communicating efficiently and making modifications to
First, the lack of task repetition (i.e. exercising the same kind of task repeatedly) might impede learners from communicating efficiently and making modifications to