Findings Addressing Research Questions

This section provides findings and supporting data from the quantitative analysis as they relate to the six research questions.

Research Question 1: To what extent and in what ways does the ENET Scheme impact teachers’ English language teaching (pedagogical practices)?

The research question was explored by two means: First, by investigating the use, by local English teachers and NETs, of a set of pedagogical practices deemed to be aligned with the envisaged role of the NET in the school context; second by asking respondents to rate the importance of designated NET roles and responsibilities.

The pedagogical practices investigated included activities recognised as being cultural in orientation. They included those characterised as ‘language arts related’, that is involving popular culture and literary elements, e.g. reader’s theatre, songs, poetry and creative writing;

activities providing more opportunities for meaningful interaction and self-expression through English including drama and debates, as well as games, use of multimodal texts and IT-based activities.

The impact of the Scheme would be considered significant if NETs modelled the use of such practices in their own teaching, and if local English teachers were influenced by this to adopt such practices in their teaching.

Results of online survey items related to these pedagogical practices suggest that, to a large extent, the Scheme was having an impact. First, NET self-reports indicate that they often made use of the pedagogical practices, or used them occasionally in their teaching – here ‘not often’

45 responses are interpreted to indicate occasional use. Second, although local English teachers used the practices significantly less frequently, when they did use them, they ascribed their use to the influence of the NET, or indicated that it resulted from collaboration with the NET.

The second means of exploring the research question was by inviting respondents to the online survey to rate the importance of designated NET roles and responsibilities. The roles and responsibilities of particular interest in relation to the research question are those which involved NETs engaging with local English teachers. All of these roles were rated as important by key stakeholders. They include the NET’s role in co-planning with English panel members, co-teaching with English panel members, introducing and implementing good teaching practices and contributing to school-based curriculum development. Again, the results suggest that to a large extent the Scheme was having a positive impact on local English teachers’

teaching since NET roles involving engagement and collaboration between NETs and locals were all rated highly.

SF4: In most cases, NETs stated that they used a greater variety of in-class activities than local English teachers. These included drama, films, songs, games, debates, poetry, short stories, creative writing, and reader’s theatre.

Local English teachers and NETs were asked to indicate the frequency with which they used a range of pedagogical practices including language arts-related and technology-oriented practices. The activities listed in this survey item were compiled in discussion with NET Section representatives. They are derived from the NET Deployment Guidelines and relate closely to the overall objectives of the Scheme, in particular the enhancement of the learning and teaching of English with linguistically and culturally authentic materials and resources.

The item stem is shown below followed by the tabulated responses from NETs and local English teachers in Table 13, which are then presented in the form of pie charts in Figures 17a to l.

Question [ACTJa-l]

[NET]/[LET]:

How often do you use the following in your lessons?

Table 13. Fisher’s exact tests/Chi-square analyses of teacher’s administration of in-class activities

Stakeholder Weighted Count Percentage

Chi-square

p value Never Not

often

Often Never Not often

Often Drama (a)

NET 7.87 29.84 27.29 12.11 45.90 41.99 32.003 .0001*

LET 116.46 275.52 63.53 25.57 60.49 13.95

Films (b)

46

NET 4.68 25.75 34.57 7.20 39.62 53.19 .0001*

LET 8.18 272.24 175.10 1.79 59.77 38.44

Songs (c)

NET 3.23 28.70 33.07 4.97 44.15 50.88 .0001*

LET 14.25 275.85 165.41 3.13 60.56 36.31

Games (d)

NET 2.91 18.98 43.11 4.47 29.21 66.3 .0001*

LET 10.02 222.80 222.70 2.20 48.91 48.11

Debates (e)

NET 13.91 24.08 27.01 21.40 37.04 41.56 29.509 .0001*

LET 155.91 235.00 64.61 34.23 51.59 14.18

Poetry (f)

NET 7.40 25.32 32.28 11.38 38.96 49.66 24.109 .0001*

LET 80.90 276.81 97.81 17.76 60.77 21.47

Short stories (g)

NET 8.25 13.87 42.88 12.69 21.34 65.97 18.337 .0001*

LET 17.19 200.41 237.90 3.77 44.00 52.23

Creative writing (h)

NET 10.20 15.30 39.50 15.69 23.53 60.78 18.407 .0001*

LET 24.23 214.49 216.80 5.31 47.09 47.59

Reader’s Theatre (i)

NET 23.53 24.03 17.44 36.20 36.97 26.83 20.802 .0001*

LET 194.28 223.53 37.71 42.65 49.07 8.28

Multimodal texts (j)

NET 8.00 29.44 27.56 12.31 45.29 42.41 6.577 0.0373*

LET 34.48 152.46 268.57 7.57 33.47 58.96

Apps/computer software (k)

NET 13.89 25.03 26.08 21.37 38.51 40.12 2.532 0.2820

LET 63.21 192.72 199.58 13.88 42.31 43.82

Learning Management Systems (l)

NET 41.11 17.05 6.84 63.25 26.23 10.53 5.409 0.0669

LET 264.04 169.22 22.25 57.97 37.15 4.88

47

a. * b. *

c. * d. *

e. * f. *

g. * h. *

48

i. j.

k. l.

Figures 17(a)-(l). Local English teachers’ and NETs’ use of in-class activities

Significant differences were found between the ratings that NETs and local English teachers gave indicating that NETs make greater use of a range of activities related to their anticipated role in promoting linguistically and culturally authentic uses of English and cultivating students’ interest in and understanding of the cultural assets of the language. However, given the importance of this aspect of the work of NETs in secondary schools, the percentages of positive responses that NETs gave to these survey items appear somewhat low. For example, there was less than 50% agreement among NETs that they often use drama (Figure 17a), debates (Figure 17e), poetry (Figure 17f) or readers’ theatre (Figure 17i) in their lessons.

Another way to look at the response pattern is that, when teachers responded to these survey items, it may not have occurred to them that these pedagogical practices represented a prominent part of their lessons. If this is a valid interpretation, the not often responses may be as significant as the often ones.

What is perhaps more surprising than the relatively low percentages of often responses is that a number of NETs indicated that they never use the activities. Percentages of NETs claiming to never use such activities range from 12% to 36%: drama 12%, debates 21%, short stories 12.69%, creative writing 15.69% and readers’ theatre 36%. This suggests that more may need to be done to achieve the Scheme objective of enriching the English language learning environment in schools by promoting the learning of English through the use of language arts resources and activities. The relationship between these response patterns and deployment modes is also worth exploring to determine whether NETs are being provided with sufficient opportunities to use these activities in their lessons (the wording of the survey item) as opposed to in co-curricular activities.

49 While acknowledging the general expectation that NETs should promote these language arts-related activities, and the observation that the low percentages of positive responses across several activities suggest that NETs were falling short in meeting such an expectation to the fullest, the results may be tempered by factors which may affect interpretation. Firstly, the wide variety of possible activities and resources available to an English teacher needs to be taken into account. When activities and resources are used too frequently, the novelty effect would be diminished. The novelty effect is the tendency for performance to improve when a new and interesting method is introduced. However, this effect could disappear when the method is overused. In the interest of variety, teachers would rotate the activities that they use, rather than using everything all the time, and thus some relatively low frequencies of use of individual activities should not be surprising.

Another factor which may help explain the results is the possible interpretations which could be placed on the survey items by stakeholders as they respond. Each of the language arts-related activities could comprise a complete course curriculum in its own right. So, a teacher who makes no claim to ‘often’ use drama, debates, poetry, short stories or readers’ theatre in their teaching, could nevertheless use drama techniques and debate elements (argument and rebuttal, for example, in discussions) in their teaching; they could draw attention to poetic devices in their handling of reading texts, they could ask students to act out a dialogue without actually perceiving themselves as ‘often’ using poetry or readers’ theatre in their lessons.

SF5: A majority of both local English teachers and NETs tended to agree that NET influence and/or their interactive collaboration contributed to the use of various in-class activities such as drama, films, songs, games, debates, poetry, short stories, creative writing, and readers’ theatre.

Local English teachers and NETs were asked to indicate the extent to which the influence of the NET and collaboration between NET and local English teachers contributed to their use of the pedagogical practices mentioned above.

The item stems are shown below followed by the tabulated responses from NETs and local English teachers in Table 14 which are then presented in the form of pie charts in Figures 18a to l.

Question [ACTKa-l]

[NET]: Please indicate to what extent you agree that your influence and/or your collaboration with local English teachers is contributing to the use of the following in your lessons.

[LET]: Please indicate to what extent you agree that the influence of the NET and/or your collaboration with the NET is contributing to the use of the following in your lessons.

50 Table 14. Chi-square analyses of NET's contribution to the use of activities

Stakeholder Weighted Count

Percentage Chi-square p value Disagree Agree Disagree Agree

Drama (a)

NET 18.29 32.55 35.98 64.02 2.479 0.1154

LET 88.18 257.61 25.50 74.50

Films (b)

NET 13.99 38.29 26.76 73.24 1.603 0.2055

LET 135.48 244.66 35.64 64.36

Songs (c)

NET 11.30 41.05 21.58 78.42 3.713 0.0540

LET 130.54 242.55 34.99 65.01

Games (d)

NET 17.02 36.96 31.52 68.48 0.042 0.8368

LET 118.74 275.07 30.15 69.85

Debates (e)

NET 19.08 28.59 40.03 59.97 0.180 0.6718

LET 130.04 170.32 43.29 56.71

Poetry (f)

NET 14.67 33.20 30.65 69.35 1.319 0.2508

LET 136.33 211.02 39.25 60.75

Short stories (g)

NET 14.64 37.13 28.28 71.72 1.170 0.2794

LET 136.96 244.26 35.93 64.07

Creative writing (h)

NET 18.55 28.63 39.31 60.69 0.102 0.7496

LET 135.68 231.73 36.93 63.07

Reader’s Theatre (i)

NET 12.90 20.36 38.79 61.21 0.234 0.6286

LET 114.52 150.65 43.19 56.81

Multimodal texts (j)

NET 27.34 19.63 58.20 41.80 0.625 0.4293

LET 172.78 159.15 52.05 47.95

Apps/computer software (k)

NET 21.13 18.36 53.52 46.48 0.309 0.5781

LET 176.11 126.67 58.17 41.83

Learning Management Systems (l)

NET 15.54 7.44 67.63 32.37 0.004 0.9495

LET 133.45 65.80 66.98 33.02

51

a.* b. *

c.* d. *

e. * f. *

g. * h. *

52

i. * j. *

k. l.

Figures 18(a)-(l). NET influence on the use of activities

In the responses to all the items related to the provision of language-arts-related activities (Figures 18a to i), the results suggest that both local English teachers and NETs agreed that the influence of the NET was instrumental in some way. There is a relatively high level of agreement between local English teachers and NETs in their responses to the items.

This provides evidence to suggest that NETs believe that they are having an influence on local English teachers’ perceptions of the importance of ‘linguistically and culturally authentic’

activities such as drama, songs, poetry, short stories as well as creative writing, debates and readers’ theatre, with some impact on local English teachers’ actual practice.

The results suggest that local English teachers who include language-arts-related activities in their teaching acknowledge the influence of the NET in their use of the activities. Even though, as discussed in relation to SF4 above, local English teachers claimed to use these activities significantly less frequently than NETs, when they do use them, they appear to have been influenced by the NET in doing so. Prior research indicated that there is always a discrepancy between intention and action no matter how good the idea is. One of the main reasons is lack of implementation intentions, meaning that when people must make conscious efforts to implement an action, it is very unlikely that they would do it as often as they expected. The solution is to automate the action so that it becomes a routine without devotion of extra efforts.

In this case, the evaluation team recommends that at the beginning these activities can be scheduled on a regular basis in order to build a momentum. After these tasks are automated and have a life on their own, the routine can be relaxed.

53 The response to these items may also suggest a possible change of attitude among local English teachers, which does not affect their actual practice. This is corroborated by findings from the qualitative analysis of case study data (see CF2) which suggest that local English teachers look to NETs for insights related to the culture of the language, an area they feel less well-equipped to teach.

SF6: NETs perceived all the duties they performed as important. Local English teachers and School Principals also perceived NET duties in general to be important, placing particular value on organising and conducting co-curricular activities and enriching the English learning environment.

Local English teachers, NETs and School Principals were asked to rate the duties performed by NETs from very important to not important. NET duties listed in the survey items (OPNAa-i) are mainly based on the information presented in the ‘Roles and Responsibilities of the School Stakeholders’ in the ENET Scheme, which can be found on the NET Scheme webpage⁷ and represents what the EDB expects NETs to perform for the successful implementation of the Scheme.

The item stem is shown below. This is followed by the tabulated responses in Table 15, which are then presented in the form of pie charts in Figures 19a to i.

Question [OPNAa-i]

[NET]/[LET]/[SP]:

Please rate the following duties of a NET:

Table 15. Fisher’s exact tests/Chi-square analyses of the importance of NET’s duties

Stakeholder Weighted Count

Percentage Chi-square p value

Less/Not important

More important

Less/Not important

More important Co-planning with English panel members (a)

NET 8.96 56.04 13.78 86.22 20.987 .0001*

LET 168.97 283.96 37.31 62.69

SP 12.24 53.76 18.55 81.45

Co-teaching with English panel members (b)

NET 29.28 35.72 45.04 54.96 0.146 0.9298

LET 205.99 246.94 45.48 54.52

SP 31.59 34.41 47.87 52.13

Undertaking assigned teaching duties (c)

NET 5.45 59.55 8.38 91.62 4.002 0.1352

7 http://www.edb.gov.hk/en/curriculum-development/resource-support/net/enet-stakeholders-roles-responsibilities.html

54

LET 75.46 377.47 16.66 83.34

SP 7.28 58.72 11.03 88.97

Contributing to school-based curriculum development (d)

NET 9.95 55.05 15.30 84.70 14.500 0.0007*

LET 175.16 277.77 38.67 61.33

SP 19.75 46.25 29.93 70.07

Contributing to school-based professional development activities (e)

NET 21.78 43.22 33.51 66.49 5.754 0.0563

LET 212.86 240.07 47.00 53.00

SP 24.54 41.46 37.18 62.82

Enriching the English learning environment in the school (f)

NET 0.74 64.26 1.14 98.86 .0001*

LET 56.03 396.90 12.37 87.63

SP 7.67 58.33 11.63 88.37

Organising and conducting co-curricular English activities (g)

NET 5.81 59.19 8.94 91.06 7.934 0.0189*

LET 88.64 364.29 19.57 80.43

SP 6.01 59.99 9.10 90.90

Advising the local English teachers on English usage and culture (h)

NET 8.20 56.80 12.62 87.38 4.230 0.1206

LET 105.45 347.48 23.28 76.72

SP 17.05 48.95 25.83 74.17

Introducing and implementing good teaching practices (i)

NET 11.13 53.87 17.12 82.88 4.559 0.1023

LET 133.73 319.20 29.52 70.48

SP 16.78 49.22 25.42 74.58

a.* b.

55

c. d.*

e. f.*

g.* h.

i.

Figures 19(a)-(i). Importance of NET’s duties

The general agreement shown by NETs, local English teachers and School Principals as to the importance of the duties assigned to NETs is noteworthy in the responses to these survey items.

A significant majority in each group of stakeholders agreed as to the importance of the NET co-planning with English panel members (Figure 19a), undertaking assigned teaching duties (Figure 19c), contributing to school-based curriculum development (Figure 19d), enriching the English learning environment in the school (Figure 19f), organising and conducting

co-56 curricular activities (Figure 19g) advising local English teachers on English usage and culture (Figure 19h), and introducing and implementing good teaching practices (Figure 19i).

The majority of these results align with the ratings assigned by the three stakeholder groups to survey items exploring the actual roles of the NETs (see the analysis of responses to Question ACTAa-n above). In responding to those items, both local English teachers and School Principals perceived the NET as often serving as language advisors to local English teachers (Figure 16a), organising co-curricular English activities (Figure 16e), using linguistically and culturally authentic resources (Figure 16l), and engaging in activities which enrich the English environment within the school (Figure 16m).

However, some of the differences in the levels of agreement between the three stakeholder groups are also worthy of comment. For example, in comparison to local English teachers, NETs and School Principals valued two NET duties more highly: co-planning with English panel members (Figure 19a), and organising and conducting co-curricular English activities (Figure 19g). NETs, on the other hand, rated enriching the English learning environment in their schools (Figure 19f) and contributing to school-based curriculum development (Figure 19d) more highly than did local English teachers and School Principals.

The survey item asks respondents to rate the importance of different duties of a NET, rather than the current NET. However, different ratings of the importance of the NET (i.e. the current NET) in enriching the English learning environment in particular schools also emerged in responses to the open-ended question, ‘What do you like most about the ENET Scheme and the NET’s work at your school?’ When responding to the question, one local English teacher wrote: ‘enriching the English learning environment (but not now in our school)’; another local English teacher wrote: ‘[the NET] can’t improve the English learning environment in our school. He is just a normal English teacher.’ Such responses reflect local English teachers’

belief in the importance of the role, and their ‘… disappointment at their current NET’s failure to perform the role effectively’. It may mean that these local English teachers are not seeing NETs providing results in enriching the English learning environment.

Research Question 2: To what extent and in what ways does the ENET Scheme improve student learning of English?

Positive findings related to this research question have been reported above under ‘Key Findings’. There the very high level of agreement among stakeholders that the ENET Scheme had contributed to student learning in the school was reported (see SF3 and Figure 15b above).

The extent to which the ENET Scheme improves student learning of English was further explored by three different items in the online survey.

First, local English teachers, English Panel Chairs and School Principals were invited to indicate the extent to which they agreed with a series of items related to the impact of the NET on student motivation, proficiency in the four skills and the frequency of their use of English.

Second, respondents were asked to rate the effectiveness of the NET Section competitions in enhancing student learning of English.

57 Third, improvement of student learning was viewed indirectly through the impact of different NET activities on students’ English proficiency, including collaboration between NETs and local English teachers; English Panel Meetings; peer lesson observation; professional development activities conducted by the NET, curriculum and/or lesson planning meetings, professional development seminars, workshops or cluster meetings organised by the NET Section and resources developed by the NET Section.

The results suggest that, to a large extent, the ENET Scheme helps to improve student learning of English. This was reflected in high levels of agreement among local English teachers and School Principals that the work of the NET contributes to student motivation, and their proficiency in reading, writing, speaking and listening. NET Section competitions were seen as effective in improving the learning and teaching of English in the school, though NETs rated them more highly than local English teachers. Views on the items which might be considered to have an indirect impact on student learning were mixed, but collaboration and co-planning meetings were generally agreed to have a positive impact.

SF7: Both local English teachers and School Principals acknowledged the contribution of NETs to student learning of English. This involved a wide spectrum of improvement in student motivation, proficiency, frequency in using English and also perceptions of the importance of English held by students and parents.

Local English teachers and School Principals were asked to indicate the extent of their agreement to ten statements related to NET impact on student learning of English. The item stem is shown below followed by the tabulated responses from School Principals and local English teachers in Table 16, which are then presented in the form of pie charts in Figures 20a to j.

Question [EFTAa-j]

[LET]/ [SP]:

To what extent do you agree that the NET contributes to students’…

Table 16. Chi-square analyses of improvement made in student learning of English by the ENET Scheme

Stakeholder Weighted Count

Percentage Chi-square p value

Disagree Agree Disagree Agree View of the importance of English (a)

LET 83.31 369.62 18.39 81.61 0.034 0.8529

SP 11.52 54.48 17.45 82.55

Motivation to learn English (b)

LET 84.78 368.15 18.72 81.28 0.014 0.9056

SP 12.76 53.24 19.33 80.67

Improved proficiency in writing (c)

LET 168.17 283.43 37.24 62.76 0.833 0.3615

SP 20.76 45.24 31.45 68.55

58 Improved proficiency in reading (d)

LET 159.75 292.56 35.32 64.68 0.046 0.8309

SP 22.42 43.58 33.98 66.02

Improved proficiency in speaking (e)

LET 21.07 431.86 4.65 95.35 1.716 0.1902

SP 5.59 60.41 8.46 91.54

Improved proficiency in listening (f)

LET 44.72 408.22 9.87 90.13 0.857 0.3545

SP 8.97 57.03 13.59 86.41

Use of English in class (g)

LET 50.10 402.83 11.06 88.94 0.016 0.9000

SP 6.96 59.04 10.54 89.46

Use of English at the school (h)

LET 88.94 363.39 19.66 80.34 1.041 0.3075

SP 16.55 49.45 25.08 74.92

Use of English outside of school (i)

LET 228.77 224.16 50.51 49.49 0.288 0.5918

SP 31.00 35.00 46.98 53.02

Parents’/ guardians’ valuing of English (j)

LET 205.48 245.62 45.55 54.45 0.200 0.6549

SP 32.00 34.00 48.49 51.51

a. b.

c. d.

59

e. f.

g. h.

i. j.

Figures 20(a)-(j). Improvement made in student learning of English by the ENET Scheme Although there are no significant differences between the perceptions of local English teachers and School Principals regarding the contributions of NETs to student’s learning of English, it is noteworthy that the percentages of agreement regarding improved proficiency in speaking and listening are higher than those for improvement in writing and reading. This may be a reflection of deployment patterns where NETs are assigned oral classes throughout the school, and hence have their greatest impact on listening and speaking skills, even though such oral skills teaching may be aimed at integrating skills. However, academic success is more dependent on reading ability and formal writing skills than on oral-aural competencies

60 (American Institutes for Research, 2013⁸; Hernandez, 2012⁹; OECD, 2012¹⁰). In future evaluations, it would be worth investigating the relationship between deployment modes and perceived impact on student learning of English in these different skill areas.

SF8: While the majority of NETs (64.87%) viewed the NET Section competitions as effective, a slight majority of local English teachers (55.64%) were sceptical about the effectiveness of the NET Section competitions in the improvement of English learning and teaching.

A key NET Section activity related to enhanced student learning of English through motivational activities is the organisation of competitions. Local English teachers and NETs were asked to indicate how effective they found the competitions in improving student learning of English.

The item stem is shown below followed by the tabulated responses from NETs and local English teachers in Table 17, which are then presented in the form of pie charts in Figure 21.

Question [ACTQ]

[NET]/[LET]:

Generally, how effective do you think the NET Section competitions are in improving the learning and teaching of English at your school?

8 American Institutes for Research (2013) Predictors of Postsecondary Success. November 2013, pp. 5‐9.

(http://www.ccrscenter.org/sites/default/files/CCRS%20Center_Predictors%20of%20Postsecondary%20Success _final_0.pdf)

9 Hernandez, D. ‘Double Jeopardy: How Third‐Grade Reading Skills and Poverty Influence High School Graduation.’ Annie E. Casey Foundation, 2012, p. 4.

(http://www.aecf.org/~/media/Pubs/Topics/Education/Other/DoubleJeopardyHowThirdGradeReadingSkillsandP o very/DoubleJeopardyReport030812forweb.pdf)

10 ‘Predicting success: Key characteristics of youth affecting transitions to education and the labour market’ In Pathways to Success: How knowledge and skills at age 15 shape future lives in Canada. OECD Publishing (http:/dx.doi.org/10.1787/9789264081925-6-en) (http://www.oecd-ilibrary.org/education/pathways-to- success/predicting-success-key-characteristics-of-youth-affecting-transitions-to-education-and-the-labour-market_9789264081925-6-en)

在文檔中 EVALUATION OF THE ENHANCED NATIVE-SPEAKING ENGLISH TEACHER SCHEME IN SECONDARY SCHOOLS IN HONG KONG 2016 (頁 54-98)