i EXECUTIVE SUMMARY OF THE FINAL REPORT ON THE THIRD EVALUATION OF THE ENHANCED NET SCHEME IN SECONDARY SCHOOLS 20181
Overview of the Evaluation and Outline of the Report
The evaluation of the Enhanced Native-speaking English Teacher (ENET) Scheme in Secondary Schools was commissioned by the Education Bureau (EDB). It commenced on 14 January 2016 and concluded on 31 December 2017. The evaluation was conducted by a team of scholars headed jointly by Dr Mary Shepard Wong and Dr Chong Ho Yu of Azusa Pacific University in collaboration with scholars from The Open University of Hong Kong, The Chinese University of Hong Kong, and New South Wales University.
The evaluation employed a sequential mixed-method approach using online surveys of key stakeholders followed by case studies of schools. 69 schools completed the surveys and they represented approximately 17% of the total number of secondary schools participating in the ENET Scheme in the 2015/16 school year. As to the case studies, eight schools took part.
Interviews and focus groups of a variety of stakeholders were conducted before, during, and after the surveys. The evaluation explored the impact of the Scheme on secondary students’ learning of English with regard to the English-speaking environment of the school, local teachers’ and NETs’
pedagogical practices, and the use of English in the school. The study collected data on stakeholders’ background and activities, as well as their views of the effectiveness of NET deployment, utilisation, integration, and support in schools, and the collaboration between NETs and their local colleagues. It also asked stakeholders about the support received from the Regional NET Coordination Team (RNCT) members and the Native-speaking English Teacher (NET) Section, the progress made on the key recommendations of the 2009 evaluation, and how the ENET Scheme might be improved.
This report includes six sections: 1) Objectives and Rationale; 2) Literature Review of Similar Programmes; 3) Findings and Key Observations of the Quantitative Data Analysis followed by the Qualitative Data Analysis; 4) Synthesis of the Quantitative and Qualitative Analysis; 5) Recommendations; and 6) Appendices.
In the literature review comparing the NET Schemes in Hong Kong with similar schemes in the region, including Japan, Korea and Taiwan, the research team pointed out that only Hong Kong had conducted and published external evaluations on its NET Schemes. In addition, Hong Kong’s NET Schemes had a more fully developed support system for professional and curriculum development.
Quantitative Data Collection Overview
Four questionnaire surveys were developed and piloted, and further refined after they were piloted before links to the online versions were released to participating schools and stakeholders. Table 1 provides the numbers of completed online surveys by each group of stakeholders who were sent
1 A Chinese version of this summary is also provided. In case there are discrepancies between the English and Chinese versions, it is the English version that should be referred to.
ii surveys. A total of 66 Principals, 65 NETs, 456 local teachers, and 6,523 students completed the online survey.
Table 1. Summary of number of surveys completed and included in data analysis
* Potential
# of respondents
# of schools involved
# of surveys submitted
# of completed surveys
# of incomplete surveys
# of invalid data to exclude
# of wrong codes to include
Total
SPQ 69 66 82 72 10 6 4 66
NETQ 69 65 76 68 8 3 2 65
LETQ 468 69 483 459 24 3 12 456
SQ 9,960
(estimate)
67 7,818 7,259 559 736 N/A 6523
10,566 7110
* SPQ = The School Principal Questionnaire
NETQ = The Native English Teacher Questionnaire LETQ = The Local English Teacher Questionnaire SQ = The Student Questionnaire
Qualitative Data Collection Overview
To enable the research team to gain a more in-depth understanding of the operation of the Scheme, in addition to the over 7,000 online surveys, 32 interviews and focus groups were conducted with the local English teachers, NETs, English Panel Chairs (EPCs) and School Principals in eight case study schools. Case studies also included 26 observations of the English environment of schools, co-planning meetings, and classrooms in which NETs were teaching and shadowing of the NET.
Observations were also conducted of a school-based workshop and a workshop by the EDB. In addition, nine non-school based interviews and focus groups were conducted with a variety of stakeholders, including EDB personnel, and groupings of various teachers, parents, students, and principals.
Overview of 57 Findings and Six Research Questions
A total of 57 findings were identified resulting in eight recommendations. There were 28 findings based on the statistical analysis of responses to the online surveys (preceded by “SF” for survey findings). These indicated that stakeholders generally held positive perceptions of the impact on the ENET scheme on students’ English learning, on the English-speaking environment of the school and on the use of pedagogical practices by English teachers. There were 29 findings based on the case studies (preceded by “CF” for case study findings) and non-school-based data analysis.
Findings indicated that the majority of all stakeholders supported the Scheme although several areas of possible improvement were noted.
The evaluation was guided by six research questions (RQs). Looking at both sets of findings together, the research team concluded that the ENET Scheme overall has a positive impact on English language teachers’ pedagogical practices (RQ1), that the Scheme helps to improve student learning of English (RQ2), helps to enhance the English-speaking environment of the school (RQ3), helps support curriculum development (RQ4), and promotes professional development and collaboration between NETs and local English teachers (RQ5). Since 2009, some progress has
iii been made towards addressing the recommendations of the University of Melbourne evaluation report, responding to curriculum initiatives and provided additional support for NETs in the schools (RQ6).
These conclusions as to the overall effectiveness of the ENET Scheme suggest that positive achievements are being made and that the Scheme is successful on the whole. At the same time, the evaluation has highlighted a number of factors affecting success and identified possible ways in which the Scheme might be modified in order to achieve greater success. Interpreting results in relation to the research questions helps to draw out these caveats to the overall success of the Scheme.
Research Question 1 investigated the extent to which the ENET Scheme impacts local teachers’
English language teaching and the ways it does so. The findings suggest that NETs adopt more culturally-focused and language-arts-oriented activities in their teaching than local teachers, and that this has an influence on the approaches local teachers adopt. The qualitative results throw some additional light on this issue. Although stakeholders agreed that the influence of the NET has a unique impact on the teaching approaches of local English teachers, and local English teachers valued the different perspective that the NET had brought to English language teaching, they themselves felt constrained in their ability to adopt similar practices in their own teaching (CF2).
Research Question 2 focused on the impact of the Scheme on student learning of English. Online survey and case study results identified perceptions of improvements in student attitude and motivation to engage with English, and increased opportunities for them to use English, although it was felt that not all students could benefit significantly from being taught by a NET, or from the NET Section competitions.
Research Question 3 focused on the extent to which the ENET Scheme enhances the English- speaking environment of the school, and the ways it does so. A number of findings both from the online surveys and from case study interviews and observations indicate that the Scheme enjoys considerable success in this area. SF6 showed that local English teachers and School Principals placed particular value on the NET’s role in organising and conducting co-curricular activities, which, they felt, enriched the English learning environment. In case study interviews, stakeholders confirmed the importance and the value of NET involvement in organising activities outside of lesson time, which, they agreed, helped enhance the English-speaking environment in the school (CF11).
Research Question 4 explored the extent to which the ENET Scheme supports curriculum development and implementation in secondary schools, and the ways it does so. Findings from the case study investigation involved one school which had applied for School-based Support Services from the NET Section, and found the support stimulating and worthwhile (CF16). Survey findings, reinforced by findings from case study interviews, suggested that NETs themselves tend not to get involved in activities which stakeholders would describe as ‘curriculum development’, although NETs often develop teaching and learning resources for sharing among panel members teaching the same levels (SF14).
iv Research Question 5 investigated the extent to which the ENET Scheme contributes to professional development and collaboration among the teachers, and the ways it does so. It is a two-pronged question, since professional development may not be related to collaboration.
Findings from the online surveys and case studies indicate that professional development in the form of centralised workshops and cluster meetings were found to be helpful (CF20) and that ideas and materials explored in them were practical and could be applied in the school context (SF18).
Research Question 6 is a three-pronged question with progress made on the key recommendations of the 2009 evaluation report, the impact of curricular initiatives since 2009 on the Scheme and the impact of changes in the support provided for NETs being explored. The 2009 evaluation report made a number of recommendations, some of which would entail major organisational changes in the teaching of English at the school level. The current evaluation found indications of the more practical of these recommendations having been acted upon. With respect to the impact of changes in the support provided for the NETs since 2009, the 3-day induction programme and RNCT support for new NETs are generally well-received, and the School-Based Support Services have been seen as valuable (SF27).
Additional Findings, though not directly related to the research questions, have provided further support for the effectiveness of the ENET Scheme. There was a perception that the number of suitable potential NETs was shrinking (CF25). Underlying this perception were a number of expectations as to the characteristics of a ‘good’ NET, which were revealing, as well as some misapprehensions as to the roles and responsibilities of the NET Section (CF27).
Summary of Recommendations
In light of the analysis of the data, the research team makes the following recommendations.
Recommendations are grouped according to the area they address, but not necessarily in order of importance. Note that the supporting findings are listed under each recommendation.
NET Deployment
1. Continue to encourage schools to integrate the NET into the English panel as a whole both socially and professionally. (Supported by SF1, 2, 3, 7, 9, 14, 15, 17, 26, 28 & CF3, 4, 5, 10, 14, 17, 18, 19, 23, 24)
2. Continue to encourage schools to consider the unique experience, qualifications, and interests of the NETs when determining deployment. (Supported by SF 1, 2, 3, 6, 7, 10, 11 & CF 1, 3, 5, 9, 11, 13)
Scheme’s Support for Schools
3. Continue to support schools with RNCT members by providing and evaluating the induction programme for new teachers recruited into the Scheme, professional development workshops in areas of interest, and providing assistance as requested with curriculum development and the improvement of pedagogy. (Supported by SF 8, 13, 18, 27 & CF 2, 6, 7, 13, 15, 16, 20, 21)
v 4. Continue to encourage and assist schools to promote a culture of collegiality and collaborative professional development. (Supported by SF 4, 5, 12 & CF 2, 4, 8, 12, 13, 17, 18, 19, 22, 23)
vi Public Relations
5. Continue to communicate the ENET Scheme’s goals to stakeholders and seek to remove misconceptions about the Scheme’s work. (Supported by SF 16, 17, 23, 24, 25, 26 & CF 26, 27, 28)
6. Continue to establish communication channels with NET communities and seek out their support to provide insight into NET issues and potential solutions. (Supported by SF 23, 24, 25, &
CF 26, 27, 28)
7. Continue to monitor and assess the response of the Education Bureau to the recommendations in this report as well as the effectiveness of the Scheme. (Supported by SF 23, 24, 25)
NET Recruitment
8. Consider how to support NETs and Schools in the selection and retention process in ways that provide transparency and flexibility. (Supported by CF 25, 26, 27, 28)
More detailed suggestions under each of the recommendations can be found in the report.
