3. Sampling weight: The sampling weight is the number for restoring the original importance of each unit within the population segment. In each stage of sampling there is a different weight
3.4 Quantitative Results and Key Findings
3.4.6 Research Question Six: What progress has been made on the key recommendations of the 2007 evaluation report and how might the PNET Scheme
Figure 13. Stakeholder opinions on the effectiveness of various aspects of NET-LET collaboration.
Table 13
Chi-square analysis of factors that foster and inhibit NET-LET collaboration (NLC) Stakeholder
Weighted Count Percentage
p valuea Effective Not Effective Effective Not Effective
How effective is your collaboration with the NET/LET?
LET 391.44 3.71 99.06 0.94 1.000
NET 66.53 0 100 0
How effective are the English panel meetings?
LET 384.93 9.41 97.61 2.39 <.0001
NET 54.60 10.13 84.35 15.65
aThe p value is yielded from the Fisher’s exact test. P values <0.05 are shown in bold.
3.4.6 Research Question Six: What progress has been made on the key
SF14. One of the changes noted in some schools that affected the Scheme was an increase in newly arrived students or cross-boundary students as indicated by 19.1% of the School Heads.
Figure 14
Has your school experienced a sharp rise in newly arrived students or cross-boundary students? Data are pictured in Figure 14.
Table 14
Chi-square analysis of School Heads’ responses to whether schools have experienced a sharp rise in newly arrived or cross-boundary students
Stakeholder
Weighted Count Percentage
No Yes No Yes
Has your school experienced a sharp rise in newly arrived students or cross-boundary students?
School Head 43.69 10.31 80.91 19.09
SF15. The majority of teachers attend English Panel Meetings 1-5 times per year, with over 10% of NETs and 5% of LETs indicating that they never attend.
Many of the recommendations discuss the importance of English Panel meetings (see recommendations 2, 3, 6, 8, and 9). Results for item 26 in the NET and LET surveys, which provides data on how often the NETs and LETs attend English panel meetings, are shown in Table 15 and Figure 15. Of interest here is that over 10% of NETs and over 5% of LETs selected ‘Never’ in response to this item. It is difficult to explain why local teachers would
respond in this way, since attendance at English Panel meetings would normally be obligatory.
As for NET non-attendance, likely explanations might be that English panel members preferred to discuss issues in Chinese, or that the NET’s role in the school did not include involvement in the mainstream curriculum, discussion of which would have taken up most of the meeting agenda. Another is that, especially in schools taking part in the Primary Literacy Programme – Reading and Writing (PLP-R/W), NETs contribute to curriculum planning by attending more focused meetings such as co-planning meetings and level meetings, instead of English panel meetings which may be more general and often also administrativein content.
Table 15
Chi-square analysis of English panel meeting attendance
Stakeholder
Weighted Count Percentage
Chi-square p value
Never
1-5 times a
year
6-12 times a
year
13+
times a year Never
1-5 times a
year
6-12 times a
year
13+
times a year
How often do you attend English panel meetings?
LET 22.06 322.61 46.98 17.27 5.40 78.89 11.49 4.22 6.745 0.0805 NET 6.94 55.91 2.88 0.80 10.44 84.03 4.33 1.20
SF16. The majority of all stakeholders want to expand the Scheme, but fewer LETs hold this view than other stakeholders.
Figure 15. Pie charts showing LET and NET self-reported attendance at English panel meetings
The final survey item asked five different stakeholders their views regarding the future of the Scheme. As shown in Figure 16 and Tables 16 and 16.1, there was an interesting response pattern regarding the perceived future of the PNET Scheme. The vast majority of parents, School Heads, NETs and ATs tended to favour expanding the current programme as opposed to continuing it as it currently is (Parents 85.07% vs.12.48%; SHs: 72.95% vs. 21.16%; NETs:
72.76% vs. 23.14%, and ATs: 70.83% vs. 29.17%). However, only 52.13% of LETs wanted to expand it and 39.81% wanted to keep it as is. The pie charts in Figure 16 show the different views on these issues.
Figure 16. Stakeholder opinions on the future of the PNET Scheme (1: Phase out the PNET Scheme, 2:
Scale down the Scheme, 3: Continue the Scheme as is, 4: Continue and expand the Scheme) Table 16
Chi-square analysis of stakeholders’ opinions on the future of the PNET Scheme Stakeholder
Weighted Count Percentage
Chi-square p value
1a 2b 3c 4d 1a 2b 3c 4d
AT 0 0 7 17 0 0 29.17 70.83 237.82 <.0001
LET 7.27 24.99 159.27 208.57 1.82 6.25 39.81 52.13
NET 0 2.72 15.40 48.41 0 4.09 23.14 72.76
Parent 21.08 29.95 260.46 1774.73 1.01 1.44 12.48 85.07
SH 1.85 1.33 11.43 39.40 3.43 2.46 21.16 72.95
aStatement 1: The PNET Scheme should be phased out.
bStatement 2: The PNET Scheme should be continued but scaled down.
cStatement 3: The PNET Scheme should be continued as it currently is.
dStatement 4: The PNET Scheme should be continued and expanded.
Non-parametric Wilcoxon Rank test yields a significant result (Chi-square = 201.39; p < .0001).
Table 16.1
Nonparametric Comparisons for Each Pair Using Wilcoxon Method
Level vs. Level Score Mean Difference Std. Err. Dif. Z p
Parent LET 400.442 28.2189 14.1906 <.0001
Parent AT 121.761 81.45151 1.4949 0.1349
Parent NET 89.666 50.39785 1.7792 0.0752
NET LET 56.789 15.76941 3.6012 0.0003
School Head LET 50.634 16.88831 2.9981 0.0027
NET AT 1.302 4.8134 0.2704 0.7868
School Head AT 0.602 4.29733 0.1401 0.8886
School Head NET -0.769 4.87897 -0.1577 0.8747
LET AT -47.126 22.94442 -2.0539 0.0400
School Head Parent -100.311 55.58467 -1.8047 0.0711
The Wilcoxon test is a non-parametric test for data that do not conform to parametric assumptions, such as normality and homogeneity of variance. Even though the original scale (1[strongly disagree], 4[strongly agree]) was used for analysis, this narrow scale does not result in data that adhere to the required assumptions. The non-parametric remedy is to ignore the parametric assumptions by examining the location of the scores. Indeed, this approach is more appropriate to this data type than the parametric test, because Likert-scale data is rank-ordered (ordinal) in essence. Specifically, each observation has a rank relative to other observations (higher, lower). In this sense, the so-called mean score in Wilcoxon is nothing more than the sum of the ranks of all observations in the whole sample. Next, the difference between the sums of ranks is computed and the probability of observing this shift of location based on the Z distribution is used to determine whether the difference is significant. For example, in the first row of Table 16.1 the group “parent” is compared against the group “LET”, and 400.442 is the difference between the sum of ranks of parents and that of LETs. Apparently it is a huge difference, and not surprisingly, the p value is extremely small (< .0001), leading to the conclusion that the null hypothesis (no difference between parents and LETs) is rejected. Rows 7 and 8 in Table 16.1 show the comparisons between school head and AT, and between school head and NET, respectively. The differences centre on zero (0.602, -0.769). Needless to say, the differences are not considered significant (p = 0.8886, p =.8747).
SF17. The majority of all stakeholders support the Scheme and would like more NETs if resources allow.
Figure 17 and Table 17 provide data on 13 statements. The three statements that show the most congruence among the three stakeholders (LETs, NETs, and School Heads) are that teachers support the Scheme, that NETs and LETs have a good relationship, and that if resources allow, they would like more NETs at their school. Almost all of them (96.99-100%) agree that NETs have contributed to student learning, although LETs and School Heads are not as willing to attribute student improvement in English directly to the PNET Scheme.
Figure 17. Stakeholder opinions on the NETs’ impact in schools.
Table 17
Chi-square analysis of stakeholder opinions on the NETs’ impact in the schools Stakeholder
Weighted Count Percentage
Chi-square p valuea
Disagree Agree Disagree Agree
If resources allow, I would like more NETs at this school.
LET 36.13 359.83 9.13 90.87 0.53 0.7672
NET 7.84 57.90 11.93 88.07
SH 4.87 49.14 9.01 90.99
Our NET makes a valuable contribution.
LET 22.96 371.52 5.82 94.18 0.0019
NET 0 66.53 0 100
SH 4.50 48.51 8.48 91.52
We follow the PNET deployment guidelines.
LET 6.82 370.46 1.81 98.19 1.000
NET 5.74 60.80 8.63 91.37
SH 0 54.01 0 100
Our NET is integrated well in our school.
LET 37.40 358.42 9.45 90.55 1.46 0.4827
NET 4.25 62.29 6.39 93.61
SH 2.78 49.02 5.37 94.63
Overall LETs don’t like working with the NET.
LET 283.54 109.30 72.18 27.82 10.66 0.0049
NET 58.01 8.53 87.19 12.81
SH 45.77 7.24 86.35 13.65
I understand the objectives of the PNET Scheme.
LET 12.64 372.65 3.28 96.72 1.000
NET 0 66.53 0 100
SH 0 54.01 0 100
NETs have contributed to student learning.
LET 11.99 386.40 3.01 96.99 0.6309
NET 0 66.53 0 100
SH 0.96 53.04 1.78 98.22
The AT helps the school to work with the NET.
NET 5.01 59.24 7.80 92.20
SH 1.75 50.29 3.37 96.63
The teachers support the PNET Scheme.
LET 15.31 376.52 3.91 96.09 <.0001
NET 4.07 62.47 6.12 93.88
SH 2.81 50.20 5.30 94.70
Students’ English has improved here in part due to the PNET Scheme.
LET 38.95 346.50 10.11 89.89 7.69 0.0214
NET 0 65.57 0 100
SH 5.94 42.66 12.23 87.77
The PNET Scheme has improved the English environment at our school.
LET 24.21 370.07 6.14 93.86 <.0001
NET 0.98 65.55 1.48 98.52
SH 2.93 49.99 5.53 94.47
Having a NET helps maintain enrolment.
LET 47.05 328.22 12.54 87.46 4.73 0.0939
NET 4.07 53.95 7.01 92.99
SH 10.62 40.32 20.84 79.16
The NET and LETs have a good relationship.
LET 14.32 379.08 3.64 96.36 0.0020
NET 2.73 63.81 4.10 95.90
SH 2.78 49.85 5.29 94.71
Note. If 20% of cells have expected count less than 5, then the Fisher’s exact test is used and no Chi-square is reported.
aIf Chi-square is missing, then the p value is yielded from the Fisher’s exact test. P values <0.05 are shown in bold.
SF18. Stakeholders hold different views regarding whether funds could be better used elsewhere.
As shown in Figure 18 and Table 18, with regard to whether the PNET funds might be better spent elsewhere, 43% of LETs agreed compared to only 2% of NETs. At first glance this is incompatible with the preceding finding shown in Figure 16 and Table 16, based on levels of agreement with Statements 1 to 4, that the majority of stakeholders, including LETs, supported continuation of the PNET Scheme. However, Statement 3 (‘The PNET Scheme should be continued as it currently is’) pertains to the PNET Scheme alone while the current statement (‘The funds might be better spent elsewhere’) does not identify where the respondents felt funding could be better spent. Since there was not a follow up question that asked for what
“elsewhere” might refer to, we cannot know what the respondents had in mind. A plausible explanation is that all stakeholders had invested so much effort and resources into the PNET Scheme that they tended to want to maintain the status quo. Even if the programme has some imperfections, they prefer improving the existing programme to cancelling it altogether. Follow up questions in the interviews regarding “where” stakeholders felt funds might be better spent is discussed in the qualitative section under CF39. In one case the reasoning was if the remuneration were somehow adjusted (two EPCs specifically mentioned reducing what they referred to as the ‘special allowances’ given to NETs), then more NETs could be hired. This is really more of a redistribution of funds within the PNET Scheme than a reduction.
Figure 18. Stakeholder opinions on use of PNET funds Table 18
Chi-square analysis of stakeholder opinions on use of PNET funds Stakeholder
Weighted Count Percentage
Chi-square p value
Disagree Agree Disagree Agree
PNET funds might be better spent elsewhere.
AT 19 4 82.61 17.39 46.21 <.0001
LET 198.19 155.54 56.03 43.97
NET 61.77 1.80 97.17 2.83
SH 24.19 24.88 49.29 50.71
It is noteworthy that the quantitative analysis did not yield any significant relationships between the parent demographics (e.g. the number of English books at home, father’s highest education qualification, etc.) and parents’ perception of the PNET Scheme.
4. QUALITATIVE DATA COLLECTION, ANALYSIS, AND RESULTS