3. Sampling weight: The sampling weight is the number for restoring the original importance of each unit within the population segment. In each stage of sampling there is a different weight
3.4 Quantitative Results and Key Findings
3.4.3 Research Question Three: To what extent and in what ways does the PNET Scheme help to improve and increase the local teachers’ use of English in the
Table 6
Chi-square analysis of student opinions on the effect of the PNET Scheme on use of English outside of school
Stakeholder
Weighted Count Percentage p valuea
Disagree Agree Disagree Agree
I do NOT have enough opportunities to talk to the NET outside of English lessons.
Key Stage 1 2593.69 1999.42 56.47 43.53 <.0001
Key Stage 2 2693.93 1551.53 63.45 36.55
I wish to have more opportunities to talk to the NET outside of English lessons.
Key Stage 1 664.01 3935.18 14.44 85.56 <.0001
Key Stage 2 934.50 3325.29 21.94 78.06
I do NOT talk to the NET outside of English lessons.
Key Stage 1 2728.40 1769.60 60.66 39.34 0.0430
Key Stage 2 2672.30 1582.96 62.80 37.20
I enjoy English activities outside of English lessons (e.g., recess activities, co-curricular activities, etc.)
Key Stage 1 603.57 3962.64 13.22 86.78 <.0001
Key Stage 2 1106.42 3130.65 26.11 73.89
I speak English at home.
Key Stage 1 1437.28 3136.90 31.42 68.58 <.0001
Key Stage 2 1881.75 2529.52 42.66 57.34
I read English books at home.
Key Stage 1 761.33 3885.51 16.38 83.62 <.0001
Key Stage 2 1239.66 3218.82 27.80 72.20
I speak English with my friends outside of class.
Key Stage 1 1876.86 2682.42 41.17 58.83 <.0001
Key Stage 2 2135.26 2229.82 48.92 51.08
I take extra English classes outside of school.
Key Stage 1 1803.65 2711.10 39.95 60.05 0.0671
Key Stage 2 1841.57 2559.67 41.84 58.16
aThe p value is yielded from the Fisher’s exact test. P values <0.05 are shown in bold.
3.4.3 Research Question Three: To what extent and in what ways does the PNET
Figure 7. Stakeholder opinions on the effect of the NET on LETs’ pedagogical practices.
Table 7
Chi-square analysis of the effect of the PNET Scheme on local English teachers Stakeholder
Weighted Count Percentage
Chi-square p valuea
Disagree Agree Disagree Agree
The NET creates innovation in my school.
LET 69.88 322.56 17.81 82.19 11.26 0.0036
NET 1.81 63.41 2.77 97.23
SH 12.61 41.40 23.35 76.65
The NET’s presence contributes to the LETs’ desire for professional development.
LET 72.14 310.61 18.85 81.15 3.87 0.1448
NET 16.18 41.70 27.95 72.05
SH 13.94 37.83 26.92 73.08
The NET’s presence contributes to the LETs’ increased use of English in their classes.
LET 27.13 370.66 6.82 93.18 4.71 0.0950
NET 5.55 57.63 8.79 91.21
SH 7.80 42.47 15.52 84.48
The NET’s presence contributes to the LETs’ expanded pedagogical practices.
LET 47.75 343.72 12.20 87.80 9.51 0.0086
NET 7.42 58.11 11.33 88.67
SH 14.45 38.19 27.45 72.55
The NET’s presence contributes to the LETs’ improved English proficiency.
LET 35.70 351.60 9.22 90.78 1.50 0.4735
NET 6.57 59.96 9.88 90.12
SH 7.67 44.97 14.57 85.43
aP values <0.05 are shown in bold.
SF8. NETs and LETs use a variety of pedagogical practices, but they believe that NETs do so more than LETs.
Regarding teachers’ use of different pedagogical practices, and consistent with expectations, there was a clear pattern showing more NETs than LETs using a variety of different practices (e.g. songs, games, group work, shared reading etc.) outside of traditional, teacher-fronted and textbook-based methods. Note that the teachers were asked how often they used the practices, and for ease of analysis ‘very often’ and ‘often’ were combined into one category which is labelled ‘yes’ in the figure, and ‘not often’ and ‘not at all’ were combined and labelled as ‘no’
as a shorthand, but does not mean ‘never’. As most NETs come from the UK, Canada, Australia, and the US, where the use of student-centred learning and teaching approaches is more widespread, this was not surprising. What is encouraging is that, except for “plays, drama/and or puppetry”, a majority of the LETs also report that they use these pedagogical practices very often or often. Figure 8 and Table 8 display findings related to the pedagogical practices used by NETs and LETs.
Figure 8. NET and LET responses to items related to pedagogical practices.
Table 8
Chi-square analysis of the frequency of NET and LET pedagogy Stakeholder
Weighted Count Percentage
p valuea Not often or Very often or Not often or Very often or
Do you use games in English?
LET 68.55 350.04 16.38 83.62 0.0015
NET 1.86 66.76 2.71 97.29
Do you use songs in English?
LET 136.59 281.01 32.71 67.29 <0.0001
NET 4.74 63.87 6.91 93.09
Do you match reading materials to student ability?
LET 44.26 374.34 10.57 89.43 0.8307
NET 5.91 62.71 8.61 91.37
Do you use supported reading of English materials?
LET 68.21 347.44 16.41 83.59 0.0266
NET 4.06 64.55 5.92 94.08
Do you use group work to encourage oral interaction?
LET 69.16 349.44 16.52 83.48 0.1068
NET 6.55 62.06 9.55 90.45
Do you use plays, drama, and/or puppetry activities?
LET 263.66 153.49 63.21 36.79 0.0005
NET 27.27 41.34 39.75 60.25
Do you use co-teaching with a LET to develop students’ literacy skills?
LET 66.14 351.80 15.83 84.17 0.0086
NET 3.09 65.53 4.50 95.50
Do you use differentiated tasks and activities to cater for learner diversity?
LET 66.27 350.22 15.91 84.09 0.0261
NET 4.07 64.55 5.93 94.07
Do you use classroom routines to facilitate learning activities?
LET 11.62 406.98 2.78 97.22 1.000
NET 1.10 67.52 1.60 98.40
Do you use different questioning techniques?
LET 13.96 404.63 3.34 96.66 1.000
NET 1.10 67.52 1.60 98.40
Do you give formative feedback to students?
LET 15.46 401.54 3.71 96.29 <0.0001
NET 11.65 56.97 16.98 83.02
Do you promote self and/or peer assessment?
LET 96.79 321.81 23.12 76.88 0.7593
NET 16.28 52.33 23.73 76.27
Do you use shared reading?
LET 57.84 359.51 13.86 86.14 0.0002
NET 0 68.62 0 100
Do you use guided reading?
LET 70.56 346.04 16.94 83.06 0.3755
NET 8.07 60.55 11.76 88.24
Do you use shared writing?
LET 103.11 314.65 24.68 75.32 0.6519
NET 15.33 53.28 22.35 77.65
Do you use process writing?
LET 118.67 299.32 28.39 71.61 0.3169
NET 23.93 44.69 34.87 65.13
Do you use modeling activities/tasks with a LET/NET?
LET 83.76 331.47 20.17 79.83 0.0035
NET 3.96 64.66 5.77 94.23
Do you share learning objectives with students?
LET 42.31 375.28 10.13 89.87 0.2921
NET 10.33 58.28 15.06 84.94
Do you set explicit expectations for learning?
LET 46.32 372.28 11.06 88.94 1.000
NET 6.80 61.82 9.91 90.09
NET 2.08 66.53 3.04 96.96 Do you give clear instructions?
LET 2.43 416.16 0.58 99.42 1.000
NET 0 68.62 0 100
Do you review lesson objectives?
LET 24.79 392.48 5.94 94.06 0.1775
NET 7.04 61.58 10.26 89.74
aThe p values are yielded from the Fisher’s exact test. P values <0.05 are shown in bold.
SF9. The majority of stakeholders believe the Scheme’s professional development activities enhance LETs’ pedagogical practices.
Analysis of relationships between perceived effectiveness of the PNET Scheme and various pedagogical practices
Generalised regression, instead of conventional OLS regression, was employed to investigate the relationship between the perception of effectiveness of the PNET Scheme and various pedagogical practices. There are 22 different pedagogical types, which were used as potential predictors of the outcome variable. With so many pedagogical practices, it needs to be pointed out that the validity of a conventional regression could be threatened by multi-collinearity when there are too many independent variables (correlations between multiple variables). To rectify this situation, the generalised regression approach tries out different models and at the end adopts the most optimal solution by penalising complexity and filtering out redundant variables.
In this analysis the effectiveness index is the average score of the answers to the following three questions:
The NET is used effectively at your school
The NET is supported effectively by your school
The NET is integrated into the school effectively
The responses to the above three questions are coded in this scheme: 1 = Strongly disagree, 2
= Disagree, 3 = Agree, 4 = Strongly agree, missing = N/A. The responses to the various pedagogical practices are coded as follows: 0 = Not at all, 1 = Not often, 2 = Often, 3 = Very often. Sampling weights were input into the modelling for adjusting oversampling and undersampling.
Table 9 shows that only “matching reading materials to student ability”, “co-teaching with the NET to develop students’ literacy skills” and “shared reading” were found to be significant at an alpha level of .05 in predicting perceptions of the Scheme’s effectiveness.
Table 9
Generalised regression results of both NETs and LETs
Variable b SE Wald X2 p
Games in English 0.02 0.05 0.10 0.7521
Songs in English -0.05 0.05 0.97 0.3259
Matching reading materials to student ability 0.16 0.06 6.06 0.0138
Supported reading of English materials -0.08 0.06 2.05 0.1518
Group work to encourage oral interaction 0.06 0.06 1.18 0.2773
Plays, drama and/or puppetry activities 0.09 0.05 2.83 0.0924
Co-teaching with the NET to develop students’ literacy skills 0.13 0.06 6.00 0.0143 Using differentiated tasks and activities to cater for learner diversity 0.01 0.06 0.03 0.8583 Classroom routines to facilitate learning activities -0.03 0.07 0.27 0.6051
Using different questioning techniques -0.03 0.05 0.21 0.6447
Giving formative feedback to students 0.07 0.06 1.39 0.2385
Promoting self and/or peer assessment 0.03 0.05 0.35 0.5525
Shared reading 0.13 0.06 4.60 0.0319
Guided reading -0.06 0.06 0.79 0.3741
Shared writing -0.02 0.05 0.16 0.6932
Process writing -0.02 0.05 0.24 0.6273
Modelling activities/tasks with the NET 0.00 0.06 0.00 0.9586
Sharing learning objectives with students 0.04 0.06 0.61 0.436
Setting explicit expectations for learning 0.08 0.08 0.92 0.3373
Providing appropriate scaffolding 0.08 0.07 1.57 0.2108
Giving clear instructions 0.00 0.08 0.00 0.9543
Reviewing lesson objectives 0.01 0.07 0.05 0.8305
Another generalised regression model was run with LETs only (n = 429). The outcomes are the same in the sense that the same variables are identified as significant predictors of perceived effectiveness. In the table when the slope (b), the standard error, and the chi-square are all zero and the p value is 1, it means that the data for those variables are insufficient for parameter estimation. In addition, generalised regression modelling for NETs only (n = 71) is not possible due to its small sample size.
Although statistical analysis did not reveal a significant relationship between most pedagogical practices and perceived effectiveness of the PNET Scheme, the three methods that are positively related to perceived effectiveness have practical implications for teachers. It is important to point out that in the perspective of statistical modelling, a parsimonious model with fewer significant predictors is superior to a complicated model. First, when there are too many significant predictors, it is very likely that the model is over-fitted and unstable. Second, if many things are important, then nothing is important. When calling for actionable items, it is more manageable to focus on two to three items than to work on 20 to 30 items.
Unsurprisingly, “matching reading materials to student ability” is crucial to effectiveness because no matter how sophisticated the teaching techniques are, students cannot learn when they are presented with material well above or well below a level of difficulty which can promote language acquisition. The importance of co-teaching indicates the success of the PNET Scheme because the very essence of the Scheme is enhancement of teaching by LET/NET collaboration. However, the finding that shared reading has more predictive power than guided reading is more difficult to explain.
Table 9.1
Generalised regression results of LETs only
Variable b SE Wald X2 p
Games in English 0.00 0.00 0.00 1
Songs in English -0.03 0.05 0.43 0.5115
Group work to encourage oral interaction 0.08 0.07 1.38 0.2394
Plays, drama and/or puppetry activities 0.10 0.06 3.28 0.0702
Co-teaching with the NET to develop students’ literacy skills 0.12 0.04 8.33 0.0039 Using differentiated tasks and activities to cater for learner diversity 0.02 0.07 0.07 0.7858 Classroom routines to facilitate learning activities -0.03 0.07 0.16 0.6929
Using different questioning techniques 0.00 0.00 0.00 1
Giving formative feedback to students 0.06 0.07 0.81 0.3682
Promoting self and/or peer assessment 0.00 0.00 0.00 1
Shared reading 0.16 0.07 5.64 0.0175
Guided reading -0.05 0.07 0.49 0.4848
Shared writing -0.03 0.06 0.21 0.6448
Process writing -0.04 0.05 0.51 0.474
Modelling activities/tasks with the NET 0.00 0.00 0.00 1
Sharing learning objectives with students 0.08 0.07 1.26 0.2612
Setting explicit expectations for learning 0.01 0.09 0.03 0.8655
Providing appropriate scaffolding 0.11 0.07 2.37 0.1234
Giving clear instructions 0.00 0.00 0.00 1
Reviewing lesson objectives 0.00 0.00 0.00 1
Other survey items related to Research Question 3 asked respondents for their views on professional development activities provided by NETs in the schools, as well as those provided by ATs at the school level and centrally. Results for these items are shown below in Figure 9 and Table 9.2. Both LETs and NETs felt that the AT’s professional support, the school-based workshops organised by the ATs, and the NET Section’s professional development seminars and workshops were effective. Although over 88% of both these groups felt these support measures were effective, more LETs (97.23%) than NETs (88.56%) felt the NET Section’s professional development seminars and workshops were effective.
Figure 9. Perceived effectiveness of professional development activities.
Table 9.2
Chi-square analysis of perceived effectiveness of professional development activities.
Stakeholder
Weighted Count Percentage
Chi-square p valuea Effective Not Effective Effective
Not Effective How effective is the ATs’ professional support?
LET 362.39 9.95 97.33 2.67 0.4217
NET 61.63 4.01 93.89 6.11
How effective are school-based workshops organised by the ATs?
LET 336.39 12.56 96.40 3.60 0.4213
NET 48.12 2.75 94.59 5.41
How effective are the NET Section’s professional development seminars or workshops?
LET 342.15 9.73 97.23 2.77 0.0021
NET 58.92 7.61 88.56 11.44
aIf Chi-square is missing, then the p value is yielded from the Fisher’s exact test. P values <0.05 are shown in bold.
3.4.4 Research Question Four: What factors determine effective NET