Final Report on Phase (II) Study on Evaluating the Effectiveness of the ‘Empowering Learning and Teaching with Information Technology’ Strategy (2004/2007) Nancy LAW W.Y., Allan YUEN H.K., Mark SHUM S.K., Y. LEE

CITE, Faculty of Education, The University of Hong Kong, Pokfulam Road, Hong Kong.

Tel: (852) 2857-8540 Fax: (852) 2517-7194 Website: http://www.cite.hku.hk E-mail: cite@hkucc.hku.hk

Final Report on Phase (II) Study on Evaluating the Effectiveness of the

‘Empowering Learning and Teaching with Information Technology’

Strategy (2004/2007)

Nancy LAW W.Y., Allan YUEN H.K., Mark SHUM S.K., Y. LEE

Centre for Information Technology in Education (CITE) Faculty of Education

The University of Hong Kong

20

December 2007

Research Team

The research team consists of the followings:

Principal Investigator

-

Professor & Head, Division of Information & Technology Studies Director, Centre for Information Technology in Education

Faculty of Education

The University of Hong Kong

Investigator (s)

-

Associate Dean, Learning and Teaching

Associate Professor, Division of Information & Technology Studies Deputy Director, Centre for Information Technology in Education Faculty of Education

The University of Hong Kong

-

Associate Professor, Division of Language & Literature Faculty of Education

The University of Hong Kong

-

Assistant Director, Centre for Information Technology in Education Faculty of Education

The University of Hong Kong

Project Co-ordinator

-

Assistant Director, Centre for Information Technology in Education Faculty of Education

The University of Hong Kong

Project Team Members

-

Computer Officer, Centre for Information Technology in Education Faculty of Education

The University of Hong Kong

-

Education Development Officer (ICT), Centre for Information Technology in Education Faculty of Education

The University of Hong Kong

Table of Contents

List of symbols

Executive Summary

1 Purpose of the Study ...II 2 Methodology ...II 3 Summary of Findings... III 3.1 Students’ Achievements in Information Literacy... III 3.2 Relationship between Students’ Information Literacy Competences in Specific Key

Learning Areas and their Technical Proficiency... V 3.3 Relationship between Students’ Information Literacy Competences in Different Key

Learning Areas ... V 3.4 Interaction Effect of Any Two Dimensions of Technical Proficiency on Information

Literacy Competences in Specific Key Learning Areas ... V 3.5 Students’ Competences in Information Literacy and their Background Factors ... V 3.6 Students’ Competences in Information Literacy and School Level Factors ... VII 3.7 Findings of Other Questionnaires... VIII 4 Conclusion ... IX 5 Major Recommendations ... IX 5.1 Ensuring Baseline Technology Access in Schools...IX 5.2 Empowering Learners with IT ... X 5.3 Empowering Teachers with IT... X 5.4 Enhancing School Leadership for the Knowledge Age... X 5.5 Enriching Digital Resources for Learning...XI 5.6 Improving IT Infrastructure and Pioneering Pedagogy using IT ...XI 5.7 Providing Continuous Research and Development ...XI 5.8 Promoting Community-wide Support and Community Building ...XI

Chapter 1 Background of the Study

1.1 Introduction ... 1

1.2 Study Objectives ... 1

1.3 Research Questions ... 2

1.4 Linkage with Phase (I) Study... 3

Chapter 2 Conceptual Framework and Methodology

2.1 Conceptual Framework ... 4

2.2 Defining Information Literacy ... 6

2.3 Instrumentation ... 6

2.3.1 Developing Indicators for Evaluating Information Literacy ... 6

2.3.2 Developing an Online Assessment Platform ... 7

2.3.3 Developing Online Performance Assessment Tasks... 8

2.3.4 The Survey Component ...11

2.4 Administration ... 14

2.4.1 Pre-pilot Study... 14

2.4.2 Pilot Study ... 14

2.4.3 Main Study ... 15

2.5 Data Analysis Method ... 15

2.5.1 Workflow of Marking of Performance Assessments... 15

2.5.2 Analysis of Performance Assessments and Questionnaires ... 16

Chapter 3 Sampling and Response Rates

3.2 Sampling Procedures... 18

3.2.1 Sampling of Schools ... 18

3.2.2 Sampling of Classes ... 20

3.2.3 Sampling of Students ... 20

3.3 Sampling Weights ... 21

3.3.1 School Weight ... 22

3.3.2 Class Weight ... 22

3.3.3 Student Weight ... 22

3.3.4 Overall Sampling Weight ... 23

3.4 Response Rates ... 23

3.4.1 Response Rates at the School Level ... 23

3.4.2 Response Rates at the Student Level ... 23

3.4.3 Performance Assessment Scripts Collected ... 24

3.4.4 Number of Questionnaires Collected ... 25

3.5 Inter-coder Reliability ... 25

3.6 Difficulties Encountered and Actions Taken... 26

3.6.1 Response Rate ... 26

3.6.2 Class Time Allocation for Conducting Performance Assessments... 26

3.6.3 Project Timeline ... 26

3.6.4 School Readiness... 26

3.6.5 Loading on the Terminal Server... 27

Chapter 4 Field Observations on Performance Assessments

4.1.1 Number of Computers ... 28

4.1.2 Hardware and Software Updating ... 28

4.1.3 School Network and Standard in School Network Setting ... 28

4.2 Problems Related to Technical Support in Schools ... 29

4.3 Problem Related to Third Party System Integrator Maintaining School Network ... 29

4.4 Problems Related to Technical Skills of Students... 29

4.5 Problems Related to Competency in Typing Chinese Characters... 30

4.6 Availability of Peripherals ... 30

Chapter 5 Findings on Technical Performance Assessment

5.2 Task Completion ... 33

5.3 Students’ Overall Performance in Information Literacy of Technical Performance Assessment ... 34

5.4 Students’ Performance at Item Level ... 38

5.4.1 An Overview ... 38

5.4.2 Students’ Responses for Each Item... 40

5.5 Students’ Performance across Schools / Levels ... 69

5.5.1 Primary School Students’ Performance across Schools ... 69

5.5.2 Secondary School Students’ Performance across Schools ... 70

5.5.3 Comparing Students’ Performance Between the Primary and Secondary Levels ... 71

5.6 Comparing the Difficulty Levels of the Seven Dimensions of Information Literacy in Technical Performance Assessment ... 73

5.7 Summary ... 76

5.7.1 Students’ Information Literacy Competences in Technical Performance Assessment ... 76

5.8 Recommendations ... 77

5.8.1 Skills of Communication and Creativity ... 77

5.8.2 Skills of Generalization and Interpretation... 77

Chapter 6 Findings on Chinese Language Performance Assessment

6.1 Description of the Assessment Tasks ... 78

6.1.1 Primary 5 Chinese Language Performance Assessment ... 78

6.1.2 Secondary 2 Chinese Language Performance Assessment... 81

6.2 Task Completion ... 85

6.2.1 Primary Schools ... 85

6.2.2 Secondary and Special Schools... 85

6.3 Students’ Overall Performance in Information Literacy of Chinese Language Performance Assessment... 87

6.3.1 Primary Schools ... 87

6.3.2 Secondary Schools and Special Schools ... 88

6.4 Students’ Performance at Item Level ... 90

6.4.1 An Overview ... 90

6.4.2 Students’ Responses for Each Item... 93

6.5 Students’ Performance across Schools / Levels ... 154

6.5.1 Primary Schools ... 154

6.5.2 Secondary Schools ... 156

6.6 Comparing the Difficulty Levels of the Seven Dimensions of Information Literacy in Chinese Language Performance Assessment ... 158

6.6.1 Primary Schools ... 158

6.6.2 Secondary Schools ... 159

6.6.3 Special Schools ... 160

6.7 Summary ... 161

6.7.1 Students’ Information Literacy Competences in Chinese Language Performance Assessment ... 161

6.7.2 Management of Internet Materials ... 162

6.7.3 On-line Communication Skills ... 162

6.7.4 Application of Software... 163

6.8 Recommendations ... 163

6.8.1 Designing Descriptors to Indicate Levels of Information Literacy across Chinese Language Curriculum ... 163

6.8.2 Enhancing Students’ Ability to Manage Information ... 164

6.8.3 Enhancing Students’ Language Consciousness ... 164

Chapter 7 Findings on Mathematics Performance Assessment

7.2 Task Completion ... 166

7.3 Students’ Overall Performance in Information Literacy of Mathematics Performance

Assessment... 167

7.4 Students’ Performance at Item Level ... 168

7.4.1 An overview... 168

7.4.2 Students’ responses for each item... 171

7.5 Students’ Performance across Primary Schools ... 189

7.6 Comparing the Difficulty Levels of the Seven Dimensions of Information Literacy in Mathematics Performance Assessment ... 191

7.7 Summary ... 192

7.7.1 Task Completion Rates ... 192

7.7.2 Performance in Key Tasks... 193

7.7.3 Performance in Individual Information Literacy Dimensions ... 193

7.8 Recommendations ... 194

Chapter 8 Findings on Science Performance Assessment

8.2 Task Completion ... 196

8.3 Students’ Overall Performance in Information Literacy of Science Performance Assessment 198 8.3.1 Secondary Schools ... 198

8.3.2 Special Schools ... 199

8.4 Students’ Performance at Item Level ... 200

8.4.1 An Overview ... 200

8.4.2 Students’ Responses for Each Item... 203

8.5 Students’ Performance across Secondary Schools ... 235

8.6 Comparing the Difficulty Levels of the Seven Dimensions of Information Literacy in Science Performance Assessment... 236

8.7 Summary ... 239

8.7.1 Students’ Information Literacy Competences in Science Performance Assessment ... 239

8.7.2 Summary of Findings on Science-specific Knowledge ... 240

8.8 Recommendations ... 240

8.8.1 Enhancing Students’ Information Literacy Proficiency ... 240

8.8.2 Discussion Approach in Learning and using Open-ended Questions in Assessment .. 240

8.8.3 Designing Descriptors to Indicate Levels of Information Literacy across Science Curriculum ... 240

Chapter 9 Findings on Questionnaires and Further Analysis on the Performance Assessments

9.1 General Findings of School Head Questionnaire... 241

9.1.1 Curriculum Goals ... 241

9.1.2 Resource Allocation ... 245

9.1.3 Method of Assessment ... 247

9.1.4 Requirement of Teachers' Knowledge and Skills ... 248

9.1.5 Competence for School Leadership Team... 251

9.2 General Findings of Teacher Questionnaire... 252

9.2.1 Types of Classroom Activities and Use of ICT ... 253

9.2.2 Types of Pedagogical Practices and Use of ICT... 256

9.2.3 Assessments and Use of ICT ... 259

9.2.4 Students’ Practices and Use of ICT... 261

9.2.5 Impact of ICT Use ... 263

9.2.6 Teachers’ Self-proclaimed Competences in Uses of ICT... 264

9.2.7 Obstacles in Using ICT ... 266

9.3 General findings of IT Coordinator Questionnaire ... 269

9.3.1 Availability of Technology-related Resources ... 269

9.3.2 Number of Computers for Different Purposes in Schools... 272

9.3.3 Number of Laptops in Schools ... 273

9.3.4 Quantity of Different Technological Equipment in Schools ... 273

9.3.5 Availability of Technical Support in Schools... 274

9.4 General Findings of Student Questionnaire ... 278

9.4.1 Years of Computer Use... 278

9.4.2 Access to Computer at Home ... 279

9.4.3 Duration of Daily Computer Use at Home ... 279

9.4.4 Purposes of Using Computer ... 280

9.4.5 Self-proclaimed ICT Competences... 281

9.4.6 Sources of Help when Encountering Difficulties ... 283

9.4.7 General Impact on ICT Use ... 284

9.4.8 Impact on Subject-Specific Content ... 285

9.5 Correlation Analyses of 8 Information Literacy Indicators in Different Key Learning Areas 288 9.5.1 Correlation Analysis of the 8 Information Literacy Indicators in Primary Chinese Language... 288

9.5.2 Correlation Analysis of the 8 Information Literacy Indicators in Primary Mathematics ... 289

9.5.3 Correlation Analysis of the 8 Information Literacy Indicators in Primary Chinese

Language and Primary Mathematics... 290

9.5.4 Correlation Analysis of the 8 Information Literacy Indicators in Chinese Language of Secondary Schools ... 290

9.5.5 Correlation Analysis of the 8 Information Literacy Indicators in Science of Secondary Schools ... 292

9.5.6 Correlation Analysis of 8 Information Literacy Indicators in Science and Chinese Language of Secondary Schools ... 292

9.6 Further Analyses of 8 Information Literacy Indicators in Specific Key Learning Areas and Technical Performance Assessment ... 293

9.6.1 Correlation Analysis of 8 Information Literacy Indicators in Technical Performance Assessment ... 293

9.6.2 Further Analyses of 8 Information Literacy Indicators in Technical and Chinese Language Performance Assessments ... 295

9.6.3 Further Analyses of 8 Information Literacy Indicators in Technical and Mathematics Performance Assessments ... 299

9.6.4 Further Analyses of 8 Information Literacy Indicators in Technical and Science Performance Assessments ... 301

9.7 Examining the Effect of Gender, Experience in Computer Use and Other Background Factors on Students’ Technical Proficiency and Information Literacy Competences ... 303

9.7.1 Gender... 303

9.7.2 Years of Experience in Using Computer ... 307

9.7.3 Access to Computer at Home ... 314

9.7.4 Duration of Daily Computer Use at Home ... 319

9.8 Analyses on Students’ Achievements in Information Literacy and School Level Factors ... 326

9.8.1 Medium of Instruction (MOI)... 326

9.8.2 Ability Grouping... 328

9.8.3 School Location... 334

9.8.4 School Sex ... 340

9.8.5 Operational Session ... 343

Chapter 10 Summary and Recommendations

10.1.1 Students’ Information Literacy Competences... 347

10.1.2 Relationship between Students’ Information Literacy Competences in Specific Key Learning Areas and their Technical Proficiency... 350 10.1.3 Relationship between Students’ Information Literacy Competences in Different Key

Learning Areas ... 351

10.1.4 Interaction Effect of Any Two Dimensions of Technical Proficiency on Information Literacy Competences in Specific Key Learning Areas ... 351

10.1.5 Students’ Competences in Information Literacy and their Background Factors ... 352

10.1.6 Students’ Competences in Information Literacy and School Level Factors ... 354

10.1.7 Findings of Questionnaires ... 356

10.1.8 Conclusion ... 357

10.2 Recommendations ... 358

10.2.1 Ensuring Baseline Technology Access in Schools... 358

10.2.2 Empowering Learners with IT ... 359

10.2.3 Empowering Teachers with IT... 360

10.2.4 Enhancing School Leadership for the Knowledge Age... 361

10.2.5 Enriching Digital Resources for Learning... 363

10.2.6 Improving IT Infrastructure and Pioneering Pedagogy using IT ... 363

10.2.7 Providing Continuous Research and Development ... 364

10.2.8 Promoting Community-wide Support and Community Building ... 365

References

List of Tables

Table 2.1 Dimensions of IL in this study ... 7

Table 2.2 Score and item allocation in each PA... 10

Table 2.3 Number of schools by duration needed for completing the Main Study by School Type... 15

Table 3.1 Sampling procedure of schools ... 20

Table 3.2 Performance assessments conducted at a sampled P5 class ... 21

Table 3.3 Number of students sampled and the actual number of students participated in the study ... 24

Table 3.4 No. of students who took part in each of the PAs ... 24

Table 3.5 No. of students who took part in the different combinations of PAs ... 24

Table 3.6 The response rates for Student Questionnaire... 25

Table 3.7 The response rates for School Head and ITC Questionnaires ... 25

Table 3.8 The response rates for Teacher Questionnaire ... 25

Table 4.1 Percentage of schools on the availability of different peripherals ... 31

Table 5.1 Task description and IL dimensions of Technical PA ... 32

Table 5.2a Mean scores of primary school students in 8 IL indicators of Technical PA ... 35

Table 5.2b Mean scores of secondary school students in 8 IL indicators of Technical PA ... 36

Table 5.2c Mean scores of special school students in 8 IL indicators of Technical PA... 36

Table 5.3 Primary school students’ mean score of each Technical PA item ... 38

Table 5.4 Secondary school students’ mean score of each Technical PA item ... 39

Table 5.5 Special school students’ mean score of each Technical PA item ... 40

Table 5.6 Percentage distribution of students of different school types for each score of Q1.1 of Technical PA ... 40

Table 5.7 Percentage distribution of students of different school types for each score of Q1.2 of Technical PA ... 42

Table 5.8 Percentage distribution of students of different school types for each score of Q1.3 of Technical PA ... 43

Table 5.9a Percentage distribution of students of different school types for each score of Q1.4.1a and Q1.4.2a of Technical PA ... 44

Table 5.9b Mean percentage distribution of students of different school types for each score of Q1.4.1a and Q1.4.2a of Technical PA... 45

Table 5.10a Percentage distribution of students of different school types for each score of Q1.4.1b and Q1.4.2b of Technical PA... 46

Table 5.10b Mean percentage distribution of students of different school types for each score of Q1.4.1b and Q1.4.2b of Technical PA ... 46 Table 5.11 Percentage distribution of students of different school types for each score of Q2

(Manage-advanced) of Technical PA ... 50 Table 5.12 Percentage distribution of students of different school types for each score of Q2

(Manage-basic) of Technical PA ... 54 Table 5.13 Percentage distribution of students of different school types for each score of Q2

(Create) of Technical PA ... 55 Table 5.14 Percentage distribution of students of different school types for each score of Q3

(Integrate) of Technical PA ... 58 Table 5.15 Percentage distribution of students of different school types for each score of Q3

(Evaluate) of Technical PA ... 61 Table 5.16 Percentage distribution of students of different school types for each score of Q3

(Create) of Technical PA ... 64 Table 5.17 Percentage distribution of students of different school types for each score of Q3

(Manage) of Technical PA... 66 Table 5.18 Percentage distribution of students of different school types for each score of Q4

(Communicate) of Technical PA ... 67 Table 5.19 ANOVA of 8 IL indicators across primary schools in Technical PA ... 70 Table 5.20 ANOVA of 8 IL indicators across secondary schools in Technical PA... 71 Table 5.21 Mean scores of 8 IL indicators in Technical PA for 40 primary and 33 secondary

schools... 72 Table 5.22 ANOVA of 8 IL indicators between the primary and secondary levels in Technical PA . 72 Table 5.23 Mean scores of primary, secondary and special school students (excluding those

“not-reached” and “non-response” students) across the 8 IL indicators of

Technical PA ... 74 Table 6.1 Task description and IL dimensions of P5 Chinese Language PA... 80 Table 6.2 Task description and IL dimensions of S2 Chinese Language PA... 83 Table 6.3 Mean scores of primary school students in 8 IL indicators of Chinese Language PA .. 88 Table 6.4 Mean scores of secondary school students in 8 IL indicators of Chinese Language PA.. 89 Table 6.5 Mean scores of special school students in 8 IL indicators of Chinese Language PA . 89 Table 6.6 Primary school students’ mean score of each Chinese Language PA item ... 91 Table 6.7 Secondary school students’ mean score of each Chinese Language PA item ... 92 Table 6.8 Special school students’ mean score of each Chinese Language PA item ... 93 Table 6.9 Percentage distributions of P5 students for each score of Q1.1 (Access) in Chinese

Language PA ... 94 Table 6.10 Percentage distributions of P5 students for each score of Q1.1 (Manage_titles) in

Chinese Language PA ... 95 Table 6.11 Percentage distributions of P5 students for each score of Q1.1 (Manage_ file

naming and saving) in Chinese Language PA... 96

Table 6.12 Percentage distributions of P5 students for each score of Q1.1 (Manage_sequence) in Chinese Language PA... 97 Table 6.13 Percentage distributions of P5 students for each score of Q1.1 (Create) in Chinese

Language PA ... 97 Table 6.14 Percentage distributions of P5 students for each score of Q1.2 (Define) in Chinese

Language PA ... 98 Table 6.15 Percentage distributions of P5 students for each score of Q2 (Communicate_email

address) in Chinese Language PA... 99 Table 6.16 Percentage distributions of P5 students for each score of Q2

(Communicate_subject) in Chinese Language PA... 100 Table 6.17 Percentage distributions of P5 students for each score of Q2

(Communicate_manner) in Chinese Language PA ... 100 Table 6.18 Percentage distributions of P5 students for each score of Q2

(Communicate_recipient & signature) in Chinese Language PA ... 101 Table 6.19 Percentage distributions of P5 students for each score of Q2 (Integrate) in Chinese

Language PA ... 103 Table 6.20 Percentage distributions of P5 students for each score of Q3 (Evaluate) in Chinese

Language PA ... 104 Table 6.21 Percentage distributions of P5 students for each score of Q4 (Access) in Chinese

Language PA ... 106 Table 6.22 Percentage distributions of P5 students for each score of Q4 (Manage) in Chinese

Language PA ... 106 Table 6.23 Percentage distributions of P5 students for each score of Q4 (Integrate_titles) in

Chinese Language PA ... 107 Table 6.24 Percentage distributions of P5 students for each score of Q4

(Integrate_information filtering) in Chinese Language PA ... 108 Table 6.25 Percentage distributions of P5 students for each score of Q4 (Create) in Chinese

Language PA ... 108 Table 6.26 Percentage distributions of S2 students for each score of Q1 (Access) in Chinese

Language PA ...118 Table 6.27 Percentage distributions of S2 students for each score of Q1 (Manage_titles) in

Chinese Language PA ...119 Table 6.28 Percentage distributions of S2 students for each score of Q1 (Manage_ file

naming and saving) in Chinese Language PA...119 Table 6.29 Percentage distributions of S2 students for each score of Q1 (Manage_sequence)

in Chinese Language PA ... 120 Table 6.30 Percentage distributions of S2 students for each score of Q1 (Create) in Chinese

Language PA ... 121

Table 6.31 Percentage distributions of Special school students for each score of Q1 (Access) in Chinese Language PA ... 122 Table 6.32a Percentage distributions of Special school students for each score of Q1

(Manage_titles) in Chinese Language PA... 122 Table 6.32b Percentage distributions of Special school students for each score of Q1

(Manage_titles) in Chinese Language PA (excluding those ‘not-reached’ and

‘non-response’ students) ... 122 Table 6.33a Percentage distributions of Special school students for each score of Q1

(Manage_ file naming and saving) in Chinese Language PA ... 123 Table 6.33b Percentage distributions of Special school students for each score of Q1

(Manage_ file naming and saving) in Chinese Language PA (excluded those

‘not-reached’ and ‘non-response’ students) ... 123 Table 6.34a Percentage distributions of Special school students for each score of Q1

(Manage_sequence) in Chinese Language PA ... 123 Table 6.34b Percentage distributions of Special school students for each score of Q1

(Manage_sequence) in Chinese Language PA (excluding those ‘not-reached’ and

‘non-response’ students) ... 124 Table 6.35a Percentage distributions of Special school students for each score of Q1 (Create)

in Chinese Language PA ... 124 Table 6.35b Percentage distributions of Special school students for each score of Q1 (Create)

in Chinese Language PA (excluding those ‘not-reached’ and ‘non-response’

students) ... 124 Table 6.36 Percentage distributions of Secondary school students for each score of Q2

(Access) in Chinese Language PA ... 125 Table 6.37 Percentage distributions of Special school students for each score of Q2 (Access)

in Chinese Language PA ... 126 Table 6.38 Percentage distributions of Secondary school students for each score of Q3.1

(Evaluate) in Chinese Language PA ... 127 Table 6.39 Percentage distributions of Special school students for each score of Q3.1

(Evaluate) in Chinese Language PA ... 127 Table 6.40 Percentage distributions of Secondary school students for each score of Q3.2

(Access) in Chinese Language PA ... 128 Table 6.41 Percentage distributions of Special school students for each score of Q3.2 (Access)

in Chinese Language PA ... 128 Table 6.42 Percentage distributions of Secondary school students for each score of Q3.3

(Define) in Chinese Language PA... 129 Table 6.43 Percentage distributions of Special school students for each score of Q3.3 (Define)

in Chinese Language PA... 130

Table 6.44 Percentage distributions of Secondary school students for each score of Q4

(Integrate_content) in Chinese Language PA ... 130 Table 6.45 Percentage distributions of Secondary school students for each score of Q4

(Integrate_morals) in Chinese Language PA ... 131 Table 6.46 Percentage distributions of Secondary school students for each score of Q4

(Communicate_subject) in Chinese Language PA... 131 Table 6.47 Percentage distributions of Secondary school students for each score of Q4

(Communicate_email address) in Chinese Language PA ... 132 Table 6.48 Percentage distributions of Secondary school students for each score of Q4

(Communicate_recipient & signature) in Chinese Language PA ... 132 Table 6.49 Percentage distributions of Secondary school students for each score of Q4

(Communicate_manner) in Chinese Language PA ... 133 Table 6.50 Percentage distributions of Special school students for each score of Q4

(Integrate_content) in Chinese Language PA ... 134 Table 6.51 Percentage distributions of Special school students for each score of Q4

(Integrate_morals) in Chinese Language PA ... 135 Table 6.52 Percentage distributions of Special school students for each score of Q4

(Communicate_subject) in Chinese Language PA... 135 Table 6.53 Percentage distributions of Special school students for each score of Q4

(Communicate_email address) in Chinese Language PA ... 136 Table 6.54 Percentage distributions of Special school students for each score of Q4

(Communicate_ recipient & signature) in Chinese Language PA ... 136 Table 6.55 Percentage distributions of Special school students for each score of Q4

(Communicate_manner) in Chinese Language PA ... 137 Table 6.56 Percentage distributions of Secondary school students for each score of Q5

(Create) in Chinese Language PA ... 138 Table 6.57 Percentage distributions of Secondary school students for each score of Q5

(Integrate_content) in Chinese Language PA ... 139 Table 6.58 Percentage distributions of Secondary school students for each score of Q4

(Integrate_information filtering) in Chinese Language PA ... 139 Table 6.59a Percentage distributions of Secondary school students for each score of Q5

(Manage_ file naming and saving) in Chinese Language PA ... 140 Table 6.59b Percentage distributions of Secondary school students for each score of Q5

(Manage_file naming and saving) in Chinese Language PA (excluding those

‘not-reached’ and ‘non-response’ students) ... 140 Table 6.60a Percentage distributions of Secondary school students for each score of Q5

(Manage_titles) in Chinese Language PA... 140

Table 6.60b Percentage distributions of Secondary school students for each score of Q5 (Manage_titles) in Chinese Language PA (excluding those ‘not-reached’ and

‘non-response’ students) ... 141

Table 6.61 Percentage distributions of Special school students for each score of Q5 (Create) in Chinese Language PA ... 147

Table 6.62 Percentage distributions of Special school students for each score of Q5 (Integrate_content) in Chinese Language PA ... 148

Table 6.63 Percentage distributions of Special school students for each score of Q5 (Integrate_information filtering) in Chinese Language PA ... 148

Table 6.64a Percentage distributions of Special school students for each score of Q5 (Manage_file naming and saving) in Chinese Language PA ... 149

Table 6.64b Percentage distributions of Special school students for each score of Q5 (Manage_file naming and saving) in Chinese Language PA (excluding those ‘not-reached’ and ‘non-response’ students) ... 149

Table 6.65a Percentage distributions of Special school students for each score of Q5 (Manage_titles) in Chinese Language PA... 149

Table 6.65b Percentage distributions of Special school students for each score of Q5 (Manage_ titles) in Chinese Language PA (excluding those ‘not-reached’ and ‘non-response’ students) ... 150

Table 6.66 Mean scores of 8 IL indicators in Chinese Language PA for 40 primary schools .... 156

Table 6.67 ANOVA of 8 IL indicators across Primary schools in Chinese Language PA... 156

Table 6.68 Mean scores of 8 IL indicators in Chinese Language PA for 33 Secondary schools 157 Table 6.69 ANOVA of 8 IL indicators across Secondary schools in Chinese Language PA... 158

Table 6.70 Mean scores of Primary 5 students (excluding those “not-reached” and “non-response” students) across the 8 IL indicators of Chinese Language PA ... 159

Table 6.71 Mean scores of Secondary 2 students (excluding those “not-reached” and “non-response” students) across the 8 IL indicators of Chinese Language PA ... 160

Table 6.72 Mean scores of Special School students (excluding those “not-reached” and “non-response” students) across the 8 IL indicators of Chinese Language PA ... 160

Table 7.1 Task description and IL dimensions of Mathematics PA ... 166

Table 7.2 Mean scores of primary school students in 8 IL indicators of Mathematics PA... 168

Table 7.3 Primary school students’ mean score of each Mathematics PA item ... 169

Table 7.4 Percentage distribution of primary school students for each score of Q1.1 (Define) of Mathematics PA ... 171

Table 7.5 Percentage distribution of primary school students for each score of Q1.2 (Access) of Mathematics PA ... 172

Table 7.6 Percentage distribution of primary school students for each score of Q1.3 (Access) of Mathematics PA ... 172

Table 7.7 Percentage distribution of primary school students for each score of Q1.4 (Access) of Mathematics PA ... 172 Table 7.8 Percentage distribution of primary school students for each score of Q1.4

(Integrate) of Mathematics PA ... 172 Table 7.9 Percentage distribution of primary school students for each score of Q2 (Create)

of Mathematics PA ... 175 Table 7.10 Percentage distribution of primary school students for each score of Q2 (Integrate)

of Mathematics PA ... 176 Table 7.11 Percentage distribution of primary school students for each score of Q2 (Manage)

of Mathematics PA ... 178 Table 7.12 Percentage distribution of primary school students for each score of Q3.1a (Create)

of Mathematics PA ... 179 Table 7.13 Percentage distribution of primary school students for each score of Q3.1b (Create)

of Mathematics PA ... 179 Table 7.14 Percentage distribution of primary school students for each score of Q3.1c (Create)

of Mathematics PA ... 179 Table 7.15 Percentage distribution of primary school students for each score of Q3.2

(Integrate) of Mathematics PA ... 180 Table 7.16 Percentage distribution of primary school students for each score of Q3.3

(Integrate) of Mathematics PA ... 181 Table 7.17 Percentage distribution of primary school students for each score of Q4 (Manage)

of Mathematics PA ... 182 Table 7.18 Percentage distribution of primary school students for each score of Q5

(Manage-advanced) of Mathematics PA ... 184 Table 7.19 Percentage distribution of primary school students for each score of Q5

(Manage-basic) of Mathematics PA... 185 Table 7.20 Percentage distribution of primary school students for each score of Q5

(Communicate) of Mathematics PA... 186 Table 7.21 Percentage distribution of primary school students for each score of Q6.1 (Access)

of Mathematics PA ... 187 Table 7.22 Percentage distribution of primary school students for each score of Q6.2

(Integrate) of Mathematics PA ... 188 Table 7.23 Percentage distribution of primary school students for each score of Q6.2

(Evaluate) Mathematics PA... 188 Table 7.24 Mean scores of 8 IL indicators in Mathematics PA for 40 primary schools ... 190 Table 7.25 ANOVA of 8 IL indicators across primary schools in Mathematics PA ... 191 Table 7.26 Mean scores of primary school students (excluding those “not-reached” and

“non-response” students) across the 8 IL indicators of Mathematics PA ... 192

Table 8.1 Task description and IL dimensions of Science PA ... 195

Table 8.2 Mean scores of secondary school students in 8 IL indicators of Science PA ... 198

Table 8.3 Mean scores of special school students in 8 IL indicators of Science PA ... 199

Table 8.4 Secondary school students’ mean score of each Science PA item ... 201

Table 8.5 Special school students’ mean score of each Science PA item ... 202

Table 8.6 Percentage distribution of students of different school types for each score of Q1.1 of Science PA ... 203

Table 8.7 Percentage distribution of students of different school types for each score of Q1.2 of Science PA ... 204

Table 8.8 Percentage distribution of students of different school types for each score of Q1.3 (Access) of Science PA ... 205

Table 8.9 Percentage distribution of students of different school types for each score of Q1.3 (Manage) of Science PA... 207

Table 8.10 Percentage distribution of students of different school types for each score of Q2.1 of Science PA ... 207

Table 8.11 Percentage distribution of students of different school types for each score of Q2.2 of Science PA ... 209

Table 8.12 Percentage distribution of students of different school types for each score of Q3.1 (Manage) of Science PA... 210

Table 8.13 Percentage distribution of students of different school types for each score of Q3.1 (Create) of Science PA ... 222

Table 8.14 Percentage distribution of students of different school types for each score of Q3.2 of Science PA ... 225

Table 8.15 Percentage distribution of students of different school types for each score of Q4 of Science PA ... 225

Table 8.16 Percentage distribution of students of different school types for each score of Q5 of Science PA ... 226

Table 8.17 Percentage distribution of students of different school types for each score of Q6 of Science PA ... 227

Table 8.18 Percentage distribution of students of different school types for each score of Q7.1 of Science PA ... 228

Table 8.19 Percentage distribution of students of different school types for each score of Q7.2 of Science PA ... 229

Table 8.20 Percentage distribution of students of different school types for each score of Q7.3 of Science PA ... 231

Table 8.21 Percentage distribution of students of different school types for each score of Q7.4 (Evaluate) of Science PA ... 232

Table 8.22a Percentage distribution of students of different school types for each score of Q7.4 (Communicate) of Science PA ... 233 Table 8.22b Percentage distribution of students (excluding “not-reached” and “non-response”

students) of different school types for each score of Q7.4 (Communicate) of

Science PA ... 234 Table 8.23a Mean scores of 8 IL indicators in Science PA for 33 secondary schools ... 236 Table 8.23b ANOVA of 8 IL indicators across secondary schools in Science PA... 236 Table 8.24 Mean scores of secondary school students (excluding those “not-reached” and

“non-response” students) across the 8 IL indicators of Science PA... 237 Table 8.25 Mean scores of special schools students (excluding those “not-reached” and

“non-response” students) across the 8 IL indicators of Science PA... 237 Table 9.1 Level of agreement on encouraging target teachers to achieve the traditionally

important /emerging curriculum goals as indicated by school heads ... 243 Table 9.2 Level of importance of the use of ICT in school in the target grade as indicated by

school heads (Q3 of School Head Questionnaire) ... 244 Table 9.3 Level of importance of the use of ICT in school in traditionally

important/emerging curriculum goals ... 245 Table 9.4 Mean scores of school heads’ views on the priority of particular resource category 247 Table 9.5 Level of agreement on encouraging teachers to use different types of assessment

at the target grade as indicated by school heads (Q11 of School Head

Questionnaire) ... 248 Table 9.6 Level of agreement on encouraging teachers to use the three types of assessment

as indicated by school heads ... 248 Table 9.7 Percentage of school heads indicating the knowledge and skills which teachers

required or were encouraged to acquire (Q12 of School Head Questionnaire) ... 250 Table 9.8 Mean scores of school heads’ views on teachers’ knowledge and skills ... 251 Table 9.9 Mean scores of school heads’ views on the priority of school leadership

competences (Q13 of School Head Questionnaire) ... 252 Table 9.10 Mean scores of the frequency of conducting the learning activities by primary

Mathematics teachers and the percentage of teachers showing that ICT was used in conducting those activities (Q7 of the Teacher Questionnaire) ... 253 Table 9.11 Mean scores of the frequency of conducting the learning activities by Chinese

Language teachers and the percentage of teachers showing that ICT was used in conducting those activities (Q7 of the Teacher Questionnaire) ... 254 Table 9.12 Mean scores of the frequency of conducting the learning activities by Science

teachers of the secondary and special schools and the percentage of teachers showing that ICT was used in conducting those activities (Q7 of Teacher Questionnaire) ... 255

Table 9.13 Mean scores of the frequency of the types of pedagogical practices by teachers

(Q12a of Teacher Questionnaire) ... 257

Table 9.14 Mean scores of the three pedagogical practice orientations and the use of ICT... 259

Table 9.15 Mean percentages of the use of assessment methods and the use of ICT to carry out those assessments as indicated by teachers... 261

Table 9.16 Mean scores of student practices (Q14 of Teacher Questionnaire)... 262

Table 9.17 Mean scores of ICT impact on students as perceived by teachers ... 264

Table 9.18 Mean scores of self-proclaimed competences as perceived by teachers ... 266

Table 9.19 Percentage of teachers indicated that they have encountered the three kinds of obstacles in using ICT in their teaching... 268

Table 9.20 Percentage of availability of technology-related resources as indicated by ITCs of the primary schools (Q4 of ITC Questionnaire) ... 270

Table 9.21a Percentage of availability of technology-related resources as indicated by ITCs of the secondary schools (Q4 of ITC Questionnaire)... 271

Table 9.21b Percentage of availability of technology-related resources as indicated by ITCs of the special schools (Q4 of ITC Questionnaire) ... 272

Table 9.22 Mean number of computers for different purposes as indicated by ITCs (Q5 of ITC Questionnaire) ... 273

Table 9.23 Mean number of laptops in schools as indicated by ITCs (Q6 of the ITC Questionnaire) ... 273

Table 9.24 Mean number of technological equipment in schools as indicated by ITCs (Q7 of ITC Questionnaire) ... 274

Table 9.25a Percentage of ITCs indicating different levels of technical support available in primary schools (Q16 of ITC Questionnaire) ... 275

Table 9.25b Percentage of ITCs indicating different levels of technical support available in secondary schools (Q16 of ITC Questionnaire)... 276

Table 9.25c Percentage of ITCs indicating different levels of technical support available in special schools (Q16 of ITC Questionnaire)... 277

Table 9.26 Mean scores of the frequency of using computer for different purposes as rated by students (Q9 of Student Questionnaire) ... 281

Table 9.27 Mean scores of the level of proficiency on the 13 technical skills as indicated by students (Q10 of Student Questionnaire) ... 282

Table 9.28 Impact on the use of ICT (Q21 of Student Questionnaire) ... 285

Table 9.29 ICT Impact on learning Mathematics (Q23 of Student Questionnaire at primary schools) ... 286

Table 9.30 ICT impact on learning Science (Q23 of Student Questionnaire at secondary and special schools) ... 286

Table 9.31 ICT impact on learning Chinese Language (Q25 of Student Questionnaire for all three school types) ... 287 Table 9.32 Correlations of students’ performance in IL of Chinese Language PA at primary

schools... 289 Table 9.33 Correlations of students’ performance in IL of Mathematics PA at primary schools 289 Table 9.34 Correlations of students’ performance in IL of Mathematics and Chinese

Language PAs at primary schools ... 290 Table 9.35 Correlations of students’ performance in IL of Chinese Language PA at secondary

schools... 291 Table 9.36 Correlations of students’ performance in IL of Science PA at secondary schools .... 292 Table 9.37 Correlations of students’ performance in IL of Science and Chinese Language PAs

at secondary schools... 293 Table 9.38 Correlations of students’ performance in IL of Technical PA at primary schools... 294 Table 9.39 Correlations of students’ performance in IL of Technical PA at secondary schools . 294 Table 9.40 Correlations of students’ performance in IL of Chinese Language and Technical

PAs at primary level ... 296 Table 9.41 Interaction effect of any two technical competences on each Chinese Language IL

competence for primary school students... 297 Table 9.42 Correlations of students’ performance in IL of Chinese Language and Technical

PAs at secondary level ... 298 Table 9.43 Interaction effect of any two technical competences on each Chinese Language IL

competence for secondary school students ... 299 Table 9.44 Correlations of students’ performance in IL of Mathematics and Technical PAs at

primary schools ... 300 Table 9.45 Interaction effect of any two technical competences on each Mathematics IL

competence for primary school students... 300 Table 9.46 Significance level of the interaction effect of any two technical proficiency on

each Mathematics IL competence for primary school students ... 301 Table 9.47 Correlations of students’ performance in IL of Science and Technical PAs at

secondary schools ... 301 Table 9.48 Interaction effect of any two technical competences on each Science IL

competences for secondary school students... 302 Table 9.49 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

gender... 303 Table 9.50 Mean scores of 8IL indicators in Mathematics PA at primary schools with regard

to gender... 304 Table 9.51 Mean scores of 8 IL indicators in Technical PA at primary schools with regard to

gender... 304

Table 9.52 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard to gender... 305 Table 9.53 Mean scores of 8 IL indicators in Chinese Language PA at Primary schools with

regard to gender ... 306 Table 9.54 Mean scores of 8 IL indicators in Chinese Language PA at Secondary schools with

regard to gender ... 307 Table 9.55 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

the years of experience in using computer... 308 Table 9.56 Mean scores of 8 IL indicators in Technical PA at primary schools with regard to

the years of experience in using computer... 309 Table 9.57 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard

to the years of experience in using computer... 310 Table 9.58 Mean scores of 8 IL indicators in Mathematics PA at primary schools with regard

to the years of experience in using computer...311 Table 9.59 Mean scores of 8 IL indicators in Chinese Language PA at Primary schools with

regard to the years of experience in using computer ... 312 Table 9.60 Mean scores of 8 IL indicators in Chinese Language PA at secondary schools with

regard to the years of experience in using computer ... 314 Table 9.61 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

computer ownership ... 315 Table 9.62 Mean scores of 8 IL indicators in Technical PA at primary schools with regard to

computer ownership ... 316 Table 9.63 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard

to computer ownership... 316 Table 9.64 Mean scores of 8 IL indicators in Mathematics PA at primary schools with regard

to computer ownership... 317 Table 9.65 Mean scores of 8 IL indicators in Chinese Language PA at primary schools with

regard to computer ownership... 318 Table 9.66 Mean scores of 8 IL indicators in Chinese Language PA at secondary schools with

regard to computer ownership... 319 Table 9.67 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

duration of computer use per day... 320 Table 9.68 Mean scores of 8 IL indicators in Technical PA at primary schools with regard to

duration of computer use per day... 321 Table 9.69 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard

to duration of computer use per day... 322 Table 9.70 Mean scores of 8 IL indicators in Mathematics PA at primary schools with regard

to duration of computer use per day... 323

Table 9.71 Mean scores of 8 IL indicators in Chinese Language PA at Primary schools with regard to duration of computer use per day ... 324 Table 9.72 Mean scores of 8 IL indicators in Chinese Language PA at secondary schools with

regard to duration of computer use per day ... 325 Table 9.73 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

Medium of instruction... 326 Table 9.74 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard

to Medium of instruction... 327 Table 9.75 Mean scores of 8 IL indicators in Chinese Language PA at secondary schools with

regard to Medium of instruction ... 328 Table 9.76 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

ability grouping ... 329 Table 9.77 Mean scores of 8 IL indicators in Technical PA at primary schools with regard to

ability grouping ... 330 Table 9.78 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard

to ability grouping ... 331 Table 9.79 Mean scores of 8 IL indicators in Mathematics PA at primary schools with regard

to ability grouping ... 332 Table 9.80 Mean scores of 8 IL indicators in Chinese Language PA at primary schools with

regard to ability grouping... 333 Table 9.81 Mean scores of 8 IL indicators in Chinese Language PA at secondary schools with

regard to ability grouping... 334 Table 9.82 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

school locations... 335 Table 9.83 Mean scores of 8 IL indicators in Technical PA at primary schools with regard to

school locations... 336 Table 9.84 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard

to school locations... 337 Table 9.85 Mean scores of 8 IL indicators in Mathematics PA at primary schools with regard

to school locations... 338 Table 9.86 Mean scores of 8 IL indicators in Chinese Language PA at primary schools with

regard to school locations ... 339 Table 9.87 Mean scores of 8 IL indicators in Chinese Language PA at secondary schools with

regard to school locations ... 340 Table 9.88 Mean scores of 8 IL indicators in Science PA at secondary schools with regard to

school sex ... 341 Table 9.89 Mean scores of 8 IL indicators in Technical PA at secondary schools with regard

to school sex... 342

Table 9.90 Mean scores of 8 IL indicators in Chinese Language PA at secondary schools with regard to school sex... 343 Table 9.91 Mean scores of 8 IL indicators in Technical PA at primary schools with regard to

operational sessions... 344 Table 9.92 Mean scores of 8 IL indicators in Mathematics PA at primary schools with regard

to operational sessions ... 345 Table 9.93 Post-hoc tests of 8 IL indicators in Mathematics PA at primary schools with regard

to operational sessions ... 345 Table 9.94 Mean scores of 8 IL indicators in Chinese Language PA at primary schools with

regard to operational sessions ... 346 Table 9.95 Post-hoc tests of 8 IL indicators in Chinese Language PA at primary schools with

regard to operational sessions ... 346 Table 10.1 Indicators in which statistically significant differences were found in relation to

access to computer at home ... 354

List of Figures

Figure 2.1 Diagram showing the relationship amongst ICT literacy, learning in the KLAs and factors affecting the use of ICT... 4 Figure 5.1 Percentages of primary, secondary and special school students in completing the

tasks of Technical PA ... 34 Figure 5.2 Mean score percentages of primary, secondary and special school students in 8 IL

indicators of Technical PA ... 37 Figure 5.3 Students’ IL performance in Technical PA across primary schools ... 69 Figure 5.4 Students’ IL performance in Technical PA across secondary schools... 70 Figure 5.5 Mean score percentages of primary, secondary and special school students

(excluding those “not-reached” and “non-response” students) in the 8 IL

indicators of Technical PA ... 75 Figure 6.1 Percentages of primary school students in completing the tasks of Chinese

Language PA ... 85 Figure 6.2 Percentages of secondary school students in completing the tasks of Chinese

Language PA ... 86 Figure 6.3 Percentages of special school students in completing the tasks of Chinese

Language PA ... 86 Figure 6.4 Students’ IL performance in Chinese Language PA across primary schools ... 155 Figure 6.5 Students’ IL performance in Chinese Language PA across Secondary schools ... 157 Figure 6.6 Mean score percentages of Primary, Secondary and Special schools students

(excluding those “not-reached” and “non-response” students) in the 8 IL

indicators of Chinese Language PA ... 161 Figure 7.1 Percentages of primary school students in completing the tasks of Mathematics PA . 167 Figure 7.2 Students group the shapes into two classes in Q4 of the assessment... 183 Figure 7.3 Students’ IL performance in Mathematics PA across primary schools ... 190 Figure 8.1 Percentages of secondary school students in completing the tasks of Science PA ... 197 Figure 8.2 Percentages of special school students in completing the tasks of Science PA ... 197 Figure 8.3 Mean score percentages of secondary and special schools students in 8 IL

indicators of Science PA ... 200 Figure 8.4 Students’ work on information search by posting the questions on “Yahoo

Knowledge” ... 204 Figure 8.5 Students’ information search of invalid information in “Yahoo Knowledge” ... 208 Figure 8.6 Students’ IL performance in Science PA across secondary schools... 235 Figure 8.7 Mean score percentages of secondary and special schools students (excluding those

“not-reached” and “non-response” students) in the 8 IL indicators of Science PA... 238 Figure 9.1 Question related to Curriculum goals in subject-specific content (Q2 of School

Head Questionnaire) ... 242

Figure 9.2 Level of agreement on encouraging target teachers to achieve the curriculum

goals as indicated by school heads... 242 Figure 9.3 Question related to the priority of resource allocation (Q6 of School Head

Questionnaire) ... 245 Figure 9.4 Percentage of school heads indicating that high priority was given to the particular

resource allocation in school... 246 Figure 9.5 Question about the knowledge and skills that teachers needed or were encouraged

to acquire (Q12 of School Head Questionnaire) ... 249 Figure 9.6 Percentage of school heads indicating the knowledge and skills which teachers

required or were encouraged to acquire ... 249 Figure 9.7 Teacher’s pedagogical practices using ICT (Q12b of Teacher Questionnaire)... 258 Figure 9.8 Types of assessment in Teacher Questionnaire (Q13 of Teacher Questionnaire) ... 260 Figure 9.9 Mean percentages of teachers indicating that ICT was used for the three

categories of student practices ... 263 Figure 9.10 List of items on impact of ICT used (Q18 of Teacher Questionnaire)... 264 Figure 9.11 List of self-proclaimed competences in uses of ICT (Q19 of Teacher

Questionnaire) ... 265 Figure 9.12 List of obstacles encountered by teachers in using ICT in teaching (Q21 of

Teacher Questionnaire) ... 267 Figure 9.13 Obstacles encountered by teachers of the primary and secondary schools... 267 Figure 9.14 Obstacles encountered by teachers of the special schools ... 268 Figure 9.15 Years of experience in using computer ... 278 Figure 9.16 Computer and Internet access at home ... 279 Figure 9.17 Number of hours using computer per day... 280 Figure 9.18 Level of competence in three kinds of ICT usage... 283 Figure 9.19 Sources of help when encountering difficulties ... 284

List of Symbols and Terms

Statistical Symbols:

N total number of valid responses SD standard deviation

Max maximum value Min minimum value df degrees of freedom

r Pearson product-moment correlation coefficient Sig. observed significance level

Terms:

ICT “ICT” stands for “information communication technology”. In this report, IT and ICT are interchangeable and carry the same meaning.

IL Information literacy

ISP Internet service provider

KLA Key learning area

PA Performance assessment

SITES Second Information Technology in Education Studies

7 IL Dimensions “7 IL Dimensions” include the “Define”, “Access”, “Manage”,

“Integrate”, “Create”, “Communicate” and “Evaluate” dimensions.

8 IL Indicators “8 IL Indicators” include the “Define”, “Access”, “Manage”,

“Integrate”, “Create”, “Communicate” and “Evaluate” dimensions as well as the “Total Score”.

Total Score “Total Score” means the sum of respective scores of “Define”,

“Access”, “Manage”, “Integrate”, “Create”, “Communicate” and

“Evaluate” dimensions.

Mean Score Percentage Mean Score Percentage = (Mean score / Full score)*100%

School Types The 3 “School Types” are primary, secondary and special schools.

Executive Summary

1 Purpose of the Study

The Education Bureau (EDB) of the Government of the Hong Kong Special Administrative Region (HKSAR) has commissioned the Centre for Information Technology in Education (CITE), the University of Hong Kong to conduct the "Phase (II) Study on Evaluating the Effectiveness of the 'Empowering Learning and Teaching with Information Technology' Strategy (2004/2007)" [Phase (II) Study] to evaluate the impact of Information Technology (IT) on students' learning in specific Key Learning Areas (KLAs) as well as for timely overall analysis of all relevant data collected within 2004/05 to 2006/07 school years for concluding the effectiveness of the Strategy based on the results of both Phase (II) Study and Phase (I) Study¹ and informing future policies. The overall objectives of the study are as follows:

• to evaluate the impact of IT on empowering students’ learning in Chinese Language and Mathematics at primary school level as well as Chinese Language and Science at secondary school level and in special schools; and

• to conclude the overall effectiveness of the Strategy and to recommend the way forward for IT in Education (ITEd).

2 Methodology

Two types of instruments were specially designed for this study. The first type of instruments was the online performance assessments (PAs) on information literacy (IL) including Technical, primary Mathematics, Science, primary Chinese Language and secondary Chinese Language. The aims of these assessments were to find out students’ level of proficiency in IL. The second type of instruments was the questionnaires including Student Questionnaire, Teacher Questionnaire, School Head Questionnaire and Information Technology Coordinator (ITC) Questionnaire. The aims of these questionnaires were to collect students’ background information on using ICT² and information on factors at school level, such as school leadership, learning and teaching practices in using ICT in school as well as IT infrastructure and support, that would affect students’ learning in using ICT.

In this study, the target population included primary 5 (P5) and secondary 2 (S2) students in the 2006/07 academic year and those teachers teaching the related subjects (Chinese Language and Mathematics at P5 level as well as Chinese Language and Science at S2 level) as well as school heads and IT coordinators (ITCs) in the sampled schools. 40 primary schools and 33 secondary

1 The “Phase (I) Study on Evaluating the Effectiveness of the 'Empowering Learning and Teaching with Information Technology' Strategy (2004/2007) (the Strategy) ” [Phase (I) Study] was also a study commissioned by the EDB (former Education and Manpower Bureau) to a local tertiary institution focusing on reviewing the progress of various ITEd initiatives as put forth in the Strategy.

2 The terms ICT and IT are interchangeable which means information communication technology.

schools as well as 4 special schools took part in this study. The overall response rates were 26.85%, 23.57%, and 80% for the primary, secondary and special schools respectively. One intact class of the target grade level was sampled from each of the participating schools. The sample sizes, actual number of participants and response rates of the performance assessments and questionnaire surveys were summarized in the table below:

Table E1 Sample sizes, actual number of participants and response rates of performance assessments and questionnaire surveys

School Type

Primary Secondary Special

Instrument Type

Sample Size

Actual no.

of participants

Response Rate

(%)

Sample Size

Actual no.

of participants

Response Rate

(%)

Sample Size

Actual no.

of participants

Response Rate

(%)

Online Performance Assessments

1340 1320 98.51 1300 1302 100.15* 41 35 85.37

School Head Questionnaire 40 37 92.50 33 31 93.94 4 3 75.00

ITC Questionnaire 40 38 95.00 33 33 100.00 4 4 100.00

Student Questionnaire 1340 1227 91.57 1300 1234 94.92 41 33 80.49

Chinese Language Teachers

42 41 97.62 39 35 89.74 6 3 50.00

Science Teachers

/ / / 35 34 97.14 4 3 75.00

Teacher Questionnaire

Mathematics Teachers

44 40 90.91 / / / / / /

N.B. *The sampling was done in July 2006 and the PAs were conducted from December 2006 till early April 2007. There were students enrolled/dropped out in schools during that period of time. Therefore, the response rate exceeds 100% for the secondary schools.

3 Summary of Findings

3.1 Students’ Achievements in Information Literacy

Student’s performances in each PA will be summarized first. Then cross-schools analysis for each set of PA will be presented for primary and secondary schools separately. As only four special schools participated in this study, no further analysis was conducted across special schools.

Results from the Technical PA indicated that students in the primary, secondary and special schools had good performances in the dimensions of “define’, “access” and “manage”. On the other hand, poor performance was found in the dimensions of “communicate” and “create”. Results showed that

secondary school students had significantly better performance than that of the primary school students with respect to all IL indicators. Results also showed that there were significant differences across schools in terms of students’ levels of IL competences in technical proficiency. For the primary school students, larger dispersion was found in the dimensions of “access” and “manage”.

For the secondary school students, larger dispersion was found in the dimension of “evaluate”.

In Science PA, results from the PA indicated that students in both secondary schools and special schools had better performance in the “define” and “access” dimensions. Poor performance was found in the “integrate” and “evaluate” dimensions for both secondary and special school students.

Results also showed that there were significant differences across secondary schools in terms of students’ levels of IL competences in Science PA. Larger dispersion was found in the dimensions of

“define”, “access” and “integrate”.

In Mathematics PA, regarding the 7 IL dimensions, better performances were found in “define” and

“create” dimensions. Poor performances were found in “evaluate” and “integrate” dimensions.

Results also showed that there were significant differences across primary schools in terms of students’ levels of IL competences in Mathematics PA. It was also observed that smaller dispersion was found in the dimensions of “define” and “evaluate” and larger dispersion was noted in “access”,

“manage”, “integrate” and “create” dimensions.

The overall performance of P5 students in Chinese Language PA was not very impressive. Students performed the best in the “define” dimension. Their performances were poor in the dimensions of

“access”, “communicate” and “evaluate”. There were significant differences across the primary schools in terms of students’ levels of IL competences in Chinese Language PA by ANOVA.

Students’ overall performance in Chinese Language PA in the secondary schools was average.

Secondary school students performed better in the dimensions of “manage”, “define” and “access”.

The lowest performance was found in the dimension of “integrate”, followed by “evaluate”. There were significant differences across the secondary schools in terms of students’ IL performance. For students in the special schools, the overall performance was not impressive. Special school students performed better in the “manage”, “define” and “access” dimensions. The lowest performance was found in the “evaluate” dimension.

In sum, when examining the variability across schools, it was found that, in primary schools, larger dispersion was found in the “access” dimension for the 3 sets of PAs and smaller dispersion in the

“define” dimension for both Mathematics and Chinese Language PAs. In secondary schools, larger dispersion was found in “access” and “integrate” dimensions for both Science and Chinese Language PAs. The dimension of “evaluate” was with smaller dispersion in secondary schools for both Chinese Language and Science PAs.

3.2 Relationship between Students’ Information Literacy Competences in Specific Key Learning Areas and their Technical Proficiency

At primary school level, the correlations of students’ technical proficiency and their IL competences in Chinese Language PA were stronger than those of their technical proficiency and IL competences in Mathematics PA. Among the one-to-one corresponding pairs of the 7 IL dimensions between Technical PA and primary Chinese Language PA, the correlation in the “manage” dimension was relatively stronger. The same was observed between the Technical PA and Mathematics PA. For the secondary school students, the correlations of their technical proficiency and their IL competences in Chinese Language PA were stronger than those of their technical proficiency and IL competences in Science PA. In both circumstances, the strongest correlation was observed in the “integrate”

dimension.

3.3 Relationship between Students’ Information Literacy Competences in Different Key Learning Areas

Significant correlations of the 8 corresponding pairs of IL indicators of primary Mathematics and Chinese Language PAs were noted except the pair of “evaluate”. A strong correlation between the

“total” score of IL competences in primary Mathematics and Chinese Language PAs was observed.

In general, the correlations of the 8 corresponding pairs of IL indicators of Science and secondary Chinese Language PAs were weak. Positively significant correlations were found in “access”,

“manage”, “integrate” and the “total” score. Also, the pair “communicate” was negatively and significantly correlated.

3.4 Interaction Effect of Any Two Dimensions of Technical Proficiency on Information Literacy Competences in Specific Key Learning Areas

There were 9 pairs of indicators in Technical PA that had interaction effect on Mathematics IL competences. It was revealed that among the 7 IL dimensions in Mathematics PA, “communicate”

and “evaluate” were affected most by such interaction. In terms of the number of IL dimensions being affected, the interaction effect of “integrate” and “communicate” in technical proficiency had a broader impact on Science IL competences. The interaction of “define” and “communicate” in technical proficiency had a broader effect on primary Chinese Language IL competences. It was found that only three combinations (“integrate-access”, “create-define” and “create-manage”) of IL dimensions in Technical PA had effect on Chinese Language IL competences in the secondary schools.

3.5 Students’ Competences in Information Literacy and their Background Factors

Some background factors collected in Student Questionnaire were used to explore whether there were any effects on the students’ performance in assessments. As small amount of special school data were collected, special school students’ data were excluded for such analysis.