Chapter 3 Conceptual Framework and Research Questions
3.2 Indicators for Evaluation
Based on the above Conceptual Framework, the eleven evaluative areas of concern are defined. In order to examine the performance of these areas, expected outcomes for each evaluative area and the indicators of the evaluation framework on ITEd have been developed through literature review, documentary study of relevant academic and policy papers and vigorous consultations with relevant experts from the HKIEd, consultants, the research team of the EMB and honorary advisers. Each question item designed with an embedded indicator is used to evaluate the achievement of an expected outcome. More than one indicator can be used to assess a given expected outcome. Examples of indicators of the expected outcomes in one of the evaluative areas are listed below.
Evaluative Area 2:
Teachers’ IT Competency as well as Teachers’ & School Head’s Perception of ITEd
Expected Outcome:
2a. Most of the teachers meet the IT competency requirements.
Indicators
Variable 1 Frequency of teachers/therapists’ use of computer in school
Variable 2 Teachers’ overall level of IT competency [Basic (BIT) Level, Intermediate (IIT) Level, Upper Intermediate (UIT) Level and Advanced (AIT) Level]4
Variable 3 School heads’ view on teachers’ IT competency in the school
Variable 4 Teachers/therapists’ self-evaluated level of competency in using hardware Variable 5 Teachers/therapists’ self-evaluated level of competency in using software Variable 6 Teachers/therapists’ self-evaluated level of confidence in selecting appropriate IT
resources for students
Expected Outcome:
2b. Most of the teachers and school heads agree on the roles of IT in the learning, teaching and assessment process.
Indicators:
Variable 1 Teachers/therapists’ view on the role of IT in learning, teaching and assessment process
Variable 2 School heads’ self-evaluation on whether school head and teachers/therapists understand the role of IT in learning, teaching/training/therapy and assessment process
Expected Outcome:
2c. Most of the teachers agree on their roles when applying IT in the learning, teaching and assessment process.
Indicators:
Variable 1 Teachers/therapists’ perceived roles when applying IT in learning and teaching Variable 2 School heads’ self-evaluation on whether teachers/therapists accept their roles when
applying IT in learning, teaching/training/therapy and assessment
4 Four levels of IT competency (Au, Kong, Ng & Pun, 1999).
Expected Outcome:
2d. Most of the teachers and school heads possess positive attitude towards the impact of IT on learning.
Indicators:
Variable 1 Teachers/therapists’ willingness to apply IT in learning and teaching
Variable 2 Teachers/therapists’ view on the factors affecting their willingness to apply IT in learning and teaching
Variable 3 School heads’ willingness to support teachers to apply IT in learning and teaching Variable 4 School heads’ self-evaluation on whether school heads and teachers have positive
attitude towards the impact of IT on students’ learning
As far as teachers’ IT competency as well as teachers’ and school heads’ perception of ITEd are concerned in the above examples, four expected outcomes are identified (Expected Outcomes 2a-2d). Teachers are expected to meet the IT competency requirements. Teachers and school heads are expected to demonstrate an understanding of the roles of IT in learning, teaching and assessment processes. Teachers are also expected to demonstrate an understanding of their roles when applying IT in learning, teaching and assessment processes. Teachers’ and school heads’
positive attitude towards the impact of IT on learning is also one of the expected outcomes. The IT competency of teachers can be assessed by six indicators, which include frequency of their use of computers in school, overall level of IT competency, self-evaluated level of competency in using hardware and software, self-evaluated level of confidence in selecting appropriate IT resources for students and school head’s view on their IT competency in the school.
A list of the Expected Outcomes of the framework is presented in Table 3.1. A detailed description of the Expected Outcomes of the framework can be found in Appendix 1 of the Main Study (I) Report. This structure guides the design of relevant indicators and survey items of the questionnaires.
Table 3.1 List of the expected outcomes of the framework
Dimensions Evaluative areas (EV) Link to strategic goals Expected outcomes EV1. Students’ Perception of
Learning with IT 1. Empowering Learners with IT 1a. Most of the students show positive attitude towards using IT in the learning process.
2a. Most of the teachers meet the IT competency requirements.
2b. Most of the teachers and school heads agree on the roles of IT in the learning, teaching and assessment process.
2c. Most of the teachers agree on their roles when applying IT in the learning, teaching and assessment process.
EV2. Teachers’ IT Competency as well as Teachers’ & School Heads’ Perception of ITEd
2. Empowering Teachers with IT;
3. Enhancing School Leadership for
the Knowledge Age 2d. Most of the teachers and school heads possess positive attitude towards the impact of IT on learning.
3a. School derives measures to engage parents’ involvement in the promotion of ITEd.
3b. School attempts to derive measures to address digital divide issue.
3c. School always makes use of community resources and takes part in community activities that support and promote the use of IT in learning and teaching.
3d. Most of the students and parents make good use of the IT resources in the community.
3e. Most of the parents are concerned about the impact of IT on students’ learning.
Input
EV3. Community-wide Support & Parents’
Involvement
7. Promoting Community-wide Support and Community Building
3f. Most of the parents possess positive attitude towards the impact of IT on learning.
4a. Most of the students are given the opportunities to learn about the knowledge and skills of IT as well as use them in various school tasks across school curriculum.
4b. School curriculum provides a learning context that requires and enables students’ use of IT as resources and tools to research issues, solve problems and communicate results.
EV4. School ITEd Curriculum 1. Empowering Learners with IT;
2. Empowering Teachers with IT
4c. Most of the teachers understand the school ITEd curriculum.
5a. Teachers continue to receive ITEd professional development focusing on good practices of the application of contemporary IT to enhance learning and teaching in different KLAs.
EV5. School Professional Development in ITEd
2. Empowering Teachers with IT;
3. Enhancing School Leadership for the Knowledge Age
5b. The school head and his/her associates have taken part in ITEd professional development programmes in equipping themselves with knowledge and skills leading to changes in schools, to use IT appropriately in schools for learning and teaching as well as to enhance the efficiency of school administration, for school-based assessment and as a communication tool for communicating with parents, students and other relevant parties.
6a. An annual IT plan which shows clear visions and goals, and the implementation strategies and actions in integrating IT into school curriculum, learning and teaching processes as well as communication and collaboration is prepared, implemented and evaluated.
6b. School has a clear school-based ITEd plan to portray the infrastructure requirements, maintenance and replacement plans. This plan is based on the needs of teachers, students, parents and society.
6c. Most of the teachers are involved in and agree on the school ITEd plan.
EV6. School Leadership 3. Enhancing School Leadership for the Knowledge Age
6d. Most of the teachers have the feeling of being supported by school head and colleagues.
7a. School has developed quality school-based digital educational resources and a repository of online resources for all KLAs.
7b. School has derived an efficient management mechanism for the digital resources to facilitate teaching, learning and sharing among teachers, parents, students and other schools.
7c. School acquires up-to-date IT-based educational digital recourses regularly.
7d. School provides students and teachers with good serviceable computers and other IT facilities (hardware and application software), well-maintained school network for communication and access to multimedia rich content inside school, and sufficient bandwidth for the connection to the Internet at all times.
7e. School provides day-to-day management and maintenance of IT facilities in schools and technical support services to students and teachers.
7f. School provides sufficient consumables arising from the use of IT facilities.
7g. School provides sufficient computer facilities for use by students after normal school hours.
7h. School always improves the IT infrastructure by upgrading and replacing obsolete IT facilities to make them better support today's needs for learning and teaching.
7i. School always improves the IT infrastructure by equipping advanced IT technology to enhance learning, teaching, assessment and school administration.
7j. School has made good use of ITEd resources and funding from EMB, QEF and others sources.
Context
EV7. Digital Resources &
Infrastructure
4. Enriching Digital Resources for Learning;
5a. Improving IT Infrastructure
7k. Most of the teachers and students have made good use of the IT facilities and services.
Table 3.1 List of the expected outcomes of the framework (Continued)
Dimensions Evaluative areas (EV) Link to strategic goals Expected outcomes
8a. School has taken part in trial or pilot project to explore innovative technologies and equipment that may enhance learning and teaching.
8b. School attempts to research or evaluate the effectiveness of the particular pedagogy of using IT in learning and teaching and to disseminate the experiences in the education community.
8c. School always encourages, enables and rewards collaborative team work and sharing among staff and schools to attempt innovative use of IT to enhance learning and teaching.
Context
EV8. School Technology- using Culture and ITEd Development
3. Enhancing School Leadership for the Knowledge Age;
5b. Pioneering Pedagogy Using IT;
6. Providing Continuous Research
and development 8d. The use of IT has been embedded into the daily practices of a school both in the learning, teaching and assessment process, and in school administration.
9a. Learning activities allow students to apply strategies and skills for information retrieval and critical evaluation of different information sources to build knowledge and to solve problems in school tasks and real-life situations.
9b. Learning activities allow students to learn independently and to extend their learning opportunities using digital resources according to individuals’ learning need and pace.
EV9. Students’ Learning
Activities with IT 1. Empowering Learners with IT
9c. Learning activities allow students to plan and apply IT as a productivity tool, a communication tool, a collaboration tool, a research tool and a decision-making tool in school tasks and real-life situations.
10a. Teachers use IT in their daily teaching and learning management such as monitoring students’ learning profile.
10b. Teachers use IT to motivate the learners, to establish the context for learning and to explain abstract concepts.
10c. Teachers use IT as a means to monitor and to assess the performance of learners for improvement.
10d. Teachers use assessment information to design activities that will cater for individual needs of using IT.
10e. Teachers use IT to create a supportive learning environment for independent learning.
Process
EV10. Teachers’ Pedagogical
Practices with IT 2. Empowering Teachers with IT
10f. Teachers create opportunities for students to work collaboratively with the use of IT to construct knowledge, to develop generic competencies and Information Literacy as well as to foster positive attitude, and value judgment in doing school tasks and in using IT.
11a. Most of the students show mastery of the necessary knowledge about IT and the basic concepts and skills in contemporary computing technologies in relation to their corresponding key learning stages.
11b. Most of the students show positive attitude towards social and ethical issues relating to the use of IT.
11c. Most of the students show the ability to select the appropriate technologies and the disposition to apply the learnt knowledge and skills to solve problems in school tasks and real-life situations.
11d. Most of the students can develop the generic skills through IT such as problem solving skills.
Outcome
EV11. Students’ Learning Outcomes in different Key Learning Areas and the Development of Information Literacy and Generic Skills
1. Empowering Learners with IT
11e. Most of the students show positive attitude towards using IT in the learning process.