Chapter 4 Research Design and Methods
4.2 Data Collection Methods
4.2.1 Questionnaire Surveys
In this Study, web-based surveys were implemented for: all school heads and school representatives/ITEd Team heads/IT co-ordinators of primary, secondary and special school sectors, ITEd Team teachers and teachers (including therapists of some special schools) of the sampled schools as well as selected students of the sampled primary, secondary and SSD (School for Social Development) schools. For the parents of the three school sectors and majority of students of special schools, traditional paper-based surveys were administered. A web-based approach was chosen because it enabled larger amount of questionnaires to be processed at the same time. As shown in many research reports, there was no difference in reliability between an online survey and a traditional paper-based survey (see e.g. Kaplowitz, Hadlock & Levine, 2004;
Perkins, 2004). The Project Team made use of the Self-evaluation Platform (SEP) on ITEd for Schools provided by the EMB for the online data collection activities. Subsequent data analysis work was conducted by using the SPSS (Statistical Package for the Social Science) programme.
The web-based questionnaire survey involved a ‘self-administered’ process. Respondents who were provided with individual account names and passwords could complete the questionnaires themselves anywhere, anytime. All question items except those items about personal information had to be answered by the stakeholders. Submission of incomplete questionnaires (except items about personal information) was not allowed in the SEP. The system also conducted security checks to ensure that respondents could not submit the questionnaires more than once.
To encourage respondents to give ‘true’ responses to the question items, all surveys were conducted anonymously. Although login accounts and passwords were required, the identities of the respondents were not revealed. Any identification of group responses such as school code and class level was used for tracking the response rate only. Moreover, the system instantly aggregated the individual data of respective respondents so that retrieval of individual data was impossible. In addition, online help or hot-line enquiry was available when respondents had difficulties in completing the questionnaires.
The questionnaire surveys were categorized into two types: inside and outside school. There were seven questionnaire types for inside school surveys and one for outside school survey.
Inside school:
y School Heads’ ITEd Questionnaires (Part 1) for school heads y School Heads’ ITEd Questionnaires (Part 2) for school heads
y School ITEd Survey for school representatives/IT Team heads/IT coordinators y ITEd Team Teachers’ Questionnaire for ITEd Team teachers
y Teachers’ ITEd Questionnaire for all teachers (ITEd Team teachers and non-ITEd Team teachers)
y Therapists’ ITEd Questionnaire for therapists in the special schools y Students’ ITEd Questionnaire for students
Outside school:
y Parents’ ITEd Questionnaire for parents
A full set of questionnaire survey instruments was set out in a separate volume, namely “Study on Evaluating the Effectiveness of the ‘Empowering Learning and Teaching with Information Technology’ Strategy (2004/07) – Finalised Instruments: Questionnaires (English and Chinese versions)”. The distribution of questionnaires by stakeholders and school sectors is presented in Table 4.1.
Table 4.1 Distribution of questionnaires with questionnaire codes by stakeholders and school sectors
Special
Without Therapists With Therapists School Sectors
Questionnaire Types (by Stakeholders)
Primary Secondary Hospital
School (H)
School for Social Development
(SSD)
School for Children with
Visual Impairment
(VI)
School for Children with Hearing
Impairment (HI)
School for Children with
Intellectual Disability
(ID)
School for Children with
Physical Disability (PD)
T o t a l School Head’s ITEd Questionnaire (Part 1):
校長資訊科技教育問卷調查 (第一部份) (小學/中學/特殊學校) School Head’s Information Technology in Education Questionnaire (Part 1) (Primary/Secondary/Special)
P1 S1 E1-1 E1-2 4
School Head’s ITEd Questionnaire (Part 2):
校長資訊科技教育問卷調查 (第二部份) (小學/中學/特殊學校) School Head’s Information Technology in Education Questionnaire (Part 2) (Primary/Secondary/Special)
P2 S2 E2-1 E2-2 4
School ITEd Survey:
學校資訊科技教育調查(小學/中學/特殊學校)
School Information Technology in Education Survey (Primary/Secondary/Special)
P3 S3 E3 3
ITEd Team Teachers’ Questionnaire:
資訊科技教育組教師問卷調查(小學/中學/特殊學校)
Information Technology in Education Team Teachers’ Questionnaire (Primary/Secondary/Special)
P4 S4 E4-1 E4-2 4
Teachers’ ITEd Questionnaire (for both Teachers and ITEd Team Teachers):
教師資訊科技教育問卷調查(小學/中學/特殊學校)
Teachers’ Information Technology in Education Questionnaire (Primary/Secondary/Special)
P5 S5 E5 3
Students’ ITEd Questionnaire:
學生資訊科技教育問卷調查(小學/中學/特殊學校)
Students’ Information Technology in Education Questionnaire (Primary/Secondary/Special)
P6 S6 E6-1 E6-2 E6-3 E6-4 E6-5 E6-6 8
Parents’ ITEd Questionnaire:
家長資訊科技教育問卷調查(小學/中學/特殊學校)
Parents’ Information Technology in Education Questionnaire (Primary/Secondary/Special)
P7 S7 N.A. E7-1 E7-2 4
Therapists’ ITEd Questionnaire:
治療師資訊科技教育問卷調查 (特殊學校)
Therapists’ Information Technology in Education Questionnaire (Special)
N.A. N.A. N.A. E8 1
Types of questionnaires for different stakeholders with respect to the evaluative areas are presented in Table 4.2.
School Heads’ Information Technology in Education Questionnaires (Part 1 and Part 2) (School Heads’ ITEd Questionnaires)
All school heads for the three school sectors were surveyed by the School Heads’ ITEd Questionnaires, which consisted of two parts: 8 items in part 1 and 18 items (except the question item for other comments) in part 2. In the School Head’s ITEd Questionnaire (Part 1), school heads were required to reflect on the levels of satisfaction (namely ‘Very satisfied’, ‘Satisfied’,
‘Quite satisfied (一般)’, ‘Not satisfied’ and ‘Totally not satisfied’) with respect to the expected outcomes described under each area. An example is shown as follows:
Q Levels of Satisfaction
EV
&
EO
Question items in School Head’s ITEd Questionnaire (Part 1)
Very
satisfied Satisfied
Quite satisfied
(一般)
Not satisfied
Totally not satisfied 2a. 2a Teachers’ IT competency meets the requirements of
Education and Manpower Bureau.
2b. 2b School head and teachers understand the function of IT in the learning, teaching and assessment processes.
2c. 2c Teachers understand their roles when applying IT in the learning, teaching and assessment processes.
2d. 2d School head and teachers demonstrate positive appraisal on the impact of IT on learning.
School Head’s ITEd Questionnaire (Part 2) was designed to survey a broad range of facts and opinions as reflected by the school heads. The items mainly focused on the evaluative areas which were related to school heads’ perception of ITEd, school leadership, school ITEd curriculum, community-wide support and parents’ involvement, school professional development in ITEd, digital resources and infrastructure, as well as school technology-using culture and ITEd development.
School Information Technology in Education Survey (School ITEd Survey)
All schools in the three school sectors were invited to complete the School ITEd Survey. The survey was designed for collecting factual information of IT facilities and ITEd curriculum in schools. In this questionnaire, there were 8 items focusing on school basic information, school IT facilities and school computer and IT curricula. Each school was requested to assign a school representative [e.g. ITEd Team head or IT coordinator (ITC)] to complete the School ITEd Survey.
Teachers’/Therapists’ Information Technology in Education Questionnaire (Teachers’/Therapists’ ITEd Questionnaire)
Teachers of primary, secondary, and special schools were surveyed by the Teachers’ ITEd Questionnaire. Therapists in special schools were surveyed by a customized set of questionnaires based on the Teachers’ ITEd Questionnaire. Major modifications were made to match better with therapists’ job duties and involvement in ITEd in special schools. The questionnaire was designed to collect information and opinions on the following evaluative areas: their IT competency as well as their perception of ITEd, community-wide support and parents’ involvement, school ITEd curriculum, school professional development in ITEd, school leadership, digital resources and infrastructure, school technology-using culture and ITEd development, students’ learning activities with IT, pedagogical practices with IT and students’ learning outcomes. Except the question item for other comments, there were 35 questionnaire items for teachers in three school sectors and 32 questionnaire items for therapists of special schools.
Information Technology in Education Team Teachers’ Questionnaire (ITEd Team Teachers’
Questionnaire)
Those teachers who were also IT team members were invited to fill in the ITEd Team Teachers’
Questionnaire in addition to the Teachers’ ITEd Questionnaire. This questionnaire was designed to probe into the school’s adoption of ITEd in terms of the school ITEd curriculum, school professional development in ITEd, digital resources and infrastructure in schools, school technology-using culture and ITEd development, community-wide support and parents’
involvement as well as school leadership. There were 7 items in this questionnaire.
Students’ Information Technology in Education Questionnaire (Students’ ITEd Questionnaire)
Selected students in the sampled schools were surveyed by the Students’ ITEd Questionnaire. The questionnaires for primary school and secondary school students were basically the same, except for some slight variations to cater for different curriculum subjects. The questionnaires for special school students were customized by the school categories. The question items were customized in terms of the subjects, the use of special assistive devices for IT and therapists’ use of IT for their therapy and training. There were 24 items for students of primary, secondary and special schools for SSD and VI. There were 23 items for students in special school for H, 26 items for students in special schools for HI and PD and 27 items for students in special schools for ID.
Parents’ Information Technology in Education Questionnaire (Parents’ ITEd Questionnaire)
Except the question item for other comments, there were 25 items in the Parents’ ITEd Questionnaire for primary and secondary schools whereas 26 items were set for that of special schools. The items focused on the evaluative areas of community-wide support and parents’
involvement, digital resources and infrastructure as well as students’ learning activities with IT.
One special item (questions 5d and 5e) was included in the Parents’ ITEd Questionnaire for special schools with therapists, regarding the therapists’ use of IT in students’ therapy and training.
Table 4.2 Types and evaluative areas for questionnaire surveys Questionnaire types
Inside school surveys a Outside school survey Evaluative areas (EV)
School Heads (Part 1)
School Heads (Part 2)
School ITEd Survey
ITEd Team Teachers
Teachers and Therapists
Students Parents
1. Students’ Perception of learning with IT
3 3
2. Teachers’ IT Competency as well as Teachers’ &
School Heads’ Perception of ITEd
3 3 3 3
3. Community-wide Support
& Parents’ Involvement
3 3 3 3 3 3 3
4. School ITEd Curriculum 3 3 3 3 3 3
5. School Professional Development in ITEd
3 3 3 3
6. School Leadership 3 3 3 3
7. Digital Resources &
Infrastructure
3 3 3 3 3 3 3
8. School Technology-using Culture and ITEd Development
3 3 3 3 3
9. Students’ Learning Activities with IT
3 3 3 3
10. Teachers’ Pedagogical Practices with IT
3 3 3
11. Students’ Learning Outcomes in different key learning areas and the development of information literacy and generic skills
3 3 3
a Minor adjustments were made in the questionnaire for special school teachers/therapists and students.
All questionnaires were developed by the Project Team with reference to the Study Objectives, Research Questions, Literature Review and Conceptual Framework. During the developmental process, review of all questionnaires was conducted by the consultants, honorary advisers and the EMB. A trial-run was carried out by using these instruments in each stage of the Pilot Study in July 2005 and September 2005. The findings from the Pilot Study, together with experience from the field work and advice from consultants and honorary advisers, were used to refine the questionnaires. Major revisions included:
a. adding new items to the pilot questionnaire(s) to tap information related to important variables derived from the construct map;
b. modifying the wording or options to improve the clarification of questionnaire items and to enhance the appropriateness of the options given in an item; and
c. structuring the questionnaires with appropriate logical sequence and length.
Finalised questionnaires were used in both MS1 and MS2.