Chapter 4 Research Design and Methods
4.3 Data Analysis Methods
This section introduces the quantitative and qualitative data analysis methods used in this Study.
4.3.1 Analysis of Quantitative Data
Quantitative data refers to the data collected from the questionnaire surveys for various stakeholders. The statistical analysis methods of the quantitative data and relevant weighting measures are described in this section.
4.3.1.1 Questionnaire Surveys
The data collected from questionnaire surveys were analysed by school sector, i.e. Primary, Secondary and Special. In order to address the Research Questions, questionnaire items were categorized into eleven evaluative areas according to the conceptual framework defined in this Study as follows:
EV1: Students’ Perception of Learning with IT
EV2: Teachers’ IT Competency as well as Teachers’ & School Heads’ Perception of ITEd
EV3: Community-wide Support & Parents’ Involvement EV4: School ITEd Curriculum
EV5: School Professional Development in ITEd EV6: School Leadership
EV7: Digital Resources & Infrastructure
EV8: School Technology-using Culture and ITEd Development EV9: Students’ Learning Activities with IT
EV10: Teachers’ Pedagogical Practices with IT
EV11: Students’ Learning Outcomes in Different Key Learning Areas (KLAs), Information Literacy and the Development of Information Literacy and Generic Skills
The items were mainly in the form of ‘Single Selection’, ‘Multiple Selection’ and ‘5-point Likert Scale’. All raw data files were exported from the “SEP on ITEd for Schools”. The statistical analysis was conducted by using the SPSS. Descriptive statistics such as total number of valid responses (N) and percentage distributions together with base number of respondents were reported for categorical or dichotomous data. For ordinal and numeric data, more statistical information including the total number of valid responses (N), mean (M), standard deviations (SD) and the standard error of means (SE) were computed. The extent of sampling error for this Study was set at 95% confidence level. Thus, the upper and lower limits of confidence interval were also reported [Upper 95% confidence limit (UCL) and Lower 95% confidence limit (LCL)]. The Listwise strategy was adopted for the missing data in the School Head’s ITEd Questionnaire (Part
1). Overall summary of descriptive analysis of the questionnaire surveys for MS1 and MS2 was documented by school sectors in Appendices 1 and 2 respectively.
In order to track the progress of the implementation of ITEd from MS1 and MS2, Mann-Whitney U Test and Chi-square Test were used to test the significant difference in selected ordinal and categorical question items of different stakeholder surveys between MS1 and MS2 respectively.
4.3.1.2 Weighting Measures
Weightings were applied to the Students’ ITEd Questionnaire and the Parents’ ITEd Questionnaire to adjust for the design of the two-stage cluster sampling method6 based on the direct proportion to the inverse of the selection probability of each student or parent in the sampled schools. Both the size of schools and classes were taken into consideration. Weightings for the subgroup analysis were performed in the Students’ ITEd Questionnaire data, while weightings for the combined group analysis were performed in the Parents’ ITEd Questionnaire data. The weighting was formulated as follows:
xi is the total number of students of the class level of the sampled school,
yi is the number of respondents of the selected class of the class level of the sampled school,
The weighting score of a class level of a sampled school
=
∑
∑
=
× =n i
i n
i i
i i
x y y
x
1
1 where
n is the total number of sampled schools of a school sector.
No weighting was applied on full enumeration and one-stage cluster sampling7, including School Heads’ ITEd Questionnaires (Part 1 and Part 2), School ITEd Survey, ITEd Team Teachers’
Questionnaire and Teachers’ ITEd Questionnaire (including Therapists’ ITEd Questionnaire for special school sector).
4.3.1.3 Reliability
Based on the data from MS1, reliability test was conducted for School Heads’ ITEd, ITEd Team Teachers’, Teachers’ ITEd, Students’ ITEd and Parents’ ITEd Questionnaires of primary and secondary school sectors using SPSS version 12. However, reliability test was not required for School ITEd Survey which was intended for garnering the factual data for school IT facilities and information on school IT curriculum only. In addition, no reliability test was conducted for the raw data of different stakeholders in special schools as the sample sizes of different stakeholder groups were too small.
6 Two-stage cluster sampling method: one class of students per selected class level was randomly selected from the sampled schools.
7 One-stage cluster sampling method: all the stakeholders in respective groups in a sampled school were invited to do the respective questionnaires.
The reliability test results for School Heads’, ITEd Team Teachers’, Teachers’, Students’ and Parents’ ITEd Questionnaires of primary and secondary school sectors are reported in Table 4.3.
Table 4.3 Reliability test results for School Heads’, ITEd Team Teachers’, Teachers’, Students’ and Parents’ ITEd Questionnaires of primary and secondary school sectors
Questionnaire types Primary school sector Secondary school sector
No. of
items No. of respondents
(Valid) a
Cronbach’s Alpha Reliability
No. of
items No. of respondents
(Valid) a
Cronbach’s Alpha Reliability School Head’s ITEd Questionnaire
(Part 1) 53 551 0.968 53 397 0.964
School Head’s ITEd Questionnaire
(Part 2) 55 539 0.835 55 (375) 390 0.809
ITEd Team Teachers’ Questionnaire 57 334 0.886 57 339 0.894
Teachers’ ITEd Questionnaire 136 (1481) 1830 0.766 136 (2136) 2727 0.763 Students’ ITEd Questionnaire 96 (2831) 3739 0.814 96 (5200) 6189 0.806 Parents’ ITEd Questionnaire 37 (2831)
3340 0.856 37 (5203) 5891 0.846
a Listwise deletion based on all variables in the procedure
4.3.2 Analysis of Qualitative Data
There were three sources of qualitative data in this Study: the EMB documents, the textual responses from questionnaire surveys (if applicable) and the interviewing notes or transcripts from the interviews of the representatives of the Community Group. Qualitative data analysis consisted of a series of data clustering process which involved “three concurrent flows of activities:
data reduction, data display and conclusion drawing or verification” (Miles & Huberman, 1994, p.10). The purpose of this data clustering process was to identify patterns or categories and relationship that might emerge from the data.
4.3.2.1 Review of EMB Documents
A set of documents relating to various ITEd initiatives or projects, launched from 2004/05 to 2006/07 school years, provided by the EMB was reviewed. The documents were classified according to the seven strategic goals and examined based on the research questions and evaluative areas.
4.3.2.2 Open-ended Items in Questionnaire Surveys
Textual responses, such as ‘other comments’ from questionnaire surveys for school heads, teachers and parents, were collected. However, no textual response was collected from students for easy administration of online survey owing to time limitation and IT competency requirement for textual input. The data of textual responses were clustered into different categories (dimensions, evaluative areas, themes and strategic goals). Responses or answers might include additional information or suggestions other than the evaluative areas. Theme(s) would then be defined. A table showing the frequency of occurrence of each category was used to summarize the information gathered. Conclusion was then drawn to explain the relationships or meaning of the data.
4.3.2.3 Focus Group Interviews
Nine focus group interviews were conducted parallel to the data collection periods of MS1 and MS2. There were two sources of qualitative data for the Community Group Interviews:
y documents including the background and missions of the organisations related to ITEd; and y interviewing notes or transcripts from the interviews of the representatives of the Community
Group.
The documents for analysis were collected through relevant websites, by mail(s) or e-mail(s) to the selected organisations. The main purpose was to supplement to and triangulate with the data collected through interviews. The main ideas from group discussions were transcribed. A transcript was classified and coded with the evaluative areas, themes and strategic goals developed, as well as any new topics or issues. At the end of this analysis process, the data was clustered into different categories (dimensions, evaluative areas, themes and strategic goals). A table showing the frequency of occurrence of each category was used to summarise the information gathered.
Conclusion was then drawn to explain the relationships or meaning of the data.