Field Observations on Performance Assessments

This chapter reports on the general observations of the Main Study Performance Assessments (PAs).

The first part of the chapter reports the problems and issues observed during school visits of the project implementation. The second part is related to the findings on the availability of peripherals and settings in computer rooms.

4.1 Problems in Relation to IT Infrastructure in Schools

4.1.1 Number of Computers

It was mentioned in the Strategy document that improving IT infrastructure in schools was one of the seven strategic goals. However, during the project implementation, it was found that 7.5% of the primary schools (i.e. 3 out of 40) which joined the project did not have enough computers inside the computer rooms for every student of the same class to complete the online assessment at the same time. The student-to-computer ratio in these surveyed classes was about 2:1. The insufficiency of computers has affected the implementation of the project. Only half of the students could take part in the PAs while the other half did not. Teachers in these schools also expressed that students needed to share computers with their classmates during normal lessons.

4.1.2 Hardware and Software Updating

During school visits, it was found that about 5% (i.e. 2 out of 40) of the primary schools had problems on upgrading some basic security systems. As the Project Team did not know such problem before the visits, the schedule of the project implementation was affected. For example, one of the primary schools installed the old version of firewall (version 5 instead of version 25) which could not support heavy loading per second. This created a big obstacle for the students in doing the PAs. As a result, there was an unexpected long waiting time during the assessment which affected the performance of students. The Project Team needed to terminate the assessment to solve the problems and arrange for another assessment schedule.

The servers in schools were another problem. 5% (i.e. 2 out of 40) of the primary schools, 3% (i.e. 1 out of 33) of the secondary schools had servers installed with very old Windows NT operating systems which did not support the remote desktop client to access the Project Team’s machine.

Re-configuration and installation had to be done by our technical staff to solve the problem.

4.1.3 School Network and Standard in School Network Setting

20% (i.e. 8 out of 40) of the primary schools and 18.2% (i.e. 6 out of 33) of the secondary schools reported that the network speed was extremely slow and had frequent network disconnection during the assessment. Contingency measures had been taken such as rearranging another date(s) for the assessments or giving extra time for the students to work on the PA tasks to compensate for the time

of disconnection. It was investigated that the problem might be caused by:

• Busy school network, especially when the PAs were conducted on school activity days;

• There was no standardized school network setting. For example, it was observed that some schools might use inappropriate network switch (e.g. network switch for home usage) to connect the Internet amongst different computer rooms in schools. Such network switch might be out of function and be disconnected with the school network when there was heavy network traffic within schools.

4.2 Problems Related to Technical Support in Schools

It was observed that the technical staff in primary schools in general had adequate knowledge in supporting the daily routine work but they were not skillful enough in handling some new and emerging technology challenges. Before the implementation of the Main Study, schools were asked to set up the connection to the Project Team’s server but problems were encountered in schools.

About 10% (i.e. 4 out of 40) of the technical supporting staff in primary schools and 3% (i.e. 1 out of 33) of those in secondary schools had problems in setting up the connection even though clear instructions were given. Besides, we were also informed that the Internet service providers (ISPs) had given a set of school network accounts to persons in charge of network systems in schools.

However, if network management problems occurred, most of the primary schools would simply call the ISPs to solve the problem.

4.3 Problem Related to Third Party System Integrator Maintaining School Network

Apart from the ISPs which provided networking services to schools, it was found that about 17.5%

(i.e. 7 out of 40) of the primary schools and 24.24% (i.e. 8 out of 33) of the secondary schools involved a third party System Integrator (SI) for the setup and/or maintenance of their internal school networks such as DNS, firewall, etc. This created problems with the management of school network. It was because the schools did not have the administrative right to manage the school networks. The administrative rights were handed over to the SIs. If network problems occurred, schools needed to seek help from the SIs and could not solve the problems immediately by themselves.

4.4 Problems Related to Technical Skills of Students

It was observed that students’ technical skills varied a lot in both primary and secondary schools.

Result from the invigilator report indicated that 47.5% (i.e. 19 out of 40) of the invigilators of primary schools, 33.3 % (i.e. 11 out of 33) of those in secondary schools and 25% (i.e. 1 out of 4) of

those in special schools reported that students had difficulties in using some common computer applications such as inserting a new slide of PowerPoint, image settings, aligning text and drawing tables by using MS Word during the assessment.

4.5 Problems Related to Competency in Typing Chinese Characters

During the PAs, both primary and secondary schools students were required to answer some questions in Chinese. Results from the invigilators’ reports indicated that about 60% (i.e. 24 out of 40) of the primary schools had students who were incompetent in typing Chinese characters and had to input Chinese with a writing pad. For the secondary schools and special schools, 24.2% (i.e. 8 out of 33) and 25% (i.e. 1 out of 4) of the invigilators reported that students needed to use Chinese writing pad for Chinese input respectively.

In sum, 87.5% (i.e. 35 out of 40) of the primary schools, 63.64 % (i.e. 21 out of 33) of the secondary schools and 25% (i.e. 1 out of 4) of the special schools had encountered at least one of the problems mentioned in sections 4.1 to 4.5 (For details, please refer to Appendices 4.1-4.3).

4.6 Availability of Peripherals

During the project implementation, the Project Team observed that the school visits involved could provide good opportunities for the Project Team to look at the actual settings of the computer rooms and respective peripherals available which might be the factors affecting the use of ICT in learning and teaching. Therefore, the Project Team initiated a small scale survey in altogether 22 primary, 16 secondary and 2 special schools some time after the first month of the data collection for the Main Study. The Project Team did not employ any sampling procedures and measures for this small scale survey. Those schools were just the remaining sampled schools during the project implementation.

As not all project schools were included in the survey, the results could not be generalized for territory-wide schools. Invigilators of those selected schools were required to fill in the survey form (Appendix 4.4) regarding the setting of the computer room, availability of peripherals, details of operating system and kinds of hardware in the computer room. Table 4.1 shows the results of the survey.

Two types of the computer room setting were identified. They were the traditional one of which computers were arranged in rows or columns and innovative one where computers were arranged in U-shape or other settings with enough space and flexibility in room arrangement to facilitate group work or discussion. The result was that the majority (82% in primary and 87.5% in secondary schools) of the computer room settings were “traditional setting”. It was found that half of the surveyed special schools arranged their computers in traditional way and half in innovative way (i.e.

emerging setting).

It was observed that about 77.27% of the surveyed primary schools had writing pads for students but only 12.50% of the secondary schools were equipped with such peripheral. There was no writing pad in special schools.

68.18% of the primary schools had earphones in the computer rooms while for the secondary schools and special schools, only 50% of them had installed earphones in computer rooms. It was found out that most of the schools (around 80%) used the same model of computers in the computer rooms while others used a mixture of different models. 50% of the surveyed special schools used LCD monitors while only 31.82% and 37.50% of the primary and secondary schools reported using this peripheral respectively. Over half of the surveyed schools had at least 2 printers installed in computer rooms. 50% of the primary and special schools had scanners in their computer rooms while higher percentage (75%) was found in the secondary schools. Concerning the operation system, most of them were using Windows XP. Regarding the version of MS Office, majority of the primary schools (54.55%) used MS Office 2000 while majority of the secondary schools (50%) used MS Office 2003. For the special schools, 50% of schools used MS Office 2003 and 50% of them used MS Office 2000.

Table 4.1 Percentage of schools on the availability of different peripherals

Survey items Primary Schools (%) Secondary Schools (%) Special Schools (%)

Room setting - traditional setting 82.00 87.50 50.00

- emerging setting 18.00 12.50 50.00

Writing pad 77.27 12.50 0.00

Earphone 68.18 50.00 50.00

Same model of computers 86.36 81.25 100.00

LCD monitor 31.82 37.5 50.00

Have 2 printers or above 63.64 68.75 50.00

Scanners 50.00 75.00 50.00

Operation system Windows 97 0.00 6.25 0.00

Windows 2000 27.27 18.75 50.00

Windows 2003 9.09 0.00 0.00

Windows 2006 4.50 0.00 0.00

Windows XP 50.00 75.00 50.00

Windows NT 4.50 0.00 0.00

Mix 4.50 0.00 0.00

MS Office MS Office 2003 31.82 50.00 50.00

MS Office XP 13.64 12.50 0.00

MS Office 2000 54.55 37.50 50.00