EDB representatives summarised this meeting:
It is a frank and constructive meeting. Suggestions by Concern Group will be passed to the Coordinating Committee on Basic Competency Assessment and Assessment Literacy for discussion, including the issue of over-drilling. It is hoped to be able to work out positive tackling strategies;
Both sides admit TSA’s positive role;
It is agreed that consultation on the arrangements of 2017 TSA should be as transparent as possible.
Annex 6 Hong Kong Examinations and Assessment Authority
Education Assessment Services Division 2016 Tryout Study (Primary 3)
Notes of Focus Group Meeting (Primary 3 Chinese)
Date: 24 November 2016 (Thursday)
Time: 3:00-5:00 pm
Venue: Room 502, Tseuk Luk Street, Hong Kong Examinations and Assessment Authority
Participants: Chinese subject teachers (4), Chinese Panel Heads (31), HKEAA representatives (2)
Organiser: Hong Kong Examinations and Assessment Authority Summary
(1) Enhanced Report Formats
The information given in the “Basic Competence Reports by Item Groups”
and “Information Analysis Reports” was welcome by teachers.
Teachers expressed that the “Information Analysis Reports” could provide detailed explanation to each distractor of multiple-choice (MC) items. This kind of analysis (including the analysis of items, reading passages and students’ learning difficulties) was done by teachers themselves in the past.
However, the “Information Analysis Reports” provided in this year could facilitate teachers’ better understanding of students’ strengths and weaknesses. This report could also reduce the workload of teachers and provide effective feedback on learning and teaching.
Teachers found that both “Basic Competence Reports by Item Groups” and
“Information Analysis Reports” have their own functions. “Basic Competence Reports by Item Groups” enabled principals, PSM(CD) and subject panels to understand the overall performance of their students on each BC. Moreover, the “Information Analysis Reports” could help subject teachers identify students’ learning difficulties.
Individual teachers suggested providing an explanation to each correct answer and the reports should cover all sub-papers.
Individual teachers expressed that they had not yet accessed the TSA reports (viewing of TSA reports requires access rights given by the school head).
Therefore, they found it hard to do follow-ups for their students’ learning difficulties. They recommended that more promotional seminars on the
purpose and rationale of TSA should be held for principals. These seminars could also help to remind principals to deliver the reports to their teachers as well as to create user accounts for teachers to login to the online item analysis report. As a result, teachers were able to analyse the data so that they could reformulate the ‘learning and teaching’ programmes.
(2) Others
Teachers found that TSA is a systematic, transparent and low-stakes assessment. It provides comprehensive data and detailed reports which enable schools to have a better understanding of students’ performance on CEM subjects. TSA can also help promote “assessment for learning” so as to give feedback on learning and teaching. However, some teachers expressed that Hong Kong Pre-Secondary 1 Attainment Test is not transparent enough and there is not much information provided to primary schools. Teachers did not know that there had been changes in the Pre-S Secondary 1 assessment design. They only knew the changes after their students had taken the test and told the school about these changes.
Moreover, since Pre-Secondary 1 is a high-stakes assessment which is related to SSPA, schools can only get limited information about it and they thought that having little information about the Pre-S Secondary 1 may have exerted much pressure on them.
Some teachers said that some parents did not fully understand what TSA was and would not allow their children to participate in the TSA. Or they would tell their children to take the assessment in an erratic manner.
Teachers were worried that this might affect the reliability and validity of the TSA. They suggested that EDB should enhance the communication with parents and promote the purpose and functions of TSA.
Some teachers hoped that they could receive two sets of data from the oral assessment, one set of data where Putonghua (PTH) was assessed and the other set where Cantonese was assessed. This helped facilitate schools’
analysis and comparison of students’ performance. The HKEAA representative responded that oral assessment assesses students’ spoken ability. Thus, no matter whether students used PTH or Cantonese to take the assessment, they should be rated according to the same marking scheme.
Some teachers said that students could only listen to the script once in the existing listening assessment. They asked if it would be possible for students to read the questions before listening to the dialogue. The HKEAA representative responded that since the listening assessment had been developed according to the BC requirements, (e.g. to memorise the
content, to comprehend the cohesive relationship between dialogues and to identify the tone of the speakers), these requirements were applicable to students’ daily use. Students would not have an outline beforehand in their daily use and they needed to respond naturally in a given language environment. Despite this, the HKEAA representative mentioned that they would be well aware of the difficulty and suitability of items when developing listening items.
Annex 7 Hong Kong Examinations and Assessment Authority
Education Assessment Services Division 2016 Tryout Study (Primary 3)
Notes of Focus Group Meeting (Primary 3 English)
Date: 25 November 2016 (Friday)
Time: 3:00-4:30 pm
Venue: Rooms 2 & 3, Stelux House, San Po Kong, Hong Kong Examinations and Assessment Authority
Participants: English subject teachers (11), English Panel Heads (24), PSMCD (1), HKEAA representatives (2)
Organiser: Hong Kong Examinations and Assessment Authority Summary
Feedback on School Report, Item Analysis Report (sorted by Basic Competency (BC) and by sub-papers)
For the existing version of the reports, teachers found them useful as the reports provided information on the performances of students. The data in the Item Analysis Report (sorted by BC) was especially useful to teachers since the performances of students under each Question Intent were shown. This helped teachers understand the strengths and weaknesses of their students in listening and reading papers.
Removal of the BC attainment rates from the school reports helped to release pressure on schools and teachers.
For the simplified version of the reports, some teachers commented that they would prefer keeping the territory percentages. They thought the data was useful to schools as it would provide information about how their students performed in comparison with those of the territory. They would use the data for reference only. They did not feel much pressure if the data was given to them.
One teacher suggested putting the data of last year in the reports for easy reference.
Feedback on Online Item Analysis (OIA) Reports
The OIA Reports were very user-friendly to teachers, in particular to subject panels. They would use the excel files to compile data to facilitate their analysis of students' performances in different skills.
The bar charts showing the performances of students in the recent three years in the reports were very useful to teachers. This helped facilitate the planning of teaching and learning in the second term as well as the whole curriculum for next year.
The writing exemplars provided in the reports were also very useful to teachers.
The exemplars could help teachers’ understanding of the requirements of the writing tasks, thus enhancing learning and teaching at schools.
Feedback on Basic Competency Report by Item Groups
In this report, teachers found it difficult to interpret the information with the triangles given on the performance scale. It was suggested that school percentages as well as those of the tryout schools should be marked with the triangles so that it would help to interpret the performances of students.
One teacher commented that the ‘lower triangles’ which indicated the performances of students of tryout schools were not useful as this would encourage comparison of performances among schools.
The exemplars and the quantitative feedback given in the reports provided detailed information about how students performed in different aspects (e.g. use of cohesive devices). Teachers would then sort out the type of items which students did not do well and plan for remedial teaching. This would facilitate effective learning and teaching.
Teachers suggested that this report should be given to all schools in future.
One teacher suggested that information about school types should be given in the report (e.g. aided, direct subsidy, government).
Feedback on Information Analysis Report
The qualitative feedback in the Information Analysis Report was very useful to teachers. It provided teachers with information about the possible problems in students’ learning. The information would best benefit teachers with less teaching experience.
The presentation of the diagnostic information in each option was very clear and useful. The wordings were easy to understand. No additional information was needed for the correct answer.
Panel teachers would make use of both the qualitative feedback and the data given in the report to analyse students’ performances in the assessment and share the findings with other subject colleagues. They would then plan their teaching ahead so as to help students improve.
Some teachers suggested that the diagnostic information should be provided for all the papers in future.
In order to facilitate teachers’ analysis of the information given in this report, it was suggested that the items should be presented in two groups:
- items showing good performances of students; and - items showing weak performances of students.
Teachers found that it would be even more useful for their analysis if the data of students with Special Educational Needs (SEN) could be provided separately in another column in the report.
The tapescripts of each listening task should be put in the report for easy reference.
Annex 8 Hong Kong Examinations and Assessment Authority
Education Assessment Services Division 2016 Tryout Study (Primary 3)
Notes of Focus Group Meeting (Primary 3 Mathematics)
Date: 28 November 2016 (Monday)
Time: 3:00-5:00 pm
Venue: Room 502, Tseuk Luk Street, Hong Kong Examinations and Assessment Authority
Participants: Mathematics subject teachers (12), Mathematics Panel Heads (26), HKEAA representatives (4)
Organiser: Hong Kong Examinations and Assessment Authority Summary
Newly added assessment reports
The current report format (with school and territory-wide percentages) and the new report formats (simplified version with school percentage only, integrated version and the information analysis report), could provide feedback to enhance teaching and learning for primary schools.
Teachers agreed that the new report formats can reduce the incentives for drilling. However, the provision of school-based examples in the integrated version would be more helpful to individual schools.
The information analysis report could provide an analysis of options of multiple-choice items, which facilitated teachers’ analysis of students’ common mistakes to provide feedback to learning and teaching. In addition, timely diagnostic information for non-MC items was also requested to teachers.
The four types of reports provided effective and sufficient information. Most teachers agreed that the inclusion of both school and territory-wide percentages of students answering the questions correctly in the integrated report for teachers’ reference would be desirable.
Teachers recommended that the questions of TSA be screened in the interactive reporting platform based on Basic Competencies, units or areas of learning
A “PRINT” button could be added to the 3-year bar chart.
Others
Teachers suggested that students could use the Student Assessment mode so that students could select the assessment topics of previous years as exercises.
Teachers have requested school-based curriculum development and teaching support in May 2016. They hoped that there would be more support in assessment literacy. For example, one of the teachers mentioned that the AQP platform of the Hong Kong Examinations and Assessment Authority could serve as a tool for question analysis. Other teachers hoped free software for school use would be provided.
Teachers believed that the difficulty of 2016 Mathematics was appropriate, and could provide useful feedback data to schools.
Annex 9 2016 Tryout Study (Primary 3)
Professional Sharing Session with Principals
Date: 8 December 2016 (Thursday)
Time: 4:00-6:00 pm
Venue: Conference Room 6, G/F, Central Government Offices Participants: Principals (23), Vice Principals (5), Middle managers (2),
EDB representatives (5), HKEAA representatives (2) Organiser: Education Bureau
Summary