2.1 In line with the Committee’s recommendations, the Tryout Study covers four major initiatives as below:
(i) Improving assessment papers and question design;
(ii) Enhancing school reports;
(iii) Strengthening diversified professional support measures; and
(iv) Including a questionnaire survey on students’ learning attitude and motivation.
2.2 The above basket of initiatives is intended to eliminate the incentives for over-drilling induced by TSA; provide targeted service to enhance the overall effectiveness of the cycle of Learning-Teaching-Assessment as an integral part to benefit students; and improve communication and deepen mutual trust between schools and parents.
(I) Improving Assessment Papers and Question Design
2.3 On improving assessment papers and question design, the major directions as stated in the report submitted by the Committee in February 2016 are maintaining the reliability and validity of TSA, ensuring the modifications serve to lessen students’
burden of learning, aligning with the spirit of the curriculum and reflecting students’
basic competencies. The principles of modification are:
- learning needs of students;
- alleviating the learning burden on students;
- aligning with the spirit of curriculum; and
- adopting appropriate choice of words and phrases in assessment materials.
2.4 The Working Group on Papers and Question Design under the Committee has, from a professional perspective, reviewed the assessment papers and question design and put forward views and recommendations. In the light of the proposed review directions and principles and the specific suggestions, HKEAA has modified the assessment papers and question design for Primary 3 TSA, and strengthened the item setting and moderation work through different working groups (e.g. item development and moderation working groups and paper review focus groups) and rigorous mechanisms.
2.5 The major modifications for the three subjects of Chinese Language, English Language and Mathematics at Primary 3 level are as follows:
(i) Primary 3 Chinese Language
- The number of texts in the reading assessment is adjusted from
three to two; the total number of words per sub-paper is limited to not more than 1 200 and the number of items does not exceed 20;
and practical writing is only included in one of the sub-papers to avoid giving undue weight to practical writing;
- In the writing assessment, certain information required for practical writing is provided, such as salutation, complimentary close, greetings and date of a letter; the marking criteria on the format of practical writing are adjusted; and student exemplars demonstrating the attainment of Basic Competency are provided;
and
- “Five-options-choose-two” items, items requiring “reverse thinking” and so forth in each paper are reviewed and adjusted.
(ii) Primary 3 English Language
- The number of parts in the reading assessment is reduced from four to three. The number of words per reading task is limited to not more than 150 and the number of words of the whole paper is capped at 400;
- To help students manage the assessment time for the reading and writing papers, invigilators announce the time twice during the examination, i.e. 15 minutes and 5 minutes before the end of the examination;
- Items expecting answers in the past tense in the writing assessment are scrapped, such as writing a recount; and
- Assessment items on basic book concepts are avoided.
(iii) Primary 3 Mathematics
- The number of items is reduced, with an immediate cut of around 20%;
- Only one Basic Competency is assessed in each item; and - Items requiring solving linking problems are minimised.
2.6 Under the Tryout Study, the oral assessments and written assessments were completed on 3 and 4 May, 14 and 15 June 2016 respectively. A total of more than 50 primary schools participated. Participating schools are from different regions (Hong Kong Island, Kowloon, New Territories East and New Territories West), of various types (government, subsidised, Direct Subsidy Scheme and private schools), and school sizes. The number, distributions and proportions of participating schools closely align with the design.
2.7 For the handling of data, the distributions and proportions of participating schools are taken into account, and statistical methodologies are adopted to process and analyse the related data to ensure the representativeness of the result.
2.8 TSA is an objective assessment tool with reliability and validity in assessing students’ overall Basic Competencies in the three subjects of Chinese Language, English Language and Mathematics upon completing the stages of Primary 3, Primary 6 and Secondary 3 education. To maintain the standards set, HKEAA conducts a research test to maintain the stability and consistency of the standards.
Meanwhile, the improved assessment papers and items, are implemented in accordance with the Basic Competency standards set in 2004, including standard setting, standard maintenance and estimation of students’ ability indices.
2.9 In order for the public and schools to promptly grasp and understand the rationale of question design for Primary 3 TSA under the Tryout Study, instead of following the usual practice of uploading relevant materials upon the release of TSA results, HKEAA has, right after the completion of Primary 3 assessment, uploaded to the website on Basic Competency Assessments (www.bca.hkeaa.edu.hk) the question papers of Chinese Language, English Language and Mathematics, suggested answers together with the information on item design (e.g. the corresponding key learning objective, Basic Competency and question intent), as well as the marking schemes.
2.10 All the Markers and Assistant Examiners were qualified serving teachers.
Markers’ Meetings were conducted by HKEAA in July to familiarise Markers with the modified marking schemes and related principles. All script marking and preliminary data processing were completed in July and August respectively. Data analysis and report writing were completed by the end of October, and school-level data was released to schools on 9 November to provide feedback to and improve learning and teaching.
2.11 After the release of the results, seminars on students’ performance in Chinese Language, English Language and Mathematics were organised on 14 and 15 November. In addition to deliberating the overall performance of students, various exemplars were also used to illustrate in detail the strengths and weaknesses of students, with the aim of facilitating the understanding of teachers of both participating and non-participating schools in the Tryout Study to understand students’ performance to facilitate the concept of “assessment for learning”.
(II) Enhancing School Reports
2.12 On enhancing the format of school reports, in order to enable schools to make better use of TSA data to benefit learning and teaching, the Working Group on Administration and Reporting under the Committee recommends that four types of reports with different coverage should be made available for schools’ selection to meet the needs of individual schools.
2.13 The four types of assessment reports are as follows: (Using Primary 3 Mathematics as an example)
(i) Existing version
Basic Competency Item
no. Option School percentage
Territory-wide percentage Perform mixed
operations of: (a) Addition and subtraction (with numbers up to 3 digits, involving small brackets);
(b) Multiplication and addition; (c) Multiplication and subtraction.
Q08 A
B C D
54.6%
15.8%
20.5%
9.1%
73.6%
6.9%
5.8%
13.7%
(ii) Simplified version
only provides data of an individual school without the overall data for reference purposes
Basic Competency
Item
no. Option School percentage
Territory-wide percentage Perform mixed
operations of: (a) Addition and subtraction (with numbers up to 3 digits, involving small brackets);
(b) Multiplication and addition; (c) Multiplication and subtraction.
Q08 A B C D
54.6%
15.8%
20.5%
9.1%
N.A.
N.A.
N.A.
N.A.
(iii) Integrated version
a consolidated report on Basic Competencies by item groups and provides exemplars on students’ overall performance
Learning unit Average of percentage of students answering items correctly
Mixed Operations School
All schools participating in the Tryout Study (iv) Information
analysis report
provides the corresponding key learning objective, Basic Competency and question intent of each item, as well as an analysis of options of multiple-choice items
Basic Competency
Item no.
Option (Information
Analysis)
School percentage
Territory-wide percentage Perform
mixed operations of:
(a) Addition and subtraction (with
numbers up to 3 digits,
Q08 A. Correct Answer B. Forgot
decomposition C. Performed the
subtraction only D. Not able to
master the rule of
54.6%
15.8%
20.5%
73.6%
6.9%
5.8%
involving small brackets);
(b)
Multiplication and addition;
(c)
Multiplication and
subtraction.
‘multiplication before subtraction’
9.1% 13.7%
2.14 Schools participating in the Tryout Study may, in the light of school-based needs and on a subject basis (Chinese Language, English Language and Mathematics), choose the reports that they consider appropriate in helping them analyse the performance of students. They are able to grasp the learning progress of students by comparing the data provided in the report with the descriptions and exemplars of students’ performance in the 2016 Territory-wide System Assessment Report (Chapters 6 to 8) on the HKEAA website.
2.15 The types of reports chosen by schools are as follows:
Type of report Chinese Language
English Language
Mathematics
i Existing version About 90% About 90% About 90%
ii Simplified version About 3% About 3% About 3%
iii Integrated version About 90% About 90% About 90%
iv * Information analysis report 100% 100% 100%
* All schools participating in the Tryout Study received the information analysis report.
(III) Strengthening Diversified Professional Support Measures
2.16 Regarding professional support measures, under the Tryout Study, schools participating in the Tryout Study may, in the light of school-based needs, opt for one or more of the following professional support measures:
(i) Workshops on better use of assessment strategies and enhancement of learning and teaching
Part 1:
- Understanding what Basic Competencies are and the importance of progressive learning
- Designing quality assessment tasks/items
- Adopting diversified assessment strategies/approaches to help students achieve different learning outcomes
- Formulating school-based assessment/homework policies and measures
Part 2:
- Making optimal use of assessment data to provide feedback to learning and teaching
(ii) School-based support services
- Application by schools on a need basis
- Analysing TSA school reports provided under the Tryout Study (on-site services)
(iii) Developing teaching and assessment materials in collaboration with tertiary institutions
- Developing teaching and assessment materials and designing learning activities in collaboration with tertiary institutions and schools participating in the Tryout Study
- Trying out Web-based Learning and Teaching Support (WLTS) materials and Student Assessment Repository (STAR)
(iv) Parent education
- parent education activities are co-organised by EDB and schools participating in the Tryout Study. The forms of activities are determined in the light of the needs of individual schools.
(i) Workshops on better use of assessment strategies and enhancement of learning and teaching
2.17 Four Regional Seminars on Enhancing School Homework Policy were organised by EDB in February and March 2016 targeting primary school principals, vice principals, curriculum leaders and teachers responsible for homework policy at school. Four primary schools were invited to share their successful experiences and implementation strategies regarding homework policy in the seminars. Ways to devise an appropriate homework policy based on school backgrounds and students’
learning needs were also discussed with primary school leaders. It is hoped that home-school cooperation can be promoted through exchanges among schools in the seminars to alleviate students’ pressure derived from handling homework, allowing enough time for students to rest and participate in activities beneficial to their physical and psychological development so as to achieve the target of whole-person development.
Diagram 3 Workshops on better use of assessment strategies and enhancement of learning and teaching
(ii) School-based support services
2.18 Regarding school-based support services, on-site support is provided by EDB to schools participating in the Tryout Study in the light of their school-based needs.
Over 80% of the participating schools selected school-based support services that could address their specific needs. Support officers collaborate with schools to determine development focuses by analysing TSA data and students’ performance in schools and using pre-tests to diagnose students’ learning needs. Lesson observation, lesson studies, student interviews, etc. are conducted to assess and analyse students’
learning outcomes with the aim of adjusting teaching plans and strategies. See Diagram 4 for details of areas of support.
Diagram 4
#Some schools receive more than one support services
2.19 The framework of school-based support services in enhancing assessment literacy is shown in Diagram 5.
Diagram 5
(iii) Developing teaching and assessment materials in collaboration with tertiary institutions
2.20 This support measure allows schools participating in the Tryout Study and tertiary institutions to jointly develop teaching and assessment materials and design teaching activities. The major implementation framework is shown in Diagrams 6 and 7, with adjustments and modifications made by individual subjects based on school-based needs.
Diagram 6
Diagram 7
(iv) Parent education
2.21 Under the Tryout Study, participating schools will work with EDB to prepare parent education programmes. The topics will include fostering assessment literacy and promoting students’ self-directed learning, etc.
(IV) Including a Questionnaire Survey on Students’ Learning Attitude and Motivation
2.22 In line with the Committee’s recommendations, a questionnaire survey is included under the Tryout Study to collect students’ non-academic data (e.g. time spent on extra-curricular activities, learning interests, learning habits and other
relevant data) so as to gain a better understanding of the factors affecting learning performance and to provide further assistance for student learning. The questionnaire survey also collects the views of schools and parents on homework load.
Following the Government’s established procurement procedures, upon an open tender exercise, the Chinese University of Hong Kong (CUHK) was commissioned by EDB to design and conduct the questionnaire survey on students’ learning attitude and motivation.
2.23 Under the Tryout Study, CUHK invited schools participating in the Tryout Study to take part in the questionnaire survey and prior consent was obtained from related parties. The survey respondents were schools, parents and students (including Chinese speaking students and non-Chinese speaking students).
Eventually, more than 50 primary schools participated in the survey and more than 4 000 questionnaires were received from Primary 3 students and their parents. The questionnaire was designed with reference to international studies on the same issue, and the questions were meant to get a picture of the learning situation in Hong Kong.