Workshop on “Application of the Assessment Program for Affective
and Social Outcomes (2nd Versio n)”
(Primary Schools)
Unit One
In this unit, participants will learn the f ollowing:
The background, structure and guiding princ iples of APASO
The core functions of the E-platform for Sc hool Development and Accountability (ESDA)
The guidelines for selecting and using scal
es and subscales
Unit Two
In this unit, participants will learn the f ollowing:
How to publish APASO-II user-defined and pr e-defined surveys using “online” and “of fline” modes in ESDA
How to complete APASO-II surveys in “onlin e mode”
How to check the status of implementation o
Unit Three
In this unit, participants will learn the f ollowing:
How to review different data reports
How to interpret the data from different re ports
Summarize the learning issues and share the
experience
Background
To provide support to schools for self-evaluation, EDB la unched the Assessment Program for Affective and Social Outcomes (APASO) in 2003
Work was conducted from 2008 to 2010 to review, validat e and revise APASO. New scales were added and the nor ms of all scales were updated
The final product becomes the 2nd version of Assessment Program for Affective and Social Outcomes (APASO-II)
Purposes
It supports schools to obtain objective student performance d ata in comparison to related indicators for self-evaluation. Thi s instrument is intended to measure a group of students or the whole school student body rather than individuals
Schools can gain an in-depth understanding of students' perfo rmance and development in the affective and social domains.
This facilitates the evaluation of the impact of support service s and activities on the students and then the formulation of an action plan for improvement
Parents can also get information on students' performance in t hese domains through such channels as parent meetings and s chool reports
Scales and Subscales
Scale Name Subscale Name
Self
General School
Mathematics
Self-Concept Parent Relationships
Peer Relation
Physical Appearance
Reading Self-others
Care for Others
Interpersonal Relationships Inappropriate Assertiveness *
Respect for Others Self-school
Achievement
Experience
General Satisfaction Attitudes to School (Quality of School Life) Negative Affect *
Scales and Subscales
Scale Name Subscale Name
Self-school
Affiliation
Competition
Effort
Praise
Motivation Social Concern
Social Power
Task
Token
Failure Ability Attribution *
Failure Effort Attribution *
Causal Attribution Failure Strategy Attribution *
Success Ability Attribution
Success Effort Attribution
Success Strategy Attribution
Creative Thinking
Learning Competency Critical Thinking
Scales and Subscales
Scale Name Subscale Name
Self-school
Academic Affect
Academic Initiation
Academic Monitoring
Academic Self Concept
Change to Improve
Costs of Help Seeking * Independent Learning Capacity Education Aims
Goal Setting
Inquisitiveness
Strategic Help Seeking
Study Environment Control
Scales and Subscales
Scale Name Subscale Name
Self-society
Code of Conduct
Commitment
Attitudes toward the Nation
Values Perseverance
Sense of Responsibility
Social Harmony
Well-behaved
Purposes of Taking APASO II:
What is the purpose of conducting an APASO-II survey?
What sort of data are collected from APASO-II?
Is it to understand certain affective and social behavior of students?
Or do you want to understand the impact of a certain inter vention or a program?
Administration Guidelines
It is not advisable to use the same tool too often. The recomm ended interval between two surveys is at least six months. Pre ferably an academic year should elapse before the administrati on of the same tool again
Schools are advised to identify a set of APASO-II scales to su it its own needs
The majority of primary (P3 or above) students can complete 80 to 100 items from APASO-II scales in around 30 to 40 min utes
Schools have to limit the number of scales to be administered at any one time in order not to overload their students
Administration Guidelines
Schools may administer the APASO-II scales on a class or group basis during class time. Schools should not arrange for a large number of students to complete APASO-II que stionnaires in one place simultaneously
Students should be separated sufficiently in their seating t o allow privacy and confidentiality of their responses to q uestions. This is vital as these are self-reporting surveys
There is no time limit on taking any of the tools
Administration Guidelines
The teacher administering an APASO-II survey must state briefly the purpose and method of responding. They shoul d emphasize that the questionnaires will be anonymous an d the information collected be kept strictly confidential an d there will be four levels of responses
Evaluative wording such as right/wrong, high/low, strong/
weak should be avoided
The instruction should not take too long
Administration Guidelines
When students are responding to questionnaires, teachers should not assume a monitoring role, or check the student s' responses, as this may discourage students to give their t rue responses
For students with difficulty in reading, teachers may read out aloud the instructions for completing the questionnaire s and also each item as the students progress through them . Problems related to reading ability may be minimized as students can hear the items as they read them
Scales Selection
Schools may select APASO-II scales and subscales accord ing to their own concern. The scales available for use from APASO-II are organized on the basis of Bronfenbrenner’s (1995) model
Scales Selection
Scale / Subscale
School Concern Learning
Strat egies
Quali ty of Scho ol Life
Learn ing Motiv ation
Socia l Relati onshi p
Moral Cond uct
Emoti
on Self- Effica cy
Value s
Self-Concept
General School
Mathematics
Parent Relationships
Peer Relation
Physical Appearance
Reading
Interpersonal Relationships
Care for Others
Inappropriate Assertiveness *
Scales Selection
Scale / Subscale
School Concern Learni
ng Strate gies
Qualit y of Schoo l Life
Learni ng Motiv ation
Social Relati onshi p
Moral Cond uct
Emoti on
Self- Effica cy
Value s
Attitudes to School
Achievement
Experience
General Satisfaction
Negative Affect *
Opportunity
Social Integration
Teacher-Student
Relationship
Motivation
Affiliation
Competition
Effort
Praise
Social Concern
Social Power
Task
Scales Selection
Scale / Subscale
School Concern Lea
rnin g Stra tegi es
Quali ty of Scho ol Life
Learn ing Motiv ation
Socia l Relati onshi p
Moral Cond uct
Emoti
on Self- Effica cy
Value s
Causal Attribution
Failure Ability Attribution *
Failure Effort Attribution *
Failure Strategy Attribution *
Success Ability Attribution
Success Effort Attribution
Success Strategy Attribution
Learning Competency
Creative Thinking
Scales Selection
Scale / Subscale
School Concern Learni
ng Strate gies
Qualit y of Schoo l Life
Learni ng Motiv ation
Social Relati onshi p
Moral Cond uct
Emoti
on Self-
Effica cy
Value s
Independent Learning Capacity
Academic Affect
Academic Initiation
Academic Monitoring
Academic Self-Concept
Change to Improve
Costs of Help Seeking *
Education Aims
Goal Setting
Inquisitiveness
Strategic Help Seeking
Study Environment Control
Study Plan
Scales Selection
Scale / Subscale
School Concern Learn
ing Strate gies
Quali ty of Scho ol Life
Learn ing Motiv ation
Socia l Relati onshi p
Moral Cond uct
Emoti
on Self- Effica cy
Value s
Values
Code of Conduct
Commitment
Attitudes toward the Nation
Perseverance
Sense of Responsibility
The Design of APASO-II Survey
Schools can administer APASO-II scales/subscales at diff erent times of the school year to achieve different purpose s, e.g. to assess the effectiveness of an intervention progra m or to monitor the development of their students across y ears
Schools can also rotate the administration of scales across cohorts of students based on their developmental character istics
Pre-test/post-test Design
Some schools may be interested to know whether their inter vention program is effective. To achieve this purpose, schoo ls can adopt the pre-test/post-test design
For example, a school is planning to launch a courtesy camp aign to improve students’ moral conduct. Before the campai gn, students need to complete a set of scales/subscales (whic h correspond to school’s concern related to moral conduct);
and after the campaign, students need to complete the same set of scales/subscales again. Post-test scores can then be co
Longitudinal Design
Some schools may be interested to track students’ developm ent across years. To achieve this purpose, schools can adopt the longitudinal design. Every year students complete the sa me set of scales/subscales that correspond to schools’ key ar eas of developmental concern
For example, if a school’s concern is on students’ social rela tionships, then students need to complete annually a set of re levant scales like social integration, teacher-student relations hip, interpersonal relationships, affiliation, social concern, p arent relationships, peer relation and/or social harmony
Developmentally Sensitive Strategy
Schools may rotate the administration of scales across coh orts of students based on their developmental characteristi cs. This means schools may administer different scales for different grade levels of students
Group Discussion
Let us suppose that your school is planning to implement an annual self-a ssessment programme regarding “Student Attitude and Behavior”. It will primarily focus on student attitude toward school, student interpersonal re lationships and their values. Please respond to the following questions:
1. Among the scales and subscales, which would you select for your progr amme? Why so?
2. How would you make appropriate arrangement for your students to res pond to the survey? For example, in terms of venue, time allocation and re quired manpower.
3. Based on the data analysis, what would you do as follow-up action if y our student performance on a certain measurement is not satisfactory?
Reports in APASO-II
Mean plot
Box Plot
Item Bar Chart
Cross-year Comparison Plot (only available for “School A ttitude” scale)
Mean Plot
The following graph gives means and confidence intervals on a particular scale
The dots indicate the Hong Kong norm whereas the small cross with upper and lower bounds gives the means and co nfidence intervals for a particular school
Raw Score and Rasch Score
The advantage of using raw scores is that interpretation can be anchored upon the original response scale (Strongly Disagree coded as 1, Disagree coded as 2, Agree coded as 3, Strongly Agree coded as 4). If the scale mean is above 2.
5, there is reason to believe that the students on average have positive attitude s. However, how they really measure will have to depend on whether the mea n for this group of students is above or below the norm (in raw scores). In oth er words, analysis using raw scores gives substantive meaning anchored on th e items and the response scale.
The disadvantage of using raw scores is that the scale is not linear. The distan ce between a pair of adjacent categories, say 1 (Strongly Disagree) and 2 (Dis agree), is not the same as that between another pair of adjacent categories, say 2 (Disagree) and 3 (Agree), making interpretation of the meaning of distance on the scale difficult. Non-linearity is particularly prominent at the two ends o
3.9
3.8 2.6
2.5
3.9
3.8
2.6
2.5
Raw Score
Raw Score Rasch Score Rasch
Score
Box Plot
Compared to Mean Plot, Box Plot better reflects the distri bution of data sample. This is because Mean Plot only refl ects the centre trend of whole sample whereas Box Plot C hart typically describes 50% of the sample spread plus and minus over the point of median. This helps to further the u nderstanding of the sample.
Item Bar Chart
A school needs to determine if scale- or item-level results are require d. Scale-level results (e.g. Attitudes to School) give a broad overview of a selected domain (e.g. students’ attitudes toward their school as r eflected in their quality of school life) and its subscales (e.g. Teache r-Student Relations). Scale-level results are useful for school policy decisions concerning the affective and social outcomes of students.
On day-to-day operations, teachers might want to refer to Item-level (e.g. “My teacher takes an interest in helping me with my work”) res ults (Note: only available for raw scores), which give detailed inform ation for each question item. Furthermore, percentages have little me aning if the number is less than 100 students. Item bar charts provide an easy visual aid for schools to detect differences between this scho
Cross-year Comparison Plot
At the current stage, Cross-year Comparison Plot is only a pplicable to the scale for “Attitude to School”. ESDA will soon add this format for other scales as well. At maximum , it allows comparisons across three years. This format allo ws the display of performances by the whole student body per selected scales across the past three years, thus reveali ng the progress or lack thereof per school
