Workshop on “Application of the Assessment Program for Affective
and Social Outcomes (2nd Versio n)”
(Secondary Schools)
Unit One
In this unit, participants will learn the f ollowing:
The background, structure and guiding princ iples of APASO
The core functions of the E-platform for Sc hool Development and Accountability (ESDA)
The guidelines for selecting and using scal
es and subscales
Unit Two
In this unit, participants will learn the f ollowing:
How to publish APASO-II user-defined and pr e-defined surveys using “online” and “of fline” modes in ESDA
How to complete APASO-II surveys in “onlin e mode”
How to check the status of implementation o
Unit Three
In this unit, participants will learn the f ollowing:
How to review different data reports
How to interpret the data from different re ports
Summarize the learning issues and share the
experience
Background
To provide support to schools for self-evaluation, EDB la unched the Assessment Program for Affective and Social Outcomes (APASO) in 2003
Work was conducted from 2008 to 2010 to review, validat e and revise APASO. New scales were added and the nor ms of all scales were updated
The final product becomes the 2nd version of Assessment Program for Affective and Social Outcomes (APASO-II)
Purposes
It supports schools to obtain objective student performance d ata in comparison to related indicators for self-evaluation. Thi s instrument is intended to measure a group of students or the whole school student body rather than individuals
Schools can gain an in-depth understanding of students' perfo rmance and development in the affective and social domains.
This facilitates the evaluation of the impact of support service s and activities on the students and then the formulation of an action plan for improvement
Parents can also get information on students' performance in t hese domains through such channels as parent meetings and s chool reports
Scales and Subscales
Scale Name Subscale Name
Self
Emotional Stability
General
Honesty / Trustworthiness
Self-Concept Mathematics
Parent Relationships
Physical Appearance
Verbal
Health &Well Being Test Anxiety*
Recreation
Stress Management Self Encouragement
Situation control Self-others
Care for Others
Interpersonal Competence Interpersonal Relationships* Respect for Others
Scales and Subscales
Scale Name Subscale Name
Self-school
Achievement
Experience
General Satisfaction Attitudes to School (Quality of School Life) Negative Affect*
Opportunity
Social Integration
Teacher-Student Relationship
Affiliation
Competition
Effort
Motivation Praise
Social Concern
Social Power
Task
Token
Scales and Subscales
Scale Name Subscale Name
Self-school
Creative Thinking
Learning Competency Critical Thinking
Problem Solving
Time Management
Academic Affect
Academic Initiation
Academic Monitoring
Academic Self Concept
Change to Improve
Costs of Help Seeking * Independent Learning Capacity Goal Setting
Inquisitiveness
Reading Strategy
Strategic Help Seeking
Scales and Subscales
Scale Name Subscale Name
Self-society
Leadership Leadership
Commitment
Ethical Conduct
Ethical Conducts No Indulgence Life Style
Perseverance
Self Control
Duty to the Nation
National Identity & Global Citizenship Emotional Attachment to the Nation
Global Citizenship
Attitudes toward the Nation Self-future
Expectation on Career
Goals of Life Goal Setting
Goals of Life
Purposes of Taking APASO II:
What is the purpose of conducting an APASO-II survey?
What sort of data are collected from APASO-II?
Is it to understand certain affective and social behavior of students?
Or do you want to understand the impact of a certain inter vention or a program?
Administration Guidelines
It is not advisable to use the same tool too often. The reco mmended interval between two surveys is at least six mon ths. Preferably an academic year should elapse before the administration of the same tool again
Schools are advised to identify a set of APASO-II scales t o suit its own needs
The majority of secondary students can complete 80 to 10 0 items from APASO-II scales in around 30 to 40 minutes
Schools have to limit the number of scales to be administe red at any one time in order not to overload their students
Administration Guidelines
Schools may administer the APASO-II scales on a class or group basis during class time. Schools should not arrange for a large number of students to complete APASO-II que stionnaires in one place simultaneously
Students should be separated sufficiently in their seating t o allow privacy and confidentiality of their responses to q uestions. This is vital as these are self-reporting surveys
There is no time limit on taking any of the tools
Administration Guidelines
The teacher administering an APASO-II survey must state briefly the purpose and method of responding. They shoul d emphasize that the questionnaires will be anonymous an d the information collected be kept strictly confidential an d there will be four levels of responses
Evaluative wording such as right/wrong, high/low, strong/
weak should be avoided
The instruction should not take too long
Administration Guidelines
When students are responding to questionnaires, teachers should not assume a monitoring role, or check the student s' responses, as this may discourage students to give their t rue responses
For students with difficulty in reading, teachers may read out aloud the instructions for completing the questionnaire s and also each item as the students progress through them . Problems related to reading ability may be minimized as students can hear the items as they read them
Scales Selection
Schools may select APASO-II scales and subscales according to their own concern. The scales available for use from APASO-II are organized on the basis of Bronfenbrenner’s (1995) model.
Scales Selection
Scale / Subscale
School Concern Learnin
g Strategi
es
Quality of School
Life
Learnin g Motivat
ion
Social Relatio nship
Moral Conduc
t
Emotio
n Self- Efficac
y
Values
Self-Concept
Emotional Stability
General
Honestly / Trustworthiness
Mathematics
Parent Relationships
Physical Appearance
Verbal
Health & Well Being *
Stress Management
Interpersonal Relationships
Care for Others
Interpersonal Competence
Respect for Others
Scales Selection
Scale / Subscale
School Concern Learnin
g Strategi
es
Quality of School
Life
Learnin g Motivat
ion
Social Relatio nship
Moral Conduc
t
Emotio
n Self- Efficac
y
Values
Attitudes to School
Achievement
Experience
General Satisfaction
Negative Affect *
Opportunity
Social Integration
Teacher-Student
Relationship
Motivation
Affiliation
Competition
Effort
Praise
Social Concern
Social Power
Task
Token
Scales Selection
Scale / Subscale
School Concern
Learning Strategie
s
Quality School of
Life
Learning Motivati
on
Social Relation
ship
Moral
Conduct Emotion Self-
Efficacy Values
Learning Competency
Creative Thinking
Critical Thinking
Problem Solving
Time Management
Independent Learning Capacity
Academic Affect
Academic Initiation
Academic Monitoring
Academic Self-Concept
Change to Improve
Costs of Help Seeking *
Goal Setting
Inquisitiveness
Reading Strategy
Scales Selection
Scale / Subscale
School Concern Learnin
g Strategi es
Quality of School Life
Learnin g
Motivat ion
Social Relation ship
Moral
Conduct Emotio
n Self-
Efficacy Values
Leadership
Ethical Conducts
Commitment
Ethical Conduct
No Indulgence Life Style
Perseverance
Self Control
National Identity and Global
Citizenship
Goals of Life
Expectation on Career
Goal Setting
Goals of Life
The Design of APASO-II Survey
Schools can administer APASO-II scales/subscales at diff erent times of the school year to achieve different purpose s, e.g. to assess the effectiveness of an intervention progra m or to monitor the development of their students across y ears
Schools can also rotate the administration of scales across cohorts of students based on their developmental character istics
Pre-test/post-test Design
Some schools may be interested to know whether their inter vention program is effective. To achieve this purpose, schoo ls can adopt the pre-test/post-test design
For example, a school is planning to launch a courtesy camp aign to improve students’ moral conduct. Before the campai gn, students need to complete a set of scales/subscales (whic h correspond to school’s concern related to moral conduct);
and after the campaign, students need to complete the same set of scales/subscales again. Post-test scores can then be co mpared with pre-test scores to examine whether there is imp rovement in students’ moral conduct
Longitudinal Design
Some schools may be interested to track students’ developm ent across years. To achieve this purpose, schools can adopt the longitudinal design. Every year students complete the sa me set of scales/subscales that correspond to schools’ key ar eas of developmental concern
For example, if a school’s concern is on students’ social rela tionships, then students need to complete annually a set of re levant scales like social integration, teacher-student relations hip, interpersonal relationships, affiliation, social concern, p
Developmentally Sensitive Strategy
Schools may rotate the administration of scales across coh orts of students based on their developmental characteristi cs. This means schools may administer different scales for different grade levels of students
Group Discussion
Let us suppose that your school is planning to implement an annual self-a ssessment programme regarding “Student Attitude and Behavior”. It will primarily focus on student attitude toward school, student interpersonal re lationships and their values. Please respond to the following questions:
1. Among the scales and subscales, which would you select for your progr amme? Why so?
2. How would you make appropriate arrangement for your students to res pond to the survey? For example, in terms of venue, time allocation and re quired manpower.
3. Based on the data analysis, what would you do as follow-up action if y
Reports in APASO-II
Mean plot
Box Plot
Item Bar Chart
Cross-year Comparison Plot (only available for “School A ttitude” scale)
Mean Plot
The following graph gives means and confidence intervals on a particular scale
The dots indicate the Hong Kong norm whereas the small cross with upper and lower bounds gives the means and co nfidence intervals for a particular school
Raw Score and Rasch Score
The advantage of using raw scores is that interpretation can be anchored upon the original response scale (Strongly Disagree coded as 1, Disagree coded as 2, Agree coded as 3, Strongly Agree coded as 4). If the scale mean is above 2.
5, there is reason to believe that the students on average have positive attitude s. However, how they really measure will have to depend on whether the mea n for this group of students is above or below the norm (in raw scores). In oth er words, analysis using raw scores gives substantive meaning anchored on th e items and the response scale.
The disadvantage of using raw scores is that the scale is not linear. The distan ce between a pair of adjacent categories, say 1 (Strongly Disagree) and 2 (Dis agree), is not the same as that between another pair of adjacent categories, say 2 (Disagree) and 3 (Agree), making interpretation of the meaning of distance on the scale difficult. Non-linearity is particularly prominent at the two ends o f the scale.
3.9
3.8 2.6
2.5
3.9
3.8
2.6
Raw Score
Raw Score Rasch Score Rasch
Score
Box Plot
Compared to Mean Plot, Box Plot better reflects the distri bution of data sample. This is because Mean Plot only refl ects the centre trend of whole sample whereas Box Plot C hart typically describes 50% of the sample spread plus and minus over the point of median. This helps to further the u nderstanding of the sample.
Item Bar Chart
A school needs to determine if scale- or item-level results are require d. Scale-level results (e.g. Attitudes to School) give a broad overview of a selected domain (e.g. students’ attitudes toward their school as r eflected in their quality of school life) and its subscales (e.g. Teache r-Student Relations). Scale-level results are useful for school policy decisions concerning the affective and social outcomes of students.
On day-to-day operations, teachers might want to refer to Item-level (e.g. “My teacher takes an interest in helping me with my work”) res ults (Note: only available for raw scores), which give detailed inform ation for each question item. Furthermore, percentages have little me aning if the number is less than 100 students. Item bar charts provide an easy visual aid for schools to detect differences between this scho ol and schools in item-level.
Cross-year Comparison Plot
At the current stage, Cross-year Comparison Plot is only a pplicable to the scale for “Attitude to School”. ESDA will soon add this format for other scales as well. At maximum , it allows comparisons across three years. This format allo ws the display of performances by the whole student body per selected scales across the past three years, thus reveali ng the progress or lack thereof per school
