Training Manual

(1)

Workshop on “Application of the Assessment Program for Affective and Social Outcomes (2nd Version)”

Training Manual

(For Secondary Schools)

Education Bureau and The Hong Kong Institute of Education

November 2010

(2)

Unit One

Learning Objectives

In this unit, participants will learn the following:

z The background, structure and guiding principles of APASO

z The core functions of the E-platform for School Development and Accountability (ESDA)

z The guidelines for selecting and using scales and subscales

(4)

A. Introduction of Assessment Program for Affective and Social Outcomes (2nd Version) (APASO-II)

A.1 Development

To provide support to schools for self-evaluation, EDB launched the Assessment Program for Affective and Social Outcomes (APASO) in 2003. This has facilitated schools to examine their students’ development and needs in the social and affective domains and the effectiveness of related measures implemented in schools. They can then use the feedback to further improve on their planning with programmes to benefit the whole person development of their students. Since its implementation, APASO has been widely adopted by schools across Hong Kong.

In view of the latest education development and schools’ needs, the APASO has been updated regularly. In 2008, EDB commissioned HKIEd to conduct a comprehensive review for the purpose of revising the APASO and updating the norm data. EDB has now completed the revision of the APASO and launched the APASO-II for schools during 2010/11 school year.

A.2 Framework

APASO-II is an enhancement of the previous version. The scales and the norm data of the APASO have been updated according to the education aims and the latest education development in Hong Kong. To cater for the needs of different schools, it encompasses a wider range of items in the social and affective domains.

(5)

A.3 Purposes

The purposes of APASO-II:

z It supports schools to obtain objective student performance data in comparison to related indicators for self-evaluation. This instrument is intended to measure a group of students or all students in the school rather than individuals

z Schools can gain an in-depth understanding of students' performance and development in the affective and social domains. This facilitates the evaluation of the impact of support services and activities on the students and then the formulation of an action plan for improvement

z Parents can also get information on students' performance in these domains through channels such as parent meetings and school reports

A.4 Guidelines

When using APASO-II, users should consider the following questions:

9 What is the purpose of conducting an APASO-II survey?

9 What sort of data are collected from APASO-II?

9 Is it to understand certain affective and social behavior of students?

9 Or do you want to understand the impact of a certain intervention or a program?

z Schools are advised to identify a set of APASO-II scales to suit the school needs.

The majority of Secondary students can complete 80 to 100 items from APASO-II scales in around 30 to 40 minutes. Schools have to limit the number of scales to be administered at any one time in order not to overload their students.

z Schools may administer the APASO-II scales on a class or group basis during class time. Schools should not arrange for a large number of students to complete APASO-II questionnaires in one place simultaneously; it would be more preferable if it is on a class or group basis. Students should be separated sufficiently in their seating to allow privacy and confidentiality in responses to questions.

z This is vital as these are self-reports and there is no time limit on any of the tools.

z The teacher responsible for the task of APASO-II data collection must state briefly the purpose and method of responding, emphasizing that the questionnaires will be anonymous and the information strictly confidential, and

(6)

that there will be four levels of responses. Nevertheless, evaluative wording such as right or wrong, high or low, strong or weak should be avoided. The briefing should not be too long (not longer than 10 minutes).

z When the students respond to the questionnaires, teachers should not assume a monitoring role, or check the students' responses, as this may cause students not to give their true responses to items.

z For students with difficulty in reading, teachers may read out loud the instructions for completing the questionnaires and also read out loud each item as the students progress through them. This way, problems related to reading ability may be minimized as students also hear the items as they read them.

z If the school uses the "online" version of APASO-II, the students must be arranged to complete the questionnaires using the computer. Teachers escort their students to the computer room and have them seated at random in order to guarantee anonymity.

(7)

z The affective and social performance of children and juveniles are affected by a number of factors. External factors include the schools’ support services, learning activities and the family; whereas internal factors include age, gender, and previous experience in school and at home. Consequently, schools should consider collecting the information from different perspectives, e.g. students’

family background, types of counseling cases and discipline data, etc. The schools should not draw up a follow-up plan based merely on the data collected from the questionnaires, but rather, they should analyze the information along with other internal data available, so as to form a basis for improving the various support services.

z The data will only reflect students’ views expressed in the various domains.

Students may have different views in different developmental stages. The data in each questionnaire will help the schools to have a better understanding of the students.

z Unlike examination results, such data are not meant to be an evaluation of their standard, nor does it reflect the students' achievement. As the data have nothing to do with standards, and students' affective and social performance is affected by various factors, we do not encourage the use of such data as the basis for comparing the effectiveness of different schools.

z For the latest news and details of the Assessment Program, please visit the following website: http://apaso.edb.gov.hk.

(8)

B. Introduction of E-platform for School Development and Accountability (ESDA)

B.1 Purposes

It supports the school self-evaluation (SSE) and promotes their efficiency in conducting SSE. It also alleviates teachers’ workload.

B.2 Functions

ESDA is used by schools to collect and manage self-evaluation data, including the administration of on-line Stakeholder Survey (SHS), assessing students’ affective and social outcomes, performing analysis of data collected, and publishing the related data reports. For the latest news and details of ESDA, please visit the following website:

http://esda.edb.gov.hk.

The version of ESDA that includes APASO-II functions is 3.1.0.

School ESDA

(9)

C. Scales Selection Guidelines

C.1 Concept

The scales available for use from APASO-II are organized on the basis of Bronfenbrenner’s (1995) model. The figure below illustrates the simplified model with different scales corresponding to different layers of social influence.

z It can be seen from the figure below that the individual (i.e., self) is the centre of the diagram and there are three scales that measure students’ affective and social performance related to the self. The three scales include Self-Concept, Health and Well Being and Stress Management.

z The self is surrounded by immediate environment such as peers, family and school. The five scales that measure students’ affective and social performance related to these immediate social influences include Interpersonal Relationships, Attitudes to School, Motivation, Learning Competency and Independent Learning Capacity.

z Immediate environment is further surrounded by community that is less immediate social influence. The scale that measures students’ affective and social performance related to community includes Leadership, Ethical Conducts and National Identity and Global Citizenship.

z The community is surrounded by the future. The scale that measures students’

affective and social performance related to future is Goals of Life.

Correspondence between Bronfenbrenner’s (1995) model and scales in APASO-II

(10)

C.2 Examples of Scales Selection

Based on the assessment objectives, schools can select related scales/ subscales or their own combinations for their questionnaires. To illustrate how schools can select APASO-II scales/subscales for different purposes, the table below presents some combinations of scales/subscales that schools can use to address their concerns. For example, if schools are interested to know more about their students’ learning strategies, they can administer two scales, i.e., Independent Learning Capacity and Learning Competency. Please note that the school concerns listed here are just examples and are not exhaustive. Schools are expected to administer different combinations of scales according to their needs.

Notes:

i. The school concerns below are for illustrative purposes only.

ii. More than one set of scales are sometimes included with a (9) for one area of school concern in the table. However in practice, it is not necessary to use all of the indicated scales to measure an area of school concern.

iii. At any APASO-II data collection session, students should not be administered more than 100 items.

iv. “ * ” indicates the subscale is negatively worded

Scale / Subscale

School Concern

Learning Strategies Quality of School Life Learning Motivation Social Relationship Moral Conduct Emotion Self-Efficacy Values

Self-Concept 3

z Emotional Stability 3 3

z General 3

z Honestly / Trustworthiness 3 3

z Mathematics 3

z Parent Relationships 3 3

z Physical Appearance 3

z Verbal 3

Health & Well Being * 3 3

Stress Management 3

(11)

School Concern

Interpersonal Relationships

z Care for Others 3 3

z Interpersonal Competence 3 3 3

z Respect for Others 3 3 3

z Share 3 3

z Social Skills 3 3

z Support 3 3

Attitudes to School

z Achievement 3 3

z Experience 3

z General Satisfaction 3 3

z Negative Affect * 3 3

z Opportunity 3 3

z Social Integration 3 3

z Teacher-Student Relationship 3 3

Motivation

z Affiliation 3 3 3

z Competition 3 3

z Effort 3 3

z Praise 3 3

z Social Concern 3 3 3

z Social Power 3 3

z Task 3 3

z Token 3 3

Learning Competency

z Creative Thinking 3 3

z Critical Thinking 3 3

z Problem Solving 3 3

z Time Management 3 3

(12)

School Concern

Independent Learning Capacity

z Academic Affect 3 3

z Academic Initiation 3 3

z Academic Monitoring 3 3

z Academic Self-Concept 3 3 3

z Change to Improve 3 3

z Costs of Help Seeking * 3 3 3 3

z Goal Setting 3 3

z Inquisitiveness 3 3

z Reading Strategy 3 3

z Strategic Help Seeking 3 3

z Study Environment Control 3 3

z Study Plan 3 3

z Value of School Work 3 3 3

Leadership 3 3

Ethical Conducts

z Commitment 3 3

z Ethical Conduct 3

z No Indulgence Life Style 3

z Perseverance 3 3 3

z Self Control 3

National Identity and Global

Citizenship 3 3

Goals of Life

z Expectation on Career 3 3

z Goal Setting 3 3

z Goals of Life 3 3

(13)

D. The Design of APASO-II Survey

Schools can administer APASO-II scales/subscales at different times of the school year to achieve different purposes, e.g. to assess the effectiveness of intervention programmes or to monitor the development of their students across years. Schools can also rotate the administration of scales across cohorts based on students’

developmental characteristics. Below are several data collection designs to illustrate how schools can determine the timing of administration of different scales.

D.1 Pre-test/post-test Design

Some schools may be interested to know whether their intervention programme is effective. To achieve this purpose, schools can adopt the pre-test/post-test design. For example, a school is planning to launch a courtesy campaign to improve students’

moral conduct. Before the launch of the campaign, students need to complete a set of scales/subscales (which correspond to schools’ concern related to moral conduct); and after the campaign, students need to complete the same set of scales/subscales again.

Post-test scores can then be compared with pre-test scores to examine whether there is improvement in students’ moral conduct. Similar administration can be applied to other school intervention programmes like programmes on social relationship, self-concept, motivation and learning strategies. One reminder is that the pre-test/post-test comparison results should be interpreted with caution as the cause-and-effect relationship is still hard to be confirmed. For example, students’

improvement in moral conduct can be due to various reasons, including students’

maturation as time passes, social events and parents’ education at home. Thus, whether the intervention programme is really effective depends on schools’

professional judgment. Data generated from pre-test/post-test comparison only provides one source of information for schools to consider.

D.2 Longitudinal Design

Some schools may be interested to track students’ development across years. To achieve this purpose, schools can adopt the longitudinal design. Every year students complete the same set of scales/subscales that correspond to schools’ key areas of developmental concern. For example, if the school concern is on students’ social relationships, then students need to complete annually a set of relevant scales/subscales like social integration, teacher-student relationship, interpersonal relations, leadership, affiliation, social concern and parent relationships. Similar administration can be applied to other school concerns like quality of school life and values. Depending on the changes of scores across years, schools can take possible

(14)

follow-up actions. For example, if teacher-student relationship drops across years more than that of the norms, then schools may consider launching some programmes to build up relationship or alerting teachers to spend more time to communicate with students. Special attention may be paid to a particular class or a particular grade level of students who get exceptionally low scores on some scales. If necessary, schools may seek professional advice from school psychologists, school social works and other professionals in tertiary institutions.

D.3 Developmentally Sensitive Strategy

Schools may rotate the administration of scales across cohorts based on students’

developmental characteristics. This means schools may administer different scales for different grade levels of students. It is advised that quality of school life and social relationship are two areas that can be assessed in early years as they play significant role in students’ development. Previous literature indicates that quality of school life or attitudes to school is one of the indictors of school effectiveness (Ainley, 1994;

Ainley, 2006). It is an important factor that predicts students’ academic achievement, their learning motivation and their general well-being (Cheng & Le, 2005; Mok &

Flynn, 2002; Philips, 1978), Interpersonal relations are also crucial for healthy psychological development, including providing support to deal with everyday pressures and facilitating school adjustment (Berk, 1997).

It is advised that independent learning capacity, goals of life and national identity and global citizenship can be assessed in later years when students become more mature.

Previous literature indicates that independent learning capacity involves a complex interaction of motivational, cognitive and metacognitive self-regulation (Boekaerts, 1997; Zimmerman & Shunk, 2001). Goals of life includes expectation on career and other long term goals which are more relevant for students in higher grade levels.

National identity and global citizenship represents how self is related to community that involves high level of cognitive functioning.

Please note that the rotation strategy suggested is only one of the possible arrangements and does not represent the “best” approach to measure students’

affective and social performance. Schools are expected to develop their own strategy in administering the scales.

(15)

E. Group Discussion

Let us suppose that your school is planning to implement a year-long self-assessment programme regarding “Student Attitude and Behavior”. It will primarily focus on student attitude toward school, student interpersonal relationships and their values.

Please respond to the following questions:

1. Among the scales and subscales, which would you select for your programme?

Why so?

2. How would you make appropriate arrangement for your students to respond to the survey? For example, in terms of venue, time allocation and required manpower.

3. Based on the data analysis, what would you do as follow-up action if your student performance on a certain aspect is not satisfactory?

(16)

Unit Two

Learning Objectives

z How to publish APASO-II user-defined and pre-defined surveys using “online”

and “offline” modes in ESDA

z How to complete APASO-II surveys in “online” mode z How to check the status of implementation of a survey z How to import survey data offline

(17)

F. User-defined Survey Publication and Data Import (Online)

1. If you want to customize your own survey, under “Survey Administration”, select

“Add Survey”.

2. Under the “Add Survey”, enter the survey title in English and Chinese and its intended purpose. Then select all the relevant Subscales under “Scale” and hit

“Submit”.

3. Press “Publish” to publish the newly created survey.

(18)

4. Under “Target Users”, choose “From Group or Individual User (Manual Selection)” and then press “Select User”.

5. Select the relevant students who will be asked to complete the survey and then

(19)

6. Choose the “Online” mode and set your Start/End Date and then press “Save”. It’s optional whether to fill in the objectives in the boxes.

7. Logout the system first, then log in with the student’s user names and passwords to complete the survey.

8. Press “View”.

(20)

9. Read “Instruction for APASO” on the screen carefully. Press “Start” when user is ready to complete the survey.

10. Answer all the questions and press “Confirm” to submit the answers.

11. After the submission of the answers, the system will indicate “Record saved successfully”. Finally remember to log out.

(21)

12. If school managers want to track the status of the survey administration, they can use the system administrator’s username and password to log in.

13. Press and pull down menu for “Survey Administration”. Choose “Survey Administration”.

14. Press “Details” button on the right for the relevant survey to view its status.

(22)

15. Press “Download User List” menu and check if the students have completed the survey.

(23)

G. Pre-defined Survey Publication and Data Import (Offline)

1. Click on ESDA to switch to ESDA window.

2. Press and pull down the menu for “Survey Administration”. Choose “Survey Administration”.

3. Press “Publish Survey”.

4. From “Assessment Program for Affective and Social Outcomes” menu, select one of the surveys. For demonstration in this manual, we choose “Motivation” and then hit “Publish”.

5. For “Target User”, choose “From Class or Form (All Users or Random Selection)”

and then hit “Select User”.

(24)

6. Select “All users within the selected group(s)” and then hit “OK”

7. Next choose “Offline” and then set “Start/End Date”. Then press “Save”. It’s optional whether to fill in the objectives in the boxes.

(25)

8. Under the pull down menu of ‘Survey Administration’, press “Offline Survey Import”.

9. Press “Details” next to the survey Motivation, next press “Download CSV Template”.

10. Press “Open” to open the template, fill in the responses to the downloaded CSV file and then save.

11. Press “Import Response”.

(26)

12. Press “Browse” button. Choose file “Sec_Motivation.csv” in “Offline_data”

folder and then press “Submit”.

13. Please note: If the SYSTEM_CODE submitted for survey doesn’t match that from imported file, it will result in “Import Data Failure”.

14. After importing data successfully, press and pull down the menu for “Survey Administration”. Choose “Survey Administration” and you can see “Response Rate” for the survey.

(27)

H. Workshop Exercise

Exercise 1

Using online mode, you should first publish the pre-set survey questions under

“Learning Competency” from APASO II and send them to the students of Class 5B.

Then you can use the identity of Student 5B017 to log in to complete the survey.

Finally you should log in as System Administrator to check the status of survey being conducted.

Exercise 2

Using offline mode, you should publish the pre-set survey questions under “Attitude to School” and send the questions to all students in the school. Then you can import all student responses to the survey questions.

(28)

Unit Three

Learning Objectives

z How to review different data reports

z How to interpret the data from different reports

z Summarize the learning issues and share the experience

(29)

I. Review of Survey Report

1. Press and pull down the menu for “Report Viewer”. Choose “APASO II Report”.

2. Under “Survey Title”, select the survey you need to review.

3. Under “Report Type”, the user can select among Mean Plot, Box Plot Chart, Item Bar Chart or Cross-year Comparison Plot (only available for “School Attitude”

scale). Under “Data Set”, the user can select “Raw Score” or “Rasch Score”. The user can also choose different respondent groups or group combinations to yield specific survey results. Schools can choose to save the created report.

(30)

J. Reports in APASO-II and their interpretation

There are four types of reports in APASO-II:

y Mean Plot y Box Plot y Item Bar Chart

y Cross-year Comparison Plot (only available for “Attitudes to School” scale)

J.1 Mean Plot

The following graph gives means and confidence intervals on a particular scale and the dots indicate the Hong Kong norm whereas the small cross with upper and lower bounds gives the means and confidence intervals for a particular school.

(31)

Scale mean, either raw scores or Rasch scaled scores, offer valuable information to the schools with regard to students’ affective and social outcomes.

The advantage of using raw scores is that interpretation can be anchored upon the original response scale (Strongly Disagree coded as 1, Disagree coded as 2, Agree coded as 3, Strongly Agree coded as 4). If the scale mean (e.g. Teacher-Student Relations) of a certain group of students (e.g. P.4) is above 2.5, there is reason to believe that the students on average have positive attitudes. However, how they really measure will have to depend on whether the mean for this group of students is above or below the norm (in raw scores). In other words, analysis using raw scores gives substantive meaning anchored on the items and the response scale.

The disadvantage of using raw scores is that the scale is not linear. The distance between a pair of adjacent categories, say 1 (Strongly Disagree) and 2 (Disagree), is not the same as that between another pair of adjacent categories, say 2 (Disagree) and 3 (Agree), making interpretation of the meaning of distance on the scale difficult.

Non-linearity is particularly prominent at the two ends of the scale.

In comparison, Rasch scales are linear and the distance between 2 units (logits) on the scale is constant. Historical data are meaningful given this constancy. Nevertheless, the origin of each scale is set at the average of the set of items in the scale, thus interpretation of the scale needs to refer to the items making up the scale.

3.9

3.8 2.6

2.5

3.9

3.8

2.6

2.5

Raw Score Rasch Score

(32)

J.2 Box Plot

The graph below shows the observation method of box plots and its representation in the report in ESDA.

A box-plot presents five statistical data of each scale in graphical format. They are:

minimum, first quartile (i.e. 25th percentile), median, third quartile (75th percentile), and the maximum value of the scale. The middle 50% of the data are enclosed within a box, and two ‘whiskers’ on two ends of the box are formed by joining the first quartile with the minimum, and by joining the third quartile with the maximum. In addition, a line is drawn within the box to indicate the location of the median. In this way, the box-plot clearly indicates the lower 25%, the middle-lower 25%, the middle-upper 25%, and the top 25% of the distribution of scale scores.

Compared to Mean Plot, Box Plot better reflects the distribution of data sample. This is because Mean Plot only reflects the centre trend of whole sample whereas Box Plot

(33)

J.3 Item Bar Chart

Item Bar Charts are used to display the proportion of students of this school as compared with students in Hong Kong who Strongly Disagree/ Disagree/ Agree/

Strongly Agree with the statement. Percentage distribution for each option of the items is indicated by a different color/pattern respectively.

A school needs to determine if scale- or item-level results are required. Scale-level results (e.g. Attitudes to School) give a broad overview of a selected domain (e.g.

students’ attitudes toward their school as reflected in their quality of school life) and its subscales (e.g. Teacher-Student Relations). Scale-level results are useful for school policy decisions concerning the affective and social outcomes of students.

On day-to-day operations, teachers might want to refer to Item-level (e.g. “My teacher takes an interest in helping me with my work”) results (Note: only available for raw scores), which give detailed information for each question item. Furthermore, percentages have little meaning if the number is less than 100 students. Item bar charts provide an easy visual aid for schools to detect differences between this school and schools in item-level.

(34)

J.4 Cross-year Comparison Plot

At the current stage, Cross-year Comparison Plot is only applicable to the scale for

“Attitude to School”. ESDA will soon add this format for other scales as well. At most, it allows comparisons across three years. This format allows the display of the overall performance of all students in the school for the selected scales across the past three years, thus informing the school of its progress.

(35)

K. Data Interpretation Exercise

K.1 Using Report Viewer

Use Report Viewer to show the data of S5 students for the scale “Attitudes to School”

K.2 Data Interpretation

Use relevant data (in Appendix II) to discuss the following problems and describe how you interpret data and what you can see.

A. How did the S5 students perform for the Scale “Attitudes to School”?

Data Observation Data Interpretation

Data Sample（1）：

Among the subscales under “Attitudes to School”, of them have a mean higher than the territory norm

(subscale is negatively worded). Subscale has a mean higher than the territory norm, the mean is , the standard

deviation is and the effect size is .

Subscale has a mean higher than the territory norm, but the difference is very small. People in the school should look into it further, trying to understand possible reasons.

In this sample, if the median of a subscale is higher than the territory norm, its and quartiles are also higher the territory norm. On the contrary, if the median of a subscale is than the territory norm, its 25th and 75th quartiles are higher than the territory norm.

The data from all the subscales under

“Attitudes to School” indicate that the students are spread and there is no case.

B. How did the S5 students perform regarding the subscales under Social Integration?

Except item , the rate of agreement of all items is than the territory norm.

The students demonstrate a difference on item .

(36)

C. How did all students in the school perform on the scale of “Attitude to School” for the past three years?

Except subscale , students in the past three years on all subscale.

Students show some regression on subscale , a concern that worths our further investigation.

(37)

Appendix I. Answer Key for the Workshop Exercise (for reference)

Exerciese 1 Step 1

- Press and pull down the menu for “Survey Administration”. Choose “Survey Administration” and press “Publish Survey”

- From “Assessment Program for Affective and Social Outcomes” menu, select the survey “Learning Competency” and then hit “Publish”

- For “Target Users”, choose “From Class or Form (All Users or Random

Selection)” and then hit “Select User”, then select “Class” and choose “5B”, after that hit right arrow and “OK”

- Next choose “Offline” and then set “Start/End Date”. It’s optional whether to fill in the objectives in the boxes. Then press “Save” and logout the system.

Step 2

- Use username and password “5B017” to login the system

- Press “View” in the row of “Learning Competency” survey. Then press “Start”to answer the items

- After the survey is completed, press “Confirm” to submit answer and logout the system

Step 3

- Use username and password “admin” to login the system

- Press and pull down the menu for “Survey Administration”, then press “Details”

for “Learning Competency” survey to view its status

- Press “Download User List” to check if the students have completed the survey.

(38)

Exerciese 2 Step 1

- Press and pull down the menu for “Survey Administration”. Choose “Survey Administration” and press “Publish Survey”

- From “Assessment Program for Affective and Social Outcomes” menu, select the survey “Attitudes To School” and then hit “Publish”.

- For “Target Users”, choose “From Class or Form (All Users or Random

Selection)” and then hit “Select User”, then select “All users within the selected group(s)” and then hit “OK”.

- Choose “Offline” and then set “Start/End Date”. It’s optional whether to fill in the objectives in the boxes. Then press “Save”.

Step 2

- Press and pull down the menu for “Survey Administration”. Choose “Survey Administration”. Then press “Offline Survey Import”.

- Press “Details” next to the survey “Attitudes To School”. Then press “Download CSV Template”.

- Press “Open” to open the file (survey_import_template_Attitudes_To_School.csv), follow the template format to enter the data and save the file.

- Press ‘Details’ on the right of ‘Attitudes To School’ and press “Import Response”

- Press “Browse” button. Choose file

“survey_import_template_Attitudes_To_School.csv” and then press “Submit”.

- After importing data successfully, press and pull down the menu for “Survey Administration”. Choose “Survey Administration” and you can see “Response Rate” for the survey.

(39)

Appendix II. Data Samples

(1) The Mean Plot of the Performance of S5 students for “Attitudes to School”

Subscale / 副量表

Lower Bound / 下限

Mean / 平均數

Upper Bound / 上限

Standard Deviation / 標準差

No. Of Respondents / 回應人數

Effect Size / 效應值

Achievement / 成就感

SACH(HK) 2.38 0.56 22279 --

SACH 2.75 2.85 2.95 0.66 165 Large / 大

Experience / 經歷

SEXP(HK) 2.40 0.56 2483 --

SEXP 2.82 2.93 3.03 0.69 166 Large / 大

General Satisfaction / 整體滿足感

SGES(HK) 2.49 0.57 22304 --

SGES 2.92 3.01 3.11 0.64 165 Large / 大

Negative Affect / 負面情感

SNEG(HK) 1.95 0.58 22284 --

SNEG 1.45 1.54 1.63 0.60 165 Moderate / 中等

Opportunity / 機會

SOPP(HK) 2.74 0.52 2484 --

SOPP 3.17 3.26 3.35 0.60 166 Very large / 極大

Social Integration / 社群關係

SSOI(HK) 2.94 0.46 2484 --

SSOI 2.68 2.79 2.91 0.75 165 Small / 小

Teacher-Student Relationship / 師生關係

STEA(HK) 2.89 0.52 22286 --

STEA 3.21 3.29 3.38 0.55 165 Moderate / 中等

1 1.5 2 2.5 3 3.5 4

SACH(HK) SACH SEXP(HK) SEXP SGES(HK) SGES SNEG(HK) SNEG SOPP(HK) SOPP SSOI(HK) SSOI STEA(HK) STEA

(40)

(2) The Box Plot for the Performance of S5 students on “Attitudes to School”

Subscale /副量表

Minimum / 最小值

25th Percentile /第 25 個百分位數

Median / 中位數

75th Percentile / 第 75 個百分位數

Maximum / 最大值

No. Of Respondents / 回應人數

SACH(HK) 1.00 2.00 2.33 2.67 4.00 22279

SACH 1.00 2.50 2.83 3.33 4.00 165

Experience / 經歷

SEXP(HK) 1.00 2.00 2.40 2.80 4.00 2483

SEXP 1.00 2.40 3.00 3.40 4.00 166

SGES(HK) 1.00 2.17 2.50 2.83 4.00 22304

SGES 1.17 2.67 3.17 3.50 4.00 165

SNEG(HK) 1.00 1.57 2.00 2.29 4.00 22284

SNEG 1.00 1.00 1.43 1.86 4.00 165

SOPP(HK) 1.00 2.43 2.86 3.00 4.00 2484

SOPP 1.00 3.00 3.29 3.71 4.00 166

SSOI(HK) 1.00 2.71 3.00 3.00 4.00 2484

SSOI 1.00 2.29 2.86 2.89 4.00 165

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50

SACH(HK) SACH SEXP(HK) SEXP SGES(HK) SGES SNEG(HK) SNEG SOPP(HK) SOPP SSOI(HK) SSOI STEA(HK) STEA

(41)

(3) Item Bar Chart of the Performance of S5 students on “Social Integration”

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Q.32(HK)

Q.32

Q.33(HK)

Q.33

Q.34(HK)

Q.34

Q.35(HK)

Q.35

Q.36(HK)

Q.36

Q.37(HK)

Q.37

Q.38(HK)

Q.38

(42)

Item / 題目

Strongly disagree /

毫不同意 (%)

Disagree / 不太同意

(%)

Agree / 相當同意 (%)

Strongly agree / 極之同意 (%)

Mean / 平均數

Standard Deviation /

標準差

No. Of Respondents

/ 回應人數

Effect Size / 效

應值

Q.32. 其他同學都接納我。

/ Other students accept me as I am.

Q.32(HK) 5.54 16.63 42.30 35.53 3.08 0.86 2544 --

Q.32 6.67 20.61 35.76 36.97 3.03 0.92 165

Negligible / 微

Q.33. 在學校，別人都信任我。 / People at school trust me.

Q.33(HK) 5.33 20.56 45.90 28.22 2.97 0.84 2534 --

Q.33 8.07 26.09 39.75 26.09 2.84 0.91 161

Negligible / 微

Q.34. 其他同學都喜歡我。

/ I am popular with other students.

Q.34(HK) 8.31 27.52 42.44 21.73 2.78 0.88 2540 --

Q.34 12.96 30.86 38.27 17.90 2.61 0.93 162

Negligible / 微 Q.35. 我能與班中其他同

學融洽相處。 / I get on well with the other students in my class.

Q.35(HK) 4.74 16.11 44.14 35.02 3.09 0.83 2533 --

Q.35 7.55 23.27 37.11 32.08 2.94 0.93 159

Negligible / 微 Q.36. 在學校，別人都尊敬

我。 / People at school look up to me.

Q.36(HK) 12.96 33.14 38.22 15.68 2.57 0.91 2538 --

Q.36 16.67 35.80 33.33 14.20 2.45 0.93 162

Negligible / 微

Q.37. 其他同學都很友善。

/ Other students are very friendly.

Q.37(HK) 6.42 18.63 39.40 35.55 3.04 0.89 2523 --

Q.37 8.64 20.37 38.27 32.72 2.95 0.94 162

Negligible / 微 Q.38. 在學校，別人都重視

我的意見。 / Other people at school care what I think.

Q.38(HK) 11.56 30.30 38.70 19.45 2.66 0.92 2535 --

Q.38 15.53 24.84 40.37 19.25 2.63 0.97 161

Negligible / 微

(43)

(4) Cross-year Comparison Plot of the Performance of all students in the school on “Attitudes to School”

1.00 1.50 2.00 2.50 3.00 3.50 4.00

2008/09 2009/10 2010/11 Year 2010 Norm 2008/09 2009/10 2010/11 Year 2010 Norm 2008/09 2009/10 2010/11 Year 2010 Norm 2008/09 2009/10 2010/11 Year 2010 Norm 2008/09 2009/10 2010/11 Year 2010 Norm 2008/09 2009/10 2010/11 Year 2010 Norm 2008/09 2009/10 2010/11 Year 2010 Norm

Experience / 經歷 General Satisfaction / 整體滿足感

Opportunity / 機會 Social Integration / 社群關係

Teacher-Student Relationship / 師

生關係

(44)

Subscale / 副量表 Lower Bound / 下限 Mean / 平均數 Upper Bound / 上限 No. Of Respondents / 回應人數

2008/09 2.33 2.44 2.56 598

2009/10 2.82 2.97 3.12 587

2010/11 2.69 2.75 2.80 584

Year 2010 Norm 2.78 9866

Experience / 經歷

2008/09 2.46 2.53 2.62 598

2009/10 2.51 2.67 2.72 587

2010/11 2.76 2.81 2.87 582

Year 2010 Norm 2.82 9855

2008/09 2.54 2.60 2.65 598

2009/10 2.81 2.89 3.06 587

2010/11 2.92 2.97 3.02 585

Year 2010 Norm 2.92 78990

2008/09 1.56 1.61 1.67 598

2009/10 1.56 1.61 1.67 587

2010/11 1.56 1.61 1.67 584

Year 2010 Norm 1.68 78964

2008/09 2.52 2.65 2.74 598

2009/10 2.80 2.92 3.11 587

2010/11 3.11 3.17 3.22 581

Year 2010 Norm 3.20 9867

2008/09 2.33 2.44 2.56 598

2009/10 2.82 2.88 2.93 587

2010/11 2.82 2.88 2.93 585

Year 2010 Norm 2.88 9879

Teacher-Student Relationship / 師生關係

2008/09 3.11 3.17 3.22 598

2009/10 3.19 3.23 3.28 587

2010/11 3.20 3.25 3.30 584

Year 2010 Norm 3.20 78997

(45)

Appendix III. Answer Key for Data Interpretation Exercise (for reference)

A. How did the S5 students perform for the Scale “Attitudes to School”?

Among the subscales under “Attitudes to School”, 6 of them have a mean higher than the territory norm

(subscale Negative Affect is negatively worded). Subscale Social Integration has a mean higher than the territory norm, the mean is 2.79 , the standard

deviation is 0.75 and the effect size is Small .

Subscale Social Integration has a mean lower than the territory norm, but the difference is very small. People in the school should look into it

further, trying to understand possible reasons

In this sample, if the median of a subscale is higher than the territory norm, its 1st and 3rd quartiles are also higher than the territory norm. On the contrary, if the median of a subscale is lower than the territory norm, its 25th and 75th quartiles are higher than the territory norm.

The data from all the subscales under

“Attitudes to School” indicate that the students are evenly spread and there is no extreme case.

B. How did the S5 students perform regarding the subscales under Social Relationship?

Except item 38 , the rate of agreement of all items is lower than the territory norm.

The students demonstrate a marked difference on item 38 .

C. How did all students in the school perform on the scale of “Attitudes to School”

for the past three years?

Except subscale Achievement , students made progress in the past three years on all subscale.

Students show some regression on subscale Achievement , a concern that worths our further investigation.

Training Manual

Workshop on “Application of the Assessment Program for Affective and Social Outcomes (2nd Version)”

Training Manual

(For Secondary Schools)

Education Bureau and The Hong Kong Institute of Education

November 2010

Table of Contents

Unit One

A. Introduction of Assessment Program for Affective and Social Outcomes (2nd Version) (APASO-II)

B. Introduction of E-platform for School Development and Accountability (ESDA)

C. Scales Selection Guidelines

D. The Design of APASO-II Survey

E. Group Discussion

Unit Two

F. User-defined Survey Publication and Data Import (Online)

G. Pre-defined Survey Publication and Data Import (Offline)

H. Workshop Exercise

Unit Three

I. Review of Survey Report

J. Reports in APASO-II and their interpretation

Raw Score Rasch Score

K. Data Interpretation Exercise

Appendix I. Answer Key for the Workshop Exercise (for reference)

Appendix II. Data Samples

Appendix III. Answer Key for Data Interpretation Exercise (for reference)