Analysis of the New Teaching Performance Assessment System for Private School Teaching Staff of Non-Tertiary Education in MacaoAbstract

(1)

Analysis of the New Teaching Performance Assessment System for Private School Teaching Staff of Non-Tertiary

Education in Macao

Abstract

This paper discusses the launch of school-based teaching performance assessment system under the new System Framework for Private School Teaching Staff of Non-tertiary Education (the Framework) in Macao on both private schools and teachers. Based on the theoretical threshold assessment framework, I analyze the potential advantages such as establishing standardized pay and rank system, and disadvantages such as unfavorable condition for teachers at the highest rank and the lowest rank. I study the new school-based teaching performance assessment system of MASS as a case study. It is designed according to the Guidelines for Establishing and Implementing a School-based Teaching Staff Performance Assessment System for Private Schools of Non-tertiary Education provided by the Education and Youth Affairs Bureau. I discuss its potential strengths, including a) has clear rubrics, b) includes triangulation approach, c) fits the purpose and d) rewards good performers; and limitations, including a) is too comprehensive, b) has central tendency error, c) is unsuitable for some subjects, d) lacks expert assessors, e) is not formative enough, f) limits chance of promotion in advance and g) has wash back effects. The findings indicate that the new school-based teaching performance assessment system has both good and bad effects, amendments should be carried out, and both school management teams and teachers need time and training to adapt to the new system.

Keywords: teaching performance assessment, professional development, Macao

Shirley M. W. Chan's E-mail: shirleymwchan@netvigator.com

Shirley M. W. Chan

Ph.D. student in Faculty of Education, University of Bristol

(2)

澳門非高等教育私立學校新教學人員工作表現評核校本制度的分析

陳明慧電子郵件：shirleymwchan@netvigator.com

陳明慧

英國布里斯托大學教育學系博士生

摘要

本文討論及分析在澳門非高等教育私立學校就新實施的《非高等教育私立學校教學人員制度框架》（簡稱私框）而建立的教學人員工作表現評核校本制度。根據升級評鑑的理論框架，分析私框潛在的優勢和劣勢，包括制定統一的職級及薪酬水平，對於最高職級及初入行的老師的不利情況。本文亦以一所私立

學校，MASS，作為一個案例研究，討論其根據教育暨青年局所提供的《非高等

教育私立學校教學人員工作表現評核校本制度的建立與推行參考資料》制定的新教學人員工作表現評核校本制度。分析顯示其優點包括清楚的評分規範、三角測量、切合目的、獎賞優異表現者；缺點包括評核指標太廣泛、趨中傾向、不適合

個別科目、欠缺專業評核員、形成性評價不足、提前晉升的機會有限、倒流效

應。研究結果表明，新校本評核制度有好的和壞的影響，需要作出適當修改，無論是學校管理隊伍和教師均需要時間和培訓，以適應新的制度。

關鍵詞：教學工作表現評核、專業發展、澳門

(3)

Introduction

According to Education and Youth Affairs Bureau (DSEJ)¹statistics, there were 83 licensed schools in Macao during the 2007/2008 school year, 13 of which were public schools and the remaining 70 were private. Private-school teachers comprise 92.9% of the total teachers in Macao and play an important role in the region’s education (Ho &

Choi, 2009a). There are historical and political reasons for the large portion of private schools in Macao. Macao was a colony of Portugal, and the Portuguese government did not develop education for the Chinese. Before the Christian and Catholic churches developed schools in the early 20th century, only one school had provided education solely for Portuguese children. These private schools have very different structures due to their different backgrounds. Because there is no standardized system in terms of teacher employment and salary scale, teachers with the same qualifications may receive very different levels of pay for performing the same job. Hence, teaching quality varies greatly among schools.

Besides, Macao has been developing rapidly since the liberalization of its gaming industry post-handover. As high quality human resources are required to cope with the development of the region’s economy, the Macao government has been actively carrying out educational reforms, including the implementation of 15-year-free- education. More private schools have since been established. The improvement of teaching quality has also been highlighted as a key to cultivating high-quality human resources. The government authorized the Beijing Normal University Center for Teacher Education Research, Ministry of Education (2006) to study the development of teacher professionalism in Macao, and the report showed that although professional development is positive, certain imbalances are present. For example, while teachers respected their jobs they were not happy with them. Teachers who were in the 3rd to 5th and 11th to 15th years of their careers respectively showed obvious signs of burnout. While their working hours were long and their workloads heavy, their salaries were low. Their autonomy of self-development was low. Because the culture of most schools did not enhance collaboration and communication within the schools, teachers did not receive support from their principals and schools, which in turn limited the teachers’ professional development. Many teachers have changed their career paths to the gaming industry (Ho & Choi, 2009b). In fact, many casinos target teachers due to their higher education qualifications and conduct. This shift has caused additional stress for the

1Direcção dos Serviços de Educação e Juventude (DESJ) is the official Portuguese name of the Education and Youth Affairs Bureau in Macao.

(4)

teachers who stay in the teaching profession. Hence, the government designed a career progression framework for private-school teachers to develop their professionalism and upgrade their status. The first consult paper was released in 2008 (Ho & Choi, 2009b). It suggested changing the payment calculation method from teaching hours to a standard overall payment. A career ladder with detailed requirements for promotion was also established. A great deal of feedback was received, and three main concerns with the proposed framework were revealed. The first concern was that there were too many levels. Although most of the teachers agreed that the proposed framework had a positive influence, they found that the year required for each promotion was too long a period.

Second, the schools and public thought the framework would make it hard for small schools to keep teachers, as they had fewer financial resources. The teachers suggested a dual system in which schools would keep paying the monthly salary according to their schools’ situations and the government would pay a standardized subsidy to teachers according to their levels on the career ladder. Third, the disunity of the teaching performance evaluation system was a major concern, as it meant that the system could not match the proposed framework. Other main concerns were the number of working hours, the further education required and the teachers’ retirement plan. The System Framework for Private School Teaching Staff of Non-tertiary Education (非高等教育私立學校教學人員制度框架) (the Framework) was ultimately passed after 5 years on 29 February 2012 and put into effect during the 2012/2013 academic year.

Brief Outline of the Framework

The Framework established the career progression of private-school teachers by dividing them into six levels (Education and Youth Affairs Bureau [DSEJ], 2012b). All of the current and new teachers must be evaluated by DSEJ to determine their ranks.

New teachers who register at DSEJ for the first time are at levels 5 or 6, depending on their educational qualifications. Current teachers’ starting ranks are determined by their academic qualifications, years of experience, job performance evaluations, and professional development. These are also the criteria for promotion. For example, to be promoted from level 5 to 4, teachers must complete 3 years of service, achieve a grade of ‘satisfactory’ or above in their work performance evaluations for 3 years and complete at least 90 hours of professional development activities. In addition, DSEJ gives the ‘Distinguished Teacher’ award to the teacher with outstanding performance.

Teachers who meet the requirements, which include the completion of the designated time of service and professional development, achieving an excellent work performance

(5)

evaluation and attaining an award or higher education achievement, can apply for rank promotion in advance. DSEJ grants subsidies to private school teachers for professional development according to their academic qualifications and levels of rank (Education and Youth Affairs Bureau, 2012b). Furthermore, teachers’ working hours are normally 36 hours per week, with around 14 to 23 teaching periods (Education and Youth Affairs Bureau, 2012b, p. 27). Overtime payment must be made if the teacher is required to work or teach overtime. Paid annual leave, free medical benefits and a provident fund system have also been introduced. The Framework establishes regulations for private schools and teachers. All these regulations such as pay and rank system did not exist in the past and so the quality and standard of teaching and education varied greatly among the private schools.

Analysis of the Framework Conceptual Framework

Based on a theoretical threshold assessment framework, the Macao government aims to motivate teachers to be more professional and carry out effective teaching practices that raise the region’s overall private-school teacher standards by establishing the “career progression and the basis of remuneration” with a threshold “as a mechanism that polices the gateway of career progression and promotion” (Mahony, Hextall, &

Menter, 2004a, pp. 132-133). The standard of each rank level and the requirements for promotion up the career ladder and rewards for good performance are specified.

A threshold assessment allows private-school teachers to see their career prospects in the teaching profession, clarify the criteria for promotion and set clear teaching and professional development targets.

A threshold assessment requires a good performance assessment system with clear performance standards. It “presumes that what is measurable (performance), via a technology of standards, can be demonstrated by completion of a form and provision of documentary evidence that demonstrates the veracity of the claims made” (Mahony et al., 2004b, p. 438). A clear, measurable, standardised performance assessment system is key to the success of a threshold assessment, as assessors need a method that matches the threshold assessment to evaluate teachers. Teachers also need to know what they will be evaluated on and how so they can adjust their work to achieve the requirements under the threshold assessment system. The clearer the standards are, the more objective the assessment can be. The assessment also shows the standard of the teachers to the public,

(6)

which in turn improves the status of the teachers. Moreover, a threshold assessment requires “teachers to demonstrate that they have met a number of standards in order to cross the threshold and to receive a pay award” (Haynes, Wragg, Wragg, & Chamberlin, 2003, p. 25).

The performance-related pay (PRP) method is often related to threshold assessment.

It allows teachers to receive better pay as a reward for their good performance. “The allocation of one-off bonuses, often on the completion of a particular project or in recognition of a specific contribution” and performance-related pay “becomes a regular part of the employees’ salary” (Chamberlin, Wragg, Haynes, & Wragg, 2002, p. 32) are methods that education sectors may adopt. The government gives a one-off bonus to teachers who train students who win international and inter school competitions. Private- school teachers with good performance may apply for promotion in advance under the Framework and it is a PRP method, which motivate teachers to perform better. It may also promote teacher retainment, new recruitment and the return of departed teachers (Chamberlin, Wragg, Haynes, & Wragg, 2002).

The application of a threshold assessment and PRP scheme in England is a good example. The Department for Education and Skills (DfES) aims to raise teaching standards by introducing a new performance management model. It assumes that the full potential of the education system can be realized if teachers are motivated by “ambition, incentives, training and support” (DfEE², 1998, cited in Mahony et al., 2004a, p. 132, 2004b, pp. 436-437). DfES aims to establish a system in which “effective teaching and career progression are partly defined through professional standards” for different levels of the career ladder, and teachers’ performances are assessed “to set targets for improvement, and a new pay policy that rewards good performance” (Mahony et al., 2004a, p. 132, 2004b, p. 437). Teachers who meet the threshold standards receive additional money and promotion to a higher rank. These are the incentives for “continuous improvement, used to reward the most effective teachers” (DfES 2002, cited in Mahony et al., 2004a, p. 132, 2004b, p. 437). However, according to Farrell and Morris’ (2004) research, over 80% of teachers felt that PRP would not increase motivation, “have little impact on teacher recruitment” and retention; over 70% felt it “would not lead to better and more effective teaching” and “little impact on pupil learning” (p. 89).

2The Department for Education and Employment (DfEE) changed its name to the Department for Education and Skills (DfES) in 2001.

(7)

Reward is an extrinsic motivation and there are many other intrinsic factors that motivate teachers including the need for autonomy, recognition and respect, etc.

Chang (2011) suggested that “both intrinsic and monetary values enhanced the work effort of employees. However, the moderation effects of monetary value were not found to be significant because the effect of intrinsic values was controlled.” (p. 3942) Hence, if these internal needs are unfulfilled, teachers may lose their motivation (Gokce, 2010).

Gius (2013) carried out a research on teachers’ job satisfaction in districts with and without merit pay system in USA. The findings showed that merit pay district teachers did not show higher job satisfaction than those in non-merit pay districts. Moreover, teachers of merit pay districts were “less enthusiastic about teaching and more likely to leave for better pay.” (Gius 2013, p. 4450) Nonetheless, teachers who received merit pay have higher job satisfaction than teachers who do not receive merit pay in the merit pay districts. Therefore, PRP may still be an effective method to motivate teachers performing better despite its disadvantages.

Advantages

There are arguably four advantages to the new system. First, private-school teachers are evaluated and ranked accordingly, and same rank-level teachers can receive the same amount of subsidy from the government. This lessens the same-job-different- pay situation in the past, and the subsidy can attract teachers to remain in the teaching profession and motivate them to further their professional development. Schools have introduced other fringe benefits such as lunch provision, travel, a book-purchasing fund and birthday and festival gifts to attract more high quality teachers and improve teachers’ working environments and living standards. The Framework also introduces a welfare system for the teachers that includes health care and a pension fund, providing more job security and protection after retirement. In addition, because working hours and workloads are lessened, teachers have more time to relax, prepare their lessons and, further their studies. Their stress levels are reduced and their quality of life is improved.

Second, the system can improve the status of private-school teachers and show them respect. It “recognizes achievement” (Mahony et al. 2004b, p. 439) and emphasizes professionalism, which in turn encourages teachers to autonomously upgrade their professionalism.

Third, good performers can apply for promotions in advance. Rewards are given out as a form of performance-related pay; hence, “good teachers feel more valued” (Mahony et al. 2004b, p.439), and teachers perform better as a result. Teachers are also made “more

(8)

aware of or more committed to certain organizational goals” (Chamberlin et al., 2002, p. 33), as those who achieve a school’s goal may be considered to perform better and may work effectively in achieving that goal.

Fourth, a teaching performance assessment “must be carried out by a collegiate body” (Education and Youth Affairs Bureau, 2012b, p. 20). This is fairer and more objective than the present assessment system used by most schools, which is conducted solely by the principal or department supervisor. It provides teachers with the opportunity to participate in the assessment committee in designing and implementing the assessment system, making it “more likely to be successful and acceptable” (Farrell & Morris, 2004, p. 83).

Disadvantages

Although there are many potential advantages to the Framework, there are also three potential disadvantages. First, teachers who have been teaching for 25 years or more before the implementation of the Framework are ranked at level 1. This means they have no opportunity for promotion under the Framework’s career progression, and thus have no incentive to perform better at either teaching or professional development.

Besides, while the subsidy amount will stay the same, those teachers who are at a lower level can get a raise when they are promoted up the career ladder. Thus, a situation may occur in which “good performers get stuck on the top of the scale, and even the poor performers will get there eventually” (Chamberlin et al., 2002, p. 32). On the other hand, a new teacher requires at least 20 years to be promoted from the entry level to the top level. Mr. Tam Ka Lok (談家樂老師) made the following statement in an interview about teachers’ perspectives on the new Framework:, “One can anticipate 20 to 23 years at least to get to the top of the rank under the Framework and it is too long and inflexible.

It makes youth hesitate in joining the teaching profession.” (Leung & Wai, 2012, p. 25) Second, continuous professional development may cause stress for teachers. DSEJ does not have clear guidelines on relevant courses, and this may result in teachers taking too many courses and becoming burned out.

Third, teachers have to apply for promotions in advance by themselves. They must provide evidence that they are good enough to be promoted in advance, and they may feel insulted. Mahony et al. (2004b) argued that “requiring experienced teachers to undergo a further process of proof to access a pay rise was a misconceived way of motivating teachers” (p. 440) and a teacher interviewed stated, “It would be nice if

(9)

somebody came to me and said, “I think you’ve done a jolly good job for the last five years, I think you deserve the extra £2000” (p. 440)

Teaching Staff Performance Assessment System at MASS

As mentioned earlier, threshold assessment requires an assessment system with clear

‘standards’ that are measurable. Whether the teachers are performing well or poorly and what extent depends on the criteria of the new teaching staff assessment system. Private schools are required to design their own teaching performance assessment systems, which will be implemented in 2013/2014, according to the Guidelines for Establishing and Implementing a School-based Teaching Staff Performance Assessment System for Private Schools of Non-tertiary Education (非高等教育私立學校教學人員工作表現評核校本制度的建立與推行參考資料) (Education and Youth Affairs Bureau, 2012a).

It suggests a standard-based evaluation model that “defines domains of teaching practice and establishes professional standards of teaching performance within the domains as the foundation of measures of teaching performance” (Milanowski & Heneman III, 2001, p.

194). The model presents four teaching domains, which are subdivided into 10 criteria, and each criterion is further divided into the different standards shown in Table 1.

Table 1

Structure of Standard of Teaching Performance Assessment Model(教師教學職責的評

核指標架構示例)

Domains Criteria Standards

School and curriculum development

School development 1. School development planning 2. Interpersonal relationships at school Curriculum development 3. Curriculum development work Practice of teaching Teaching organization 4. Teaching plan

5. Goal achievement and content presentation

Teaching process 6. Use of teaching method 7. Lesson management

Learning assessment 8. Learning assessment practice and use of the result

Student learning attitude 9. Display of learning attitude Classroom

management and student support

Classroom management 10. Classroom regulation and environment Student support 11. Student counselling

12. Home-school cooperation Professional

development Professional growth 13. Professional growth performance Professional attitude 14. Professional attitude performance

Note. This table is from Education and Youth Affairs Bureau (2012a, appendix 6)

(10)

There are five performance levels for each standard: ‘excellent’, ‘very satisfactory’,

‘satisfactory’, ‘not very satisfactory’ and ‘unsatisfactory’. These qualitative levels are assigned marks ranging from 5 to 1 respectively. The rubrics for each standard level have been specifically written for teachers with teaching duties only and with both teaching and non-teaching duties.

Every private school must form a number of assessment committees to carry out a teaching staff performance evaluation. The number of committees formed can reflect the structures and sizes of the schools, which perform the assessments in different forms or at different levels. According to the regulation, each assessment committee should have at least three teaching staff members, one of whom serves as chairman and one third or more of which should consist of non-middle or high management teaching staff. It is suggested that schools should use multiple methods to evaluate teacher performance such as class observations, review documents and self-assessments. The guidelines suggest that three assessment meetings should be held with teachers each academic year. The first meeting must be arranged at the beginning of the academic year to account for the teachers’ responsibilities and clarify their job expectations. Assessment committees can make appointments with individual teachers who are experiencing performance problems to review the problems and offer feedback and recommendations for improvement.

Education and Youth Affairs Bureau (2012b) emphasizes that the assessment “must be carried out by a collegiate body” (p. 20), and one final mark must be agreed upon and given to each teacher after discussion within the assessment committees at the end of each academic year.

Analysis of the New Assessment System

Method. I study the teaching performance assessment system at MASS, a private school in Macau, as a case study within the academic year 2013-2014. I analyze the potential strengths and limitations of the new assessment system, and whether it matches the new Framework and measures the teacher performances. I interviewed members of the school’s Assessment System Preparatory Committee, which was formed to design the assessment system. A general interview guide approach was employed which “is a set of issues, developed before the interview takes place that the interviewer wants to discuss with the respondent.” (LeCompte & Preissle, 1993, p.169) It provides a checklist assuring that all relevant topics are covered during the interviews and relevant information may be addressed at any time during the interviews. Besides, it “is more systematic than the conversational interview and less restrictive than the standardized

(11)

open-ended one.” (Ho, 2010, p.270) It also provided me freedom to raise relevant questions during the interviews which I had not thought of before the interviews. I attended two teaching staff meetings, which mainly discussed the new assessment system, and carried out observation, especially focus on the teachers’ comments to the new assessment system. I also reviewed the relevant school and government documents, including drafts of the assessment standard, teachers’ comment summary report and the system trial result. Consent to use the school documents was requested from and approved by the school authority.

Strength. The new system has four main potential strengths: it a) presents clear rubrics for each standard level, b) takes a triangulation approach, c) displays a fit for the purpose and d) rewards good performers:

a) There are clear rubrics for each standard level, and “specific behavioral descriptions” (Milanowski & Heneman III, 2001, p. 195) of five performance levels for each standard. The first draft of Standard and performance levels of the teaching performance assessment for the teaching staff (subject teacher) at MASS is shown in Appendix.³ Distinctions between performance levels are clear as illustrated by the first standard, school development planning. Teachers must make ‘important contribution’, actively ‘facilitate’ the construction of the school-based development and show ‘exceptional’ performance to achieve level 5. Meanwhile, teachers who ‘understand clearly’, and actively ‘implement’ the school-based development and can ‘suggest concrete opinion’ are evaluated at level 4. The specific behavioral elements that are required at the different levels can be identified immediately. The assessors can rate the teachers’ performances easily by identifying their specific behavioral elements. Teachers should also clearly understand “the targets and standards of performance required” (Farrell

& Morris, 2004, p. 82) to achieve a higher performance rating and ensure the success of the threshold assessment, as the system provides “a degree of clarification of roles and expectations” (Milanowski & Heneman III, 2001, p.

208).

b) A triangulation approach is used. Broadfoot (2007) stated that: the assessment made can be ‘objective’ only to the extent that it comprises multiple subjectivities, involving the judgements of several different people, in different contexts and at different times. In the world of evaluation, this approach is

3The｀Standard and performance levels of the teaching performance assessment for the teaching staff (subject teacher) at MASS´was written in Chinese, and I translated it into English.

(12)

known as ‘triangulation’, and is an important mechanism for reaching the best possible understanding of a particular phenomenon. (p. 96)

MASS’s new teaching staff assessment system covers pretty much all of the aspects on which teachers should be evaluated. Besides, teachers with different duties are evaluated according to different standards, e.g., teachers who act as class advisors have to be evaluated according to the extra standard no. 10 (Appendix). As MASS provides education from the kindergarten to senior secondary levels in addition to vocational education, minor differences in standard elements are present to match the scope of the teachers’ work in different divisions. Moreover, MASS uses weighted average method to reflect individual teacher’s different job nature and the corresponding assessment criteria. All committee members ultimately have to discuss their evaluations and award a final mark. Education and Youth Affairs Bureau (2012b) emphasizes that the assessment

“must be carried out by a collegiate body” (p. 20), and one final mark must be agreed upon and given to each teacher after discussion within the assessment committees at the end of each academic year. This is fairer and more objective than the present assessment system used by MASS, which is conducted solely by the principal or department supervisor. It also provides teachers with the opportunity to participate in the assessment committee in designing and implementing the assessment system, making it “more likely to be successful and acceptable” (Farrell & Morris, 2004, p. 83). Furthermore, different assessment methods such as observation, peer assessment and self-assessment are used and carried out at different times during the academic year. This is the triangulation approach, and it enables justice and equality in the assessment.

c) The system is fit for both “quality assurance (summative) and professional learning (formative)” purposes (Danielson & McGreal, 2000, cited in Milanowski & Heneman III, 2001, p. 193), which match the purposes of the Framework. MASS’s assessment committees will arrange the first meeting with teachers at the beginning of the academic year to account for the teachers’

responsibilities and clarify their job expectations. Assessment committees can make appointments with individual teachers who are experiencing performance problems to review the problems and offer feedback and recommendations for improvement. The final mark given at the end of each academic year serves as a summative assessment and summary of “the progress of an individual in relation to” (Broadfoot, 2007, p. 110) the criteria. According to Broadfoot (2007), it is also similar to an appraisal, “the two most common purposes are what we

(13)

might term ‘managerial’ – for selection, promotion and the reward of merit;

and ‘professional development’ – for enhancing skills and professionalism” (p.

97). The appraisal is used to rate teachers’ performances in consideration of promotion and special rewards, and as “means of targeting professional updating needs more accurately and in helping individuals with their career planning”

(Broadfoot, 2007, p. 97). In addition, it may serve the additional purpose of system monitoring. It provides an overall standard for private-school teachers and shows the rise and fall of a school’s teaching performances over time. In turn, we come to know whether the new Framework is succeeding (Newton, 2007).

d) Rewards are given out as a form of performance-related pay; hence, “good teachers feel more valued” (Mahony et al., 2004b, p.439), and teachers perform better as a result. Teachers are also made “more aware of or more committed to certain organizational goals” (Chamberlin et al., 2002, p. 33), as those who achieve a school’s goal may be considered to perform better and may work effectively in achieving that goal. For example, at MASS, an one-off bonus is given to music teachers whose students win competition reflects what is considered important to the management of MASS. Good performers may also apply for promotions in advance and get pay and subsidy rise. This motivates teachers to perform better.

Limitations. There are arguably seven main problems with the new assessment system: it may a) be too comprehensive, b) present a central tendency error, c) be unsuitable for some subjects, d) present a lack of expert assessors, e) not be formative enough, f) limit the chance of promotion in advance and g) have washback effects:

a) Although detailed rubrics are available, an overabundance of evaluation standards may make it hard for the teachers and assessors to fully understand all of the elements. It is very important for teachers to understand the details of rubrics in order to make the system work (So & Lee, 2011) but not many teachers have read the draft issued. Most of them stated that they did not have time and it was too comprehensive. The only question and comment raised during the teaching staff meeting is the weighed average method used.

For example, a primary teacher asked. “How do determine the mark ratio of different standards?” Some teachers also suggested different mark ratios

(14)

according to their roles in the teachers’ comment summary report. Moreover, assessment committees may also take a long time to go through and discuss teachers’ performance evaluation reports. Some elements are “less definable and observable” (Broadfoot, 2007, p. 99) such as ‘arousing students’ growth’ through the ‘use of the teaching method’ according to standard No. 6 (Appendix), which is very difficult to identify from a simple class observation. Teachers may need to provide a great deal of evidence to support their performance. This makes the workloads of both the teachers and assessors heavier. Moreover, according to teachers’ comment summary report, some teachers are concerned with the quantitative terms used such as ‘seldom’ and ‘often’ and suggest specific range or number should be used instead.

b) As there are five performance levels, a central tendency error may occur.

The assessors may tend to give middle grades to the teachers, especially if it seems very difficult for a teacher to attain the highest standard levels. I once again use standard No. 1 as an example. Teachers have to make an ‘important contribution’ and have ‘exceptional’ performance to attain level 5, but what constitutes an important contribution, and what kind of performance is considered exceptional enough to allow one to transcend level 4? Teachers do not know what they must do to achieve level 5, and assessors in turn may be persuaded to give them middle grades. Staff A, an administrator and member of the Assessment System Preparatory Committee, made the following statement during the interview:

Four performance levels can prevent the central tendency error and are easier to rate. The model is too comprehensive and I do not think there is any person who can be so perfect to achieve the highest level of all standards. (Staff A)

Danielson’s (1996) four performance levels, including ‘unsatisfactory’, ‘basic’,

‘proficient’ and ‘distinguished’, may be more suitable for teacher evaluation.

Explanation of the level of performance is shown in table 2.

(15)

Table 2

Danielson’s Four levels of Performance Level of performance Explanation

Unsatisfactory Does not understand concepts underlying the standards and should work on the fundamental practices associated with the elements of different standards.

Basic Understands concepts underlying the standards and attempts to implement their elements but is not entirely successful, additional training should be carried out, particularly supported by a mentor.

Proficient Clearly understands the concepts underlying the standards and implements them well. Experienced and capable on different standards.

Distinguished Master teachers and make contribution both in and out of their schools. Their classrooms operate at a high quality level with highly motivated and engaged students, and responsible for their own professional learning.

The vice-principal (VP) and staff A once raised this issue at the preparatory committee meeting and suggested the use of four performance levels instead.

However, other members observed that DSEJ required an assessment system of five performance levels, and the suggestion was turned down. Nevertheless, because the five performance levels in the guidelines comprise only a model, and because the Framework only requires qualitative comments, there is no statement in either document that indicates that five performance levels are required. This reflects another problem: members of the preparatory committee have different interpretations of the guidelines. The VP’s comment may explain the situation, “We are attending different training sessions offered by DSEJ. The one I attended was a more comprehensive one with workshops for practicing designing and evaluation, which was not held afterwards.” These different interpretations may lead to conflicting evaluation results and confusion within the committees. Furthermore, a trial of the new assessment system was carried out in June 2013 and 25% of teachers were evaluated. The result reflects that central tendency error may actually occur as most teachers achieve ‘satisfactory’

and ‘very satisfactory’ grades. Two teachers have ‘not very satisfactory’ and nobody gets ‘excellent’. Besides, when compared with the result of the present assessment system, all teachers get lower marks. This reflects problem of the grading system, the ‘excellent’ grade is too hard to achieve. The new grading system is shown in table 3.

Note. It is from Danielson (1996, p. 36)

(16)

Table 3

MASS’ new teaching staff grading system

Grade Mark Range

Excellent 86 or above

Very satisfactory 71-85

Satisfactory 56-70

Not very satisfactory 46-55

Unsatisfactory Under 45

c) The standards may not “apply well to the subject areas of art, language, music, and special education” (Milanowski & Heneman III, 2001, p. 205) as the teachers of these subjects may conduct totally different teaching methods, student assessments and interactions with the students. However, no special standard is designed particularly for these teachers. A music teacher expressed her doubt on some criteria, e.g., professional development, “DSEJ doesn’t offer any music- teaching professional development course. I conduct many performances which are important to my career development but do they count as professional development under the new system?” I am a drama and special education teacher and I cannot identify any specific details of the standards that are relevant to my work. It cannot truly tell my performance in reality.

d) The expertise and justice of the assessors is questionable. The committee members may have to evaluate teachers of different subjects. Teachers may doubt these members’ qualifications to evaluate them, “including their subject knowledge…and the type and quality of feedback they provided.” (Milanowski

& Heneman III, 2001, p. 208). This directly affects the teachers’ confidence in the new system. A teacher of MASS told me, “my performance rating is totally depended on the assessors’ subjective impression on me, not the kind of assessment system used.” If teachers do not believe in the assessment system, they will not perform accordingly and assessors may judge teachers’

performance as ‘unsatisfactory’ and in turn influences teachers’ promotion and pay.

e) The system is not formative enough. If the assessment report is not given until the end of the academic year, teachers may receive no timely or useful feedback from the assessors and miss the opportunity to make modifications or upgrade themselves to attain a better performance level. Although teachers who perform

(17)

unsatisfactorily may receive feedback mid-academic year, they may not receive enough assistance for improvement afterwards. Teachers who perform well may not meet with their assessors enough to improve their performances and reach a higher level. The system is not formative enough to achieve the enhancement of teachers’ professional development, which is one of the main goals of the Framework.

f) Teachers who complete the designated time of service and professional development, achieve excellent work performance evaluation and attain an award or higher education achievement, can apply for rank promotion in advance. However, as it is difficult to achieve ‘excellent’ grade, the chance of getting promotions in advance is limited. This is a PRP used to motivate teachers but the new assessment system does not cope with it.

g) Although teachers may understand the schools’ goals and work towards them to achieve better grades and awards, it may lead to washback effects, such as a “largely negative emotional impact on teachers”, “discipline problems and disaffection” in students and the diversion of “teachers’ time and attention from the core business of teaching” (Broadfoot, 2007, p. 102). For example, the one- off bonus given to music teachers causes them to arrange extra practices for the students, even during other subject lesson times. This does not only affect the students’ studies but also causes other subject teachers to complain. Students may feel bad in turn, and thus their relationships with their teachers are harmed:

Students could become further reduced to the means through which teachers meet their targets (or not). Staff frustration with children perceived as impeding teachers’ progress against targets is likely to find its counterpart in pupil resistance to being viewed instrumentally. (Mahony et al., 2004b, p. 453)

Moreover, these music teachers are likely to be evaluated as better performers and get promotion and pay rise because they can provide evidence that they are achieving the goal of the school, which may mean making an ‘important contribution’ and showing ‘exceptional’ performance. This may cause further resentment from other teachers who are performing well in their subjects and other aspects, but are not being recognized.

(18)

Conclusion and Recommendations

The new teaching performance assessment system has many potential advantages.

The standards are clear and comprehensive. It motivates teachers to perform better and develop their profession. It matches the Framework. However, there are many potential disadvantages too. The trial result also shows that teachers may get lower performance ratings and will definitely demotivate the teachers because the system directly affects the teachers’ job promotion and pay under PRP scheme. In turn, it may have negative effect on teachers’ performance and professional development, which contradicts the Framework. I strongly suggest that MASS’s school management and Assessment System Preparatory Committee should consider the comments of the teachers and revise the details of the standards to make it clearer and more measurable. For example, the number of hours of professional development should be included instead of just using

‘often’ and ‘seldom’. I also suggest an amendment on the grading system so that the

‘excellent’ grade is achievable. Moreover, one of the purposes of the establishment of the Framework is to set regulations protecting teachers. School assessors have to give reasons and explanations for the marks given to teachers. There is new appeal system and regulation so that teachers can make an appeal to DSEJ, which increases teachers’ job security (Anonymous, 2012). MASS must enforce this regulation and ensure teachers understand the new appeal system to increase their confidence in the new assessment system.

References

Anonymous. (2012, April 23). 教師關注私框評核公平性. Macau Daily News.

Beijing Normal University Center for Teacher Education Research, Ministry of Education Key Research Bases for Humanities and Social Sciences. (2006). Report on the research of the development of professionalism of teaching staff in Macao (in Chinese). Retrieved from http://portal.dsej.gov.mo/webdsejspace/internet/Inter_main_page.jsp?id=#

Broadfoot, P. (2007). An introduction to assessment. New York, NY: Continuum.

Chamberlin, R., Wragg, T., Haynes, G., & Wragg, C. (2002). Performance-related pay and the teaching profession: A review of the literature. Research Papers in Education, 17(1), 31-49.

doi:10.1080/02671520110102534

Chang, E. (2011). Motivational effects of pay for performance: A multilevel analysis of a Korean case. The International Journal of Human Resource Management, 22(18), 3929-3948.

Danielson, C. (1996). Enhancing professional practice: A framework for teaching. Alexandria, VA: Association for Supervision and Curriculum Development.

(19)

Education and Youth Affairs Bureau. (2008). Educational statistics: 2007/2008 statistics of schools. Macao Special Administrative Region of the People´s Republic of China. Retrieved from http://portal.dsej.gov.mo/webdsejspace/internet/Inter_main_page.jsp?id=8525&#Inter_

main_page.jsp?id=8525

Education and Youth Affairs Bureau. (2012a). Guidelines in establishing and implementation of school-based teaching staff performance assessment system for private school of non- tertiary education (in Chinese). Macao Special Administrative Region of the People´s Republic of China. Retrieved from http://portal.dsej.gov.mo/webdsejspace/addon/upload/

Upload_viewfile_page.jsp?id=30197&sid=&

Education and Youth Affairs Bureau. (2012b). System framework for private school teaching staff of non-tertiary education. Macao Special Administrative Region of the People´s Republic of China. Retrieved from http://www.dsej.gov.mo/~webdsej/www/edulaw/201203/

lei201203_new-e.pdf

Farrell, C., & Morris, J. (2004). Resigned compliance: Teacher attitudes towards performance- related pay in schools. Educational Management Administration & Leadership, 32(1), 81- 104. doi:10.1177/1741143204039301

Gokce, F. (2010). Assessment of teacher motivation. School Leadership & Management:

Formerly School Organisation, 30(15), 487-499.

Gius, M. (2013). The effects of merit pay on teacher job satisfaction. Applied Economics, 45(31), 4443-4451.

Haynes, G., Wragg, C., Wragg, T., & Chamberlin, R. (2003). Threshold assessment: The experiences of teachers who were unsuccessful in crossing the threshold. Research Papers in Education, 18(1), 25–44. doi:10.1080/0267152032000048569

Ho, C. S. B., & Choi, K. L. R. (2009a, March). Improving teacher professionalism: Issues, policy, and development (in Chinese). Teachers’ Magazine, 26, 29-33.

Ho, C. S. B., & Choi, K. L. R. (2009b). Policy consultation on institutionalizing private school personnel system in Macao (in Chinese), Revista de Administração Pública de Macau, 85(3), 525-535.

Ho, D. C. W. (2010). Leadership for school improvement: Exploring factors and practices in the process of curriculum change. Early Education & Development, 21(2), 263-284.

LeCompte, M. D. & Preissle, J. (1993). Ethnography and qualitative design in educational research. San Diego, CA: Academic.

Leung, W. W., & Wai, F. K. (2012, March). Interview of four teaching staff from different schools about their perspectives on ｀System Framework for Private School Teaching Staff of Non-tertiary Education´ (in Chinese). Teachers’ Magazine, 36, 22-26.

(20)

Mahony, P., Hextall, I., & Menter, I. (2004a). Threshold assessment and performance management: Modernizing or masculinizing teaching in England? Gender and Education, 16(2), 131-149. doi:10.1080/09540250310001690546

Mahony, P., Menter, I., & Hextall, I. (2004b). The emotional impact of performancerelated pay on teachers in England. British Educational Research Journal, 30(3), 435-456. (doi:10.1 080/01411920410001689724)

Milanowski, A. T., & Heneman III, H. G. (2001). Assessment of teacher reactions to a standards- based teacher evaluation system: a pilot study. Journal of Personnel Evaluation in Education, 15(3), 193-212.

Newton, P. E. (2007). Clarifying the purposes of educational assessment. Assessment in Education: Principles, Policy & Practice, 14(2), 149-170.

So, W. W. M., & Lee, T. T. H. (2011). Influence of teachers´ perceptions of teaching and learning on the implementation of assessment for learning in inquiry study. Assessment in Education: Principles, Policy & Practice, 18(4), 417-432.

Appendix

Standard and performance levels of the teaching performance assessment for the teaching staff (subject teacher) at MASS (first draft)

(Excellent)5 4 (Very satisfactory)

(Satisfactory)3 2 (Not very satisfactory)

(Unsatisfactory)1

1. School development planning

Have important contribution and facilitate the construction or amendment of the school-based development planning actively, have exceptional performance.

Understand clearly and implement actively the school-based development planning, and suggest concrete opinion.

Understand and work in coordination with the school-based development planning.

Participate passively in the implementation of the school- based development planning.

Do not work in coordination with the school-based development planning.

2. Inter- personal relationships at school

Have mutual- support with internal staff, firmly linked together, enhance team spirit of teachers into full play, able to build harmonious working atmosphere.

Have harmonious and close relationship with internal staff, encourage good working atmosphere, able to bring team spirit.

Have friendly relationship with internal staff, coordinate with the harmonious working environment, show a certain team spirit

Lack of interaction with internal staff and team spirit.

Serious conflict with internal staff often occurs, destruct harmonious working

environment, lack of team spirit.

Marks Standards

(21)

3. Curriculum development work

Facilitate the work in curriculum planning under school-based development, response to students´

learning need, make great contribution initiatively and actively.

Eager to institute curriculum objective, content, teaching tactic, ensure all courses of each level match effectively, study the linkage of the subject curriculum with other subjects, look for collaboration opportunity initiatively.

Participate actively in curriculum development work, show commitment, encourage curriculum development in school.

Institute curriculum objective, content, teaching tactic, try hard to link courses of different levels reasonably, cooperate in the collaboration with other subjects.

Understand and participate in curriculum development work, assist in promoting curriculum development in school.

Attempt to institute curriculum objective, content, teaching tactic, make linkage among courses of each level.

Participate passively in curriculum development work.

Attempt to institute curriculum objective, content, teaching tactic, but cannot link up courses of all levels.

Seldom participate in curriculum development related activity, with negative attitude.

Do not institute curriculum objective, content, teaching tactic.

4. Teaching

plan Teaching plan is brilliant, enhances students´

interest and ability to learn autonomously, worth promoting.

Fully master teaching progress, review at the right time and make suitable amendment.

Share the workload of the subject panel initiatively and implement actively the teaching and subject activity work given by the subject panel.

Teaching plan is student- orientated, creative, consider students individual differences, encourage students to learn.

Master teaching progress, always review, and make suitable amendment.

Implement actively the teaching and subject activity work given by the subject panel.

Teaching plan fits in curriculum target and match students mental and intellectual development, with complete contents and feasible.

Review teaching plan progress and make suitable amendment.

Implement the teaching and subject activity work given by the subject panel.

Teaching plan can still fit in curriculum target, but content incomplete.

Seldom review teaching plan progress.

Implement the teaching and subject activity work given by the subject panel passively.

Teaching plan do not match the curriculum target sufficiently, have loopholes, not feasible enough.

No review on teaching plan progress.

Do not implement the teaching and subject activity work given by the subject panel.

(22)

5.Goal achievement and content Presentation

Organized, systematic and precise presentation of teaching content, achieve teaching goal effectively, respect

students´ study performance and make suitable modification to the teaching content, enhance students establish knowledge effectively and learn autonomously.

Participate actively in group lesson preparation, make effective, creative and multi- dimensional suggestion on teaching content, have initiative to assist other teachers in achieving teaching goals.

Organized, systematic and precise presentation of teaching content, achieve teaching goal effectively, make modification to the teaching content according to the students´

situation, encourage students to learn.

Participate in group lesson preparation, make suitable suggestion and assist other teachers in achieving teaching goals.

Present teaching content correctly and clearly, achieve teaching goals.

Attend group lesson preparation, make suggestion occasionally.

Unable to fully present teaching content according to teaching goal or students´

situation, unable to reflect clear teaching goal.

Attend group lesson preparation passively, seldom make suggestion.

Not able to present teaching content according to teaching goal, explain wrongly.

Often absent from group lesson preparation without reason.

(23)

6. Use of teaching method

Master the individual differences of student development, and use diverse teaching methods according to their needs, strongly inspirational and inductive, show brilliant communication skill, enhance student thinking, establish active and interactive learning atmosphere, make good use of every learning environment and resource, arousing students´

growth and improve their learning outcomes.

Research actively on improving teaching materials and methods, choose the course materials that match the students´

learning need and ability. Review regularly, reflect and improve teaching quality.

Select suitable teaching methods and materials to match students´

development, inspirational and inductive, show good communication skill, enhance interaction and students participation.

Pay attention to students´

situation and choose suitable teaching methods and materials, have communication skill, maintain lesson learning atmosphere.

Improve teaching materials and methods.

Incapable of paying full attention to students situation and choose suitable teaching methods and materials, communication skill is okay but insufficient interaction.

Unsuitable teaching methods, lack of communication skill, express unclearly and no interaction.

(24)

7. Lesson

managemen Make good use of lesson management skill, strengthen student good performance at the right time, handle sudden incident properly, lesson very lively but maintain good order.

Manage lesson in good order, enhance student good performance, able to handle sudden incident.

Able to manage lesson in order, maintain good learning environment.

Incapable of managing basic lesson order occasionally

Disorder in lesson often occurs.

(25)

8. Learning assessment practice and use of the result

According to the practical need, make use of suitable and diverse assessment methods, evaluate students learning in all aspects effectively. Make good use of the assessment result, analyze

and review students´

assessment performance, and modify teaching tactic.

Enable students and parents understand the assessment result and meaning clearly, give feedback at the right time, optimize teaching, able to arrange continuing lesson learning opportunity, strengthen and deepen the lesson learnt, give good guidance patiently, respect and encourage student learning and self- assessment.

Set test paper according to the preset syllabus, fully master the assessment standards so as to achieve the unity standard of assessment.

According to the practical need, make use of diverse assessment methods, evaluate students learning at the right time, able to help students improve learning according to the assessment result, and improve teaching, enable students and parents understand the assessment result and meaning.

Set test paper according to the preset syllabus;

understand the assessment standards before marking so as to achieve the unity standard of assessment.

Implement assessment, review students learning situation and their assessment performance, modify teaching accordingly so as to enhance student learning but the result is not obvious.

Implement assessment for learning passively, evaluation accurately but incapable of reviewing overall performance of student assessment fully.

Set test paper according to the preset syllabus, but incomplete.

Incapable of master the assessment standards, cannot achieve the unity standard of assessment.

Disorder in lesson often occurs.

Incapable of implementation of learning assessment, wrong evaluation, incapable of reviewing student learning situation.

Do not set test paper according to the preset syllabus. Do not assess according to the assessment standard.

(26)

9. Display of learning attitude

Enable all students interested and confident in learning the subject greatly, actively plan and carry out in-class and extracurricular activity plan in order to enhance students to learn autonomously.

Enable most students interested and confident in learning the subject greatly, set and organize in-class and extracurricular activity plan so as to make students learn autonomously.

Enable some students interested in learning the subject. Arrange in-class and extracurricular activity plan, enhance students interest and confidence in learning.

Incapable of raising students´

interest in the subject.

Most students are not interested in learning the subject.

(27)

10. Classroom regulation and environment

Set reasonable classroom rules with students, establish clear vision and target together, make good use of effective tactic to arouse class cohesion and learning atmosphere, students are highly autonomic and obedience, support each other, be a model. Create an environment where teacher and student can interact well.

Establish good learning culture.

Assign students suitable class duties and posts according to individual ability, character and interest. Instruct students how to handle different class duties, train them to be independent and responsible.

Set reasonable classroom rules with students, establish vision and target, make effort to arouse class cohesion and learning atmosphere, students are autonomic and obedience.

Establish an environment where teacher and student may interact. Establish good learning culture.

Set classroom rules with students; take action to facilitate the establishment of class vision and target, good class order.

Establish an environment where teacher and student may interact.

Assign students class duties according to individual ability.

Set classroom rules, take action to facilitate the class interaction, but does not implement completely.

Do not implement any class rules;

classroom is in dispute, low learning atmosphere.

Do not assign student any class duty and post.

(28)

11. Student

counselling Use effective tactic to understand student autonomously, care about students individual needs, capture the learning sensitive period and make good use of every resources, prepare to influence their intellect and physical development, provide diverse and suitable student counselling actively, accompany their growth, reflect love to students totally.

To understand student autonomously, care about students individual needs, provide suitable counseling actively, reflect love to students.

Care about students´

individual needs, provide counseling, and love the students.

Provide counseling passively, haven´t search for suitable counseling for students.

Do not care student needs, show negative attitude towards counseling.

(29)

12. Home- school cooperation

Contact closely with parents in an understanding, positive attitude and effective tactic, establish supporting cooperative relationship, support students growth effectively.

Listen to parents´

suggestion and complaint, follow up and make modification, improve quality of home-school cooperation.

Contact closely with parents, enhance supporting cooperative relationship, support students growth.

Listen to parents´

suggestion and complaint, and follow up.

Contact with parents, maintain effective communication, and help students´

growth.

Listen to parents´

suggestion and complaint.

Contact with parents passively，do home-school cooperation work passively.

Insufficient communication with parents, negative attitude towards home-school cooperation.

(30)

13. Professional growth performance

Set life planning for professional development which can manage both personal live and school work, pursue continuously for personal growth and development in order to upgrade teaching career.

Understand deeply about educational reform trend, match with the trend in the right way, evaluate and reflect teaching actively, carry out research, make modification and study in the inadequate area.

Share

professional work and research experience with teachers through formal or informal channel, be a facilitator, make lots contribution.

Plan personal professional development, pursue

continuously for personal growth and development in order to upgrade teaching career.

Understand the objective and main point of educational reform, evaluate and reflect personal teaching seriously, participate in research, improve the inadequate area.

Share professional experience with other teachers through every channel, have good effect.

Participate in relevant professional development training seriously.

Share professional experience with other teachers, use it in teaching.

Participate in relevant professional development training passively, do not improve teaching.

Seldom discuss teaching experience with colleagues.

Do not participate in professional development training.

Hardly share with colleagues, never improve teaching.

(31)

14. Professional attitude performance

Have professional will, strong sense of commitment, keep teaching professional integrity and responsibility.

Passionate in teaching career, consider student welfare as the most important thing, put effort into teaching student and every school work, participate autonomously in inter and outer school activity.

To teach by word and deed. Be a model.

Make good use of resources and opportunities provided by government, school and society;

participate in professional development activity actively in order to upgrade professional knowledge and spirit.

Respect and like the job, follow teaching professional integrity, consider student welfare as the most important thing, put effort in student development, teaching student and every school work, participate autonomously in inter and outer school activity.

Often use resources and opportunities provided by government, school and society, participate in professional development activity in order to upgrade professional knowledge and spirit.

Follow teaching professional integrity, consider student welfare as the most important thing, do well in teaching relevant work, participate in inter and outer school activity upon request.

Use resources and opportunities provided by government, school and society;

participate in professional development activity occasionally.

Carry out teaching activity in a negligent manner, not passionate in improving school and student study, act as an outsider.

Sometimes late or absent with reason, participate in inter and outer school activity passively.

Seldom us resources and opportunities provided by government, school and society, and participate in professional development activity.

Seldom participate in teaching relevant activity, often absent unreasonably, unpunctual, always late without reasonable reason, negative attitude towards participation in inter and outer school activity.

Delinquent, hinder student or school rights.

Do not use or waste resources and opportunities provided by government, school and society.

(32)

Analysis of the New Teaching Performance Assessment System for Private School Teaching Staff of Non-Tertiary Education in MacaoAbstract