Chapter 5 The Interview Analysis and Discussion (B): Passing Ratio of
5.1 The research on maintaining the failure ratio of evaluation
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chapter 5 The Interview Analysis and Discussion (B):
Passing Ratio of Evaluation and the Exercise of Civil Right
This chapter continues to record the statements during interviews of this study, and the discussion of interview results. The major topics of this chapter are the passing ratio of evaluation and the exercise of civil rights.
5.1 The research on maintaining the failure ratio of evaluation
Students study and take exam at school hoping to pass basically, or they will restudy, delay graduation and even drop out of school seriously. The departments have the same mood when participating in higher education evaluation. They are thinking of passing the evaluation and hope to get better review comments. HEEACT takes the recognition system, but there are the evaluation results of "To be observed".
The results of vocational evaluation system show the ranking, and are directly related to grant awards of MOE. In order to maintain accreditation standards, do evaluation results have to maintain a certain failure ratio? This is the subject that the programs being evaluated mostly worried about. It is also suggested that evaluation results should demonstrate the effectiveness of elimination. That is, educational performance of poor schools being given a certain degree of punishment. Departments which cannot meet the requirements of MOE, less than 13 full-time faculty of the department, or less than 7 faculty in a graduate institute, have been ordered by MOE to reduce enrollment. This action also causes a rebound of these departments. In the issues of passing ratio and the punishment, almost all respondents do not agree to set the failure percentage in evaluation or accreditation.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Evaluation does not have to maintain the failure ratio. If the programs being evaluated always keep improving, the passing ratio will become more and more.
The question you mentioned "not pass by firm proportion" is not a problem. As for the question of penalty, this is not the penalty. For example, the school doesn’t pass the evaluation owing to not enough faculty members. What penalties do you give him? Maybe he needs to be given the fund for development. From the evaluation perspective of assisting him, I think this is not a penalty, because evaluators are not judges. (G)
The above-mentioned opinions show the significance of evaluation of finding problems and making recommendations for improvement, rather than punish the programs. Another interviewed professor also proposes that the evaluation is a recognized threshold. The result doesn’t need to make ranking table and no limit of passing. For example, the university professors can get away the self-assessment from 1 every 5 years, as long as they pass the setting step of school. The evaluation is not like quota restriction of the professor promotion.
There should be no problem on the passing ratio. We offer pass, failure and to be observed when we do evaluation. It should only be a threshold of a department or a university. If the basic requirement is not met, this is “not pass.”
If every university meets this basic standard, there is not a problem of no-pass. In the evaluation, we do not greatly emphasize the ranking table. There is no quota restriction for passing the evaluation, but the universities being evaluated must meet the basic conditions. (C)
No matter what the evaluation result is no-pass or to be observed, university
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
should not cause the fear of the department. The respondents thought that evaluation results should focus several indexes to assessment separately, rather than make the after-ranked department fail through quantitative rating system.
The so-called no-pass rate is just like some teachers in the beginning of semester to intimidate students that they want to take away one-third. I think that there should not be such a ratio. Evaluation members should look at overall development of this school, rather than insist on a threshold, followed by thirty percent of no-pass. It’s not fair for those universities who made great efforts.
Evaluation is to make sure that students receive the most basic teaching quality education. The school may be given three evaluation indicators. If it does not pass the standard of an indicator, it will be given a warning to improve in this aspect. If the other two indicators are satisfied, it is hoped that it gets better and achieves the standard settled by MOE. (F)
The interviewed experts believe that the significance of evaluation is to assist the department to improve continuously. It is the self-review system, not for comparison between schools. Even the implemented respondents also hope those evaluated units can pass evaluation. Therefore, the units being evaluated only need to prepare for information carefully and achieve self-requirement effectively. In fact, they don’t have to fear the assessment of evaluation agencies. The following listed the arguments of two respondents:
There is no need to set the ratio of no-pass. In fact, we are asking for a mechanism for self-improvement. If we cannot achieve our own goals, we have to further improve. So this is a self-challenge and does not stress that we're going to compare with others. (A)
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
I do not think that there is need to set the no-pass ratio in the evaluation.
Such a regulation must not be set, for it will be malignant. Accreditation is based on the recognition system with the results of pass, to be observed, and no-pass.
The result is determined through the evaluation report, including the scale and hierarchy. We hope that every university can pass, but the improvement is sustained, even if the university has passed the evaluation, not meaning that it is very perfect. (L)
Because the purpose of evaluation or accreditation is to complete educational goals, the results of evaluation and the set of reasonable goals of education are closely related. One interviewed university director obviously pointed out that the spirit of this outcomes-based evaluation, and explained that the evaluated units should be subject to continuous improvement. The government department with public authority was responsible for a reasonable evaluation of the threshold rather than giving the pressure of punishment to the university.
On the passing rate, in fact, there is not a fixed standard, but universities must reach their set goals. If they fail to reach the goals, how can they ask someone to give you accreditation? If MOE wants to train everyone to be president, education will not be what it is. In fact, it helps us to train management personnel at all levels and various fields. MOE and the public sector should maintain the most basic requisite for evaluated units to make efforts and succeed. Success depends on his position, and this is the so-called sufficient condition. I think that MOE should manage the necessary condition, which MOE just gets. MOE can’t manage the sufficient condition. (D)
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Interviewed experts also pointed out the importance of professional judgments of the evaluation committee members. They also accepted the various appraisal of departments. In the future, different professional evaluation agencies may carry out their missions, so evaluation committee members should not follow the same standard of assessment for different departments. The following comments echoed the importance of the characteristics of educational goals.
On the passing ratio of the evaluation, I do not agree to maintain a certain ratio of no-pass. I think it is handed over to evaluation committee members with professional judgments. The result is everything. For example, they can not confuse right and wrong. We scholars can very easily talk about SCI journals, but the other person cannot comment that he is not a good scholar only because his publication is not so good. How can apples be compared with tomatoes? They are different fruits. In the same way, we cannot treat the education evaluation in this way. Must National Chengchi University be worse than National Tsing Hua University? It depends. In the same way, must National Tsing Hua University be worse than National Taiwan University? It depends. The three universities are very good and have their own characteristics. We should not insist that what should be done. If so, it is a bad act. (K)
For government departments, although not endorsing the ratio of no-pass, the past quantitative evaluation results easily made the government sector refer to the effectiveness of education. From the following comments by the competent government, we can see that it is still difficult for the recognition system to replace the ranking system. The policy of the government subsidizing the university is in fact the major reason why the department feels the pressure.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Evaluation is never set by the rate of no-pass. Committee members can compare similar universities, so there will be high and low scores. In principle, there will be no limits of the passing rate. Unless the university is really inferior, it will be ranked third and fourth. We really need to improve ourselves if we are ranked third. The universities of science and technology are hardly ranked third.
The subsidy is given to the university which performs better. How MOE subsidies is to calculate the points based on the ranking. (N)
In the interview results, there is an opinion noteworthy. Both domestic and international evaluation and accreditation are sub-annual implementation, not the evaluation of the implementation of all departments in the same year. So it does not provide the significance of the passing rate for evaluation.
I personally participated in the domestic evaluation or that in the United States. These evaluations did not require a certain percentage of no-pass. Like the United States, they put forward to Colleges of Medicine more than 100 standards. The committee members examined each standard separately. Not passing the standard of logic, this university will be likely assessed “to be observed” or “a warning.” If there are some special financial problems or major losses in Colleges of Medicine, the university may have to face the accreditation result of non pass. We do not say that the whole nation receive the evaluation in the same year. Maybe 10 medical colleges attended the accreditation in one year.
There is certainly no percentage of no-pass. (J)
According to scholars and experts, accreditation agencies just gave professional judgments. The penalty was just what the government should consider. Respondents
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
departments did not pass the evaluation. If the departments were voluntary to participate in evaluation or accreditation, the government policy could function well.
Department evaluation did not have the fixed passing ratio. MOE instead of HEEACT punished the programs of no-pass. If a university program in the United States is not accredited, it can’t get the federal government grants and students can’t apply for student loans. Taiwan has no such provisions. Student loans will not be cancelled, but departments at most cannot expand the enrollment or decrease the recruitment. Our conditions are different from those of foreign universities. (B)
Of course there are interviewees who thought that there should be punishment according to evaluation results. Unqualified programs should be closed. It is the national public authority, rather than the accreditation body, should give punishments to unqualified departments.
Educational administration within MOE should play his good part to close those unqualified departments rather than commit HEEACT to evaluate. Public interest groups of U.S. represent the State to carry out part of public authority.
The division which Taiwan currently has this power is Department of Higher Education of MOE, not HEEACT. If Department of Higher Education of MOE does not carry out his mission, we can’t do anything to impose sanction against the unqualified programs. (E)
The interviewed policy maker agreed to maintain the no-pass ratio of the accreditation. The interviewee thought that the deletion of the school endowment fund from MOE grant as a method of punishment may prompt the school principal to pay
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
attention to the quality control of the programs. The following comments are very different to those of other respondents. These also show that the lawmakers’ different opinions of the punishments for the higher education accreditation.
Evaluation of course had to maintain a certain ratio of no-pass. To be observed did not make sense but just give a warning. But “to be observed” had to be reexamined. Currently there was no failure after reexamination. I thought the reduction of the student admission was the direction of a set penalty. It might not be a good way to decrease the recruitment. There was of course no development power if number of students decreased. Reducing subsidies might be a way, for reducing the endowment fund for school development made them feel more pain.
To some extent the president might start to be more responsible, and had a sense of crisis to defend the evaluation of each program. It is also a good way, in my opinion, instead of the present act where evaluation is a matter of each department. As the accreditation results are involved with the school fund, there cannot be irresponsible for the president. (O)
Most of the respondents believed that the purposes of higher education evaluation included the self-review, self-evaluation and continuous improvement.
Evaluation results do not need to set the passing rate. Accreditation body played a role of mentoring or reviewing. The interview results showed different views. A scholar thought that an academic evaluation agency, such as IEET, was a counselor, and showed that they met international standards and implemented the task of counseling.
Basically, the foreign professional evaluation institutes are like this and should be a counselor. There is no penalty. Consider IEET, who follows the
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
rate is too high. Now we, HEEACT, come to reflect that in fact, IEET is right, because IEET follows foreign professional development model. Nowadays, universities voluntarily improve and receive the counseling. Finally, it is normal for them to pass. On the contrary, universities which did not pass the evaluation may have to make more efforts. (G)
Another interviewee also thought that the professional evaluation institution played the role of a counselor proposing the suggestion to the evaluated unit. This relationship is similar to the relationship between patients and physicians. Patients improve their health by means of physicians’ diagnosis of disease.
Of course, assessment body played the role of the counselors, who were not the police. Educational evaluation committee members served as consultants.
Just as we went to the doctor, if a physician were like a policeman, punish me again after the interrogation enquiry. I would try my best to hide my illness for fear that I would be punished. But if the patient did not tell the doctor where he was hurt after seeing the doctor, then he just stayed at home, lying and seeing no doctor. Like the physical checkup, evaluation most importantly helped evaluation members improve themselves. If they hide the problems, the evaluation committee members could not give any advices at all. (I)
But another professor with different views exemplified the U.S. accreditation bodies and thought that they were evaluators. In need of other experts’ counseling, the university had to pay additional fees.
Accreditation body played the role of evaluation. Counseling is not what HEEACT did but should be done by MOE. HEEACT was of course responsible
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
for only evaluation. How to ensure the quality of the evaluated unit should also be urged by MOE instead of HEEACT. Like the accreditations in the United States, they would not be counseling but just telling universities problems. If universities need professional help, they had to pay extra money and request that other professionals offer help. (C)
Another evaluation agency respondent believed that the evaluation agency was evaluating and strictly carrying out tasks, as a teacher was a counselor in class and an examiner in giving students exams.
The evaluation agencies of the United States have two roles. Before they evaluated, their secretary-general could go to universities to give them guidance and advice. Actually paying a visit, that person could not go. As for the penalty problem, HEEACT was not the unit who superintended university rights, but MOE had the right of giving punishments. HEEACT and foreign evaluation units could only give oral warnings, hoping that the program could improve. (J)
A university director participated in the AACSB accreditation gave foreign accreditation units as examples, and explained that they played roles of both counseling and reviewing.
Evaluation agencies played two roles of the counselor and evaluator.
AACSB, for example, did not require the universities to meet the standards of the evaluation committee members but their own standards. The evaluation committee members came to see your achievements. The goals of universities were legitimate and reachable. For example, they could prove what they could achieve or would strive to achieve in a few years. In this condition, the program
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
passes the AACSB accreditation. It did not coerce universities into becoming like Harvard University. If universities ran like Harvard University around the world, which units cared small and medium-sized enterprises? (D)
From the point of view of the domestic academic programs, the interviewed professors agreed that the role of counselor was to assist the department to identify problems, counseling the latter to improve the weakness rather than hope that evaluation agencies would bring fears to subjects. The following listed the views of the two interviewed professors.
The entire accreditation process was a consulting system, because the agency had to do according to the standard operating procedure (SOP). Just as we are applying for ISO9000, basically the whole process was consultation. It is the same of the accreditation process. The whole process is consultation, which was that universities created a mechanism for self-improvement. (A)
I thought that it should be better for the evaluation institutions to be counselors, because the former evaluation (possibly like the teacher or a judge) gave us the feelings of scary and fear. So I thought that it would be better to be a counselor. Consultants should help universities establish a mechanism for self-improvement instead of mentioning that they determined the penalty. These two roles brought quite different feelings to the programs being evaluated. (F)
There were different views of the respondents who received the interviews of the domestic evaluation agency. Some people thought that evaluation agency is a counseling one. Other people thought that the two roles should not be confused but be
There were different views of the respondents who received the interviews of the domestic evaluation agency. Some people thought that evaluation agency is a counseling one. Other people thought that the two roles should not be confused but be