Chapter 4 The Interview Analysis and Discussion (A): Outcome-based
4.1 The suitability of the outcome-based evaluation or accreditation
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chapter 4 The Interview Analysis and Discussion (A):
Outcome-based Approach and the Role of HEEACT
This research adopted the in-depth research method and interviewed the domestic fifteen professors, research experts, and government officials who were very familiar with higher education evaluation or accreditation. The purpose of this study was aimed to improve our higher education quality control, to analyze and discuss, and to compare with literature. The analyses and discussions of the interviews of this study are divided into 3 chapters (Chapters 4 to 6) on three major topics. Chapter 4 focuses on the outcomes-based approach and the role of HEEACT. The interview results on the issues of passing ratio and exercises of civil rights are discussed in Chapter 5. Chapter 6 emphasizes on other topics of global connection and license examination. The Chinese version of the interview results and discussions are listed in Appendix C.
4.1 The suitability of the outcome-based evaluation or accreditation
Taiwanese former higher education evaluation adopted the input-basis, such as teacher-student ratio. The recent international higher education evaluation has suggested outcome-based approaches. IEET’s work follows the outcomes-based spirit.
HEEACT has also adjusted itself to the outcomes-basis. The outcomes-based evaluation or accreditation examined whether education was successful. Different from the traditional input-based evaluation, was the outcome-based approach a better one? This was the first problem of consulting the professors and the experts in the in-depth interview research. From the interview, the present domestic and international institutions participated in the outcomes-based accreditation deeply
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
agree on this method. One professor and high-ranking director who participated in the AACSB accreditation definitely agreed with the spirit of accreditation. He thought that if the input did not produce output products, it is a waste of the educational resources. However, he also pointed out that the outcomes-based accreditation needed the clear definition for the contents to be examined.
It was necessary for the state policy to upgrade the higher education and the international competitiveness. Running a school was to nurture personnel and ultimately to show the achievement. The outcomes-basis was completely right. If the “input” did not become the “output,” the former was a waste. For the foreign AACSB accreditation, we emphasized KSA: knowledge, skill, and attitude. Now we teach more knowledge in universities than the skill and the attitude. The ability and the attitude were indispensable to the outcome basis. More difficult is that “outcome based” is not easy to define, for one has to know which profession he suits in advance (D).
The other two professors who joined the internal evaluation or accreditation also point out the outcomes-based evaluation or accreditation, including the teaching assessment and prolonged, improved mechanism helped improve the insufficiency of the input-based approach. Therefore, the outcomes-based approach was thought to be the superior quality control guarantee.
The outcomes-based educational accreditation was mainly to finish the course design, the teaching assessment, the loop mechanism, and to make every department exert their features. This kind of accreditation or the evaluation was certainly superior to the former input-based approach, because the
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
and makes sure of the best method of the quality control with the continuous examinations according to the trend (H).
Now or in the future, the evaluation (will) emphasize the learning result. For example, you know how students (will) learn and how to grasp his learning result.
When students face the difficulties or setbacks, will you have some counseling mechanism to grasp the whole learning effect which fits your standard (I).
An interviewed policy leader can’t agree on the outcomes-based evaluation approach more, and thinks that such an evaluation result can ensure that students have enough core abilities of employment and can have the ability of working abroad.
The idea “outcomes-based” is certainly right. We have to evaluate students according to their applicable abilities. If students do not know what to do, have their poor thinking ability, and write the composition and the Chinese character poorly, they are of little or no use to the personnel competitiveness of the country.
So why to evaluate now is obviously right. I support this point highly. Our education should develop students’ core abilities. For example, they can find jobs, want to go abroad for further studies or work abroad. Otherwise, higher education is a waste of time, the resource, and the taxpayer’s money (O).
In this interview, about one-fourth interviewees agreed that the outcomes-based evaluation was a better way. Nonetheless, they also thought that it could not replace the input-based approach or that time was insufficient from the traditional way to the outcomes-based approach. There are many assessment methods to establish. The following are to list their views.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
I think that the outcomes-based way is better, but we can’t wholly use this approach. In the past, the “input base” was in fact not very good because it became the formula. Universities could not develop their features. I think that adding the “output,” all the educational quality can upgrade. Instead, we should not comment teaching only out-of-date KSA (knowledge, skill, and attitude), disconnected to the theory and the reality. In addition, the outcome assessment was indispensable. In this way, universities can adjust courses with the time progress. Consider the vocational school students who will join the work market after graduation, the “output” is in fact suitable.(N)
I think that the input basis and the outcomes basis are different aspects. The outcome basis is aimed to examine the function, the effect of teaching and the educational quality. Concerning these, the achievement plays a crucial role. For example, how did education persuade everyone that these resources I used indeed helped me get these output. We did not emphasize these before. However, now especially for the age in which universities become very competitive, we have to see if these students can become part of useful citizens in the future. Accordingly, under this circumstance, outcome basis makes sense.(L)
We find that no matter what they are evaluation or accreditation, they have become outcomes-based gradually. Outcomes basis is to assess universities through multi-evaluation, such as students’ performance of “core competency,”
graduates’ employment, and so on. “Outcomes basis” assesses four-year educational process and students’ progresses—students’ learning effect performance. Basically, I think that this approach is good. Possibly because people were used to the input-based approach, therefore, in the transformation,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
the multi-evaluation, including the track of the alumni and their later developments, also becomes our future concern.(F)
About one-fourth interviewees did not favor the outcome-based evaluation.
Several interviewed educational scholars found it hard to control direct information needed to be examined in the outcomes based approach.
In fact, up to now, everyone only “attempts” to see students’ outcomes as one key point and the collected students’ learning results are very limited, for there is no national training for assessment. Our so-called outcomes basis is to understand that upon graduating, students learned related knowledge, skill, and attitude. These actual results are very important. However, according to the present situation, we cannot fully grasp these “direct” results, but the “indirect”
results, that is, his career development of the employment and the graduate school after graduation (G).
Interviewees, experts and scholars also mentioned that “outcomes basis” was an approach of evaluation, but at present, lacking the review and the improved mechanism was a fly in the ointment.
We cannot comment “outcomes basis.” In fact, it is only part of the evaluation. Our present so-called educational evaluation should include formative evaluation and summative evaluation. Accordingly, outcome basis is originally part of it. However, HEEACT only conducts more “outcomes basis”
than formative evaluation. Formative evaluation is mostly applied to “self.” For example, in an organization, we keep on examining ourselves in the meetings.
Another example was that the outward scholars and experts come to examine our
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
system. These are formative evaluation. Summative evaluation is the present adopted way for our higher education most of the time, that is the outcome basis.
Consequently, we cannot comment it but express that in the future, adding formative evaluation, it will become more perfect (C).
Professors who received engineering education certificates also point out that not wholly the same, accreditations put emphasis on outcomes basis and included SOP (Standard Operation Procedure). But now many education units and their members cannot understand the differences between evaluation and accreditation. It is not totally clear which way is better.
There are some differences between evaluation and accreditation, including the basic spirit and start point. Some people can’t make accreditation and evaluation clear at all. The key point for both of them is to improve the teaching environment. I cannot answer which is better.(A)
Of course, senior interviewees of the scholars also think that input-based achievement needs more examination. They can’t completely decide the evaluation results according to outcome basis. A high-ranking official who actually conducted the evaluation also thinks that outcome basis needs time to make the public understand and that at present the effect can’t be shown.
Outcome-based evaluation will be adopted in the next round evaluation. The future evaluation is mainly based on students’ competency—the present core competency indicator for every department. I think that the evaluation may not emphasize outcomes basis so much. However, the outcomes will also be examined. The evaluation also needs to examine students’ learning. Input-based
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
evaluation includes student-teacher ratio, the budget, and the library. However, the key point is in the outcomes of the students.(B)
It is very hard for you to evaluate according to outcomes basis. At first, committee members are not very easy to find. Industry committee members are not easy to arrange in pairs or groups. Everyone has his/her opinions and has difficulty in reaching an agreement. In 2004, the university evaluation started and up to now, I have thought that the outcome is very good. Of course, outcomes basis is a tendency and needs several years to make the public digest. At present, the effect cannot be shown. (K)
On the basis of the above-mentioned interview, more than half of the interviewees agreed on outcomes-based evaluation or accreditation. However, this new tendency cannot be familiar to the public yet. Quantity index in the traditional input-based evaluation still needs to be considered by evaluators and the people being evaluated. It needs time to fulfill outcomes-based evaluation or accreditation. Now the professional departments of engineering technology, the medicine, and MBA adopted outcomes-based evaluation or accreditation. Would other departments adopt this kind of evaluation later on? The following listed the interviewees’ views. Concerning medical students who needed the professional accreditation, the outcome-based approach is highly recognized.
I agreed that the outcomes-based evaluation was a better way and more suitable for our medical field. We would have the definite core competency when we graduated. If colleges of Engineering and Science could list their standard, it would be very good. I did not know whether colleges of Liberal Arts and Law could lay down core competency very definitely. Consider our College of
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Medicine, we can estimate what special knowledge students must have, their care of the sick according to their clinic ability, their communication with the sick, their ways of sympathy, ethics, and moral. Without satisfying these standards, students could not graduate. If other departments also could do so, outcomes-based evaluation could be adopted. (J)
The interviewees of the scholars and experts thought that outcomes-based evaluation or accreditation was applicable to all departments which had to draw their individual evaluation index. Every department should not adopt the same standard to evaluate. Engineering technology could draw evaluation index. The evaluation indexes of the departments of Liberal Arts and Law were obviously different from those of department of Engineering. At present time, formulation of these indicators is undergoing. The following lists comments of this question.
Different departments can have the outcomes-based evaluation which is more objective. Such an act is very few. Now everyone speak of many outcomes-based issues but do not develop this system. I think that it will be slowly developed. (G)
Do we really shoulder the responsibility of measuring the achievement duty and formulate some most suitable indexes for every field? We can’t treat medicine in ways of Colleges of Liberal Arts and History. Neither can we evaluate art in terms of College of Medicine. Concerning the time and the process of developing professionals, every academic field is different. (L)
The traditional input-based approach includes the same quantified standard for different departments. The scholars and experts expect various programs to establish
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
their features in every field, examine the mechanism to compare themselves through the outcomes-based accreditation. The traditional evaluation includes the scores and ranking. Outcomes-based evaluation, however, adopted the recognition system.
Ranking and recognition are very different. More than half of the interviewees think that the public do not understand outcomes-based approach yet. Therefore, if outcome-based evaluation and recognition are promoted to every field, there still needs time. The following list the interviewees’ opinions.
Outcomes-based evaluation is applicable to every department or program, depending on their achievement performance and competency indexes for themselves. For instance, their competency index decides their musical hosting.
After you draw up a core competency, you have to have an assessment mechanism. After formulating the assessment mechanism, you will not only see the academic performance as the only standard. Speaking of art, you may refer to the performance. (B)
No matter what they are evaluation or accreditation, you had better not use the same standard to measure everyone. Therefore, different standards instead of one standard are used to measure Colleges of Liberal Art, Law, Science, and Engineering. At present, it is hoped that everyone can compare with themselves.
Then they evaluate themselves according to their own. It is also hoped toward such a direction, but some difficulty still exists. For instance, Taiwan Assessment and Evaluation Association evaluated the universities of science and technology using the ranking system. The ranking system and the recognition system are different. (K)
‧
國立 政 治 大 學