Chapter 6 The Interview Analysis and Discussion (C): Global
6.5 The analysis and discussion of other interview opinions
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
6.5 The analysis and discussion of other interview opinions
In addition to the discussion of the six interview questions, this study also ask scholars and experts the relevant and extended questions and obtain many valuable opinions. This section focuses on those relevant issues and the analyses of them. First of all, we discuss whether it is too much load for accredited departments due to higher education accreditation or evaluation promoted by the government. Currently, the accreditation or evaluation does not involve in the entire faculty members of the department. Instead, it involves only minority of teaching faculty and is lead by the department head. Thus, most faculty members do not understand the meaning of teaching assessment and feel bothered by busy filling out the evaluation documents.
Department heads worried about the accreditation results, thus, they ponder around about the best way of reporting the information as well as inviting alumni and enterprises representatives to attend the meetings. Therefore, the administrative work really brings lots of pressure. The results of the interview show that some scholars and experts think accreditation need the investment of resources. The purpose is to understand problem and self-improvement, thus, it should be an ordinary work and release the unnecessary load.
Many resources are required for accreditation, and you always think what the results will be? Therefore, it must be moderate. Accreditation should be performed regularly to lessen the unnecessary load. Accreditation should provide information and allow those being evaluated to know how to improve.
Accreditation is like a health examination to find out the problems for improvement for our own benefits. Do we get recovered naturally after getting the health examination? Accreditation itself will not bring the improvement, it is
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
impossible. Department being evaluated will need to follow up some actions to improve it. This requires lots of resources. (G)
A senior educators interviewed in this study also agrees that accreditation is like regular health examination, which does not cause any trouble. Enterprises regularly examine their performance for improvement, so the universities should certainly conduct similar work.
In fact, accreditation is not trying to trouble anyone. Rather, it attempts to filter out the bad ones. This is a good opportunity to examine oneself, and private company does this often. Why university doesn’t do it? We think accreditation is a regular health examination. If the accreditation result is not good but one does not try to improve it, we may request the universities or programs to reduce the classes. (C)
Of course, there are some opposed opinions that interviewees think accreditation indeed gives load to departments. This study interviewed one department head, who brought up the issue of pressure that caused by many practical operation perspectives.
The interviewee thinks department is busy meeting accreditation requirements.
Accreditation organizations should initiate to inform the required information.
I think evaluation will bring load to departments. Every university does many improvements for evaluation, and many measures are taken for the purpose of evaluation. When collecting information, if you can tell us beforehand, it will be less burdened for us to prepare. However, if you request suddenly, we need to date back to two or three years and search for the information. If we can’t find it, it will be very troublesome. I mean evaluation is better not to submit the sudden
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
request. For example, if they say from a certain year, we need to commence the accreditation work. It will make everyone panic. It would be better to give us some buffer time. I personally think the department will be influenced by evaluation to meet the evaluators’ requirement and devotes many efforts in some aspects. It may seem like the universities are led and constrained by the evaluation. (F)
The accreditation agencies understands that department heads and departments being evaluated have been working hard. The opinions of interviewed experts are listed here.
Taking part in accreditation will certainly bring extra load. In fact, I often feel that department heads are very hard working as well as the departments being evaluated. They need to collect a lot of information as well as filling out and organizing the documents. Those burdens fall on the shoulders of different faculties. But I believe due to this organizing work, they self-examine the department itself and finds out the problems for integration. Although it is a hard work, the outcomes are helpful for universities and departments. It is more appropriate to define accreditation as self health examination. (K)
Another critique on higher education evaluation is that it should not only concern about the number of articles that teachers published in SCI or SSCI journals.
Interviewed scholars and experts think the purpose of accreditation is to improve teaching quality and curriculum design. The large amount of paperwork needed filling out for accreditation is the burden to the department being evaluated. The operation method should be improved. The followings are the opinions of two scholars or experts.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
I think faculty in the university need to balance the work of teaching, research and service. A university professor needs to do research, but his research performance should not be measured by SSCI publications. SSCI is a small, profit-oriented, and money-making company in America. Their collection of journals and periodicals is very limited. For example, there are thousands of law-related journals in America and SSCI only collects over a hundred of them.
(E)
Some educators think the teacher’s performance should not be included in evaluation, and it should be included in the students’ performance. The better the student perform, the better the teachers’ performance it means. No matter how many more TSSCI articles the teachers write, their students are not guaranteed to find jobs. We suggest that the Ministry of Education should concern more about curriculum design and the preparation for students’ core competencies in the evaluation system. An evaluator once mentioned that evaluation should include students’ employment rate. In theory, we hope “the right man in right place”.
What you learn is where you should go for development. Too much paper work is heavy load for university. I know they spend nearly half a year busy with evaluation work and wait nervously for the evaluation results. Evaluators may write that many students saying that English class hours are not enough when filling out the interview opinions, which may over expand the opinions of minority. I think this system has room for improvement. (O)
The performance of evaluation work relies on the experts’ judgment. The employment and training of evaluators are questions worthy of consideration. In this issue, all interviewed scholars and experts reach consensus that evaluators need to be
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
trained. The professional accreditation institute must be careful and consider the professionalism of evaluators when appointing evaluators. The followings are the opinions of interviewed scholars.
Having the appropriate evaluators can then understand and help universities rather than doing evaluation with far distance. Evaluation agencies are at the equal position of universities. You are assisting them but not to stand above them. Evaluators are the same as university professors, aren’t they? I, myself, am an evaluator and I also work in the university. There is no point to say that because you evaluate them and you are superior to them. Such views should be given at the training program. We should be trained, we then have objective evaluation. The promotion of evaluators’ qualities is through professional training. (G)
HEEACT indeed concerns about the training of evaluators. One interviewed expert mentioned that accreditation is a qualitative work. The worry is the inconsistent criteria among different evaluators, which may cause the dispute of evaluation results. Therefore, it needs the training workshop for evaluators to form the mutual viewpoint among all evaluators.
We, HEEACT, start promoting the training program for evaluators. For example, writing the report and evaluation ethics. The purpose is to set up a standardized procedure for evaluation, for fairness. Evaluators should only evaluate the same field as his/her profession. For instance, it is meaningless if he specializes in art but evaluates the business school. Why people have many puzzles about accreditation? In fact, it is a qualitative performance but you want to quantify it. Everyone has one’s perception of something, and his professional
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
is built differently. What we worried is perhaps it is a good thing, but because of your high standard, it becomes bad. It may require more observation or evaluation, but you are too generous and let it pass. Therefore, we can make the potential extreme opinions to the middle through the evaluator workshops and enable everyone to reach a consensus. (L)
The interviewed scholars or experts who have been invited in international accreditation agencies said the employment and training of evaluators abroad is very strict. Evaluators need to go through the stage of observer before becoming the official evaluators. Currently, there are too many evaluators in Taiwan and the interview results obtained can act as the reference for professional agencies to train evaluators. The followings are the opinions of three interviewed scholars.
The training of evaluators is very important. I was invited by American accreditation unit last year. I attended the training workshop in May, then being the observer in the accreditation committee in November. After that, I became the evaluator. American evaluators themselves are also trained this way. (J)
Evaluators need experiences. AACSB now only invites the present or former deans as mentors. That is the first stage. Europe emphasizes on the cooperation of industry and academy. It employs industrial experts, such as former executives, general managers or presidents. The second stage is the review team. Review means the visiting of the universities and examining the paperwork, and see whether they perform well or whether they meet the requirement as well as their goal of strategies. (D)
The key of evaluation relies on the quality of evaluators and the training
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
process. Accreditation is a profession and evaluators need to be trained. We know that currently, Hong Kong requires their evaluators to take twelve courses, both core courses and electives. After completing the courses, they will be official evaluators. Of course, they are orientation meeting for evaluation. Every evaluation has different rules and procedures. There is a need for relevant orientation. (I)
The departments being evaluated have some complaints about evaluators in Taiwan, and they think the current training for evaluators need to be strengthened.
Professional agencies need to follow the procedure of selecting evaluators. The critiques of some interviewed scholars in this study also present the opinions of department being evaluated.
Higher education evaluators must be very strict. They may not need training because the evaluators in Nobel Prize are not trained either. Those evaluators have their long-term status in academic field as an outstanding scholar. They are not strangers to the matter of evaluation because they concern about education as well as concerning about academy. I think these are requirements. We provide some training to evaluators, but one hour or one morning does not make any differences. I think they appoint the higher education evaluators without such consideration of requirements. You can write this down, it’s okay. (E)
At present, the Ministry of Education starts to recognize more domestic and international professional accreditation or evaluation organizations. IEET and Taiwan Assessment and Evaluation Association have already been approved. In the future, it is expected that more organization will submit their applications. Should government set up guidelines for professional accreditation association? What is the performance
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
effectiveness of professional accreditation association? The interview of this study includes the opinions of experts and scholars. The following is the viewpoint of a senior scholar. He is in favor of that government setting up the criteria and allowing professional accreditation association to run its function, so as to self improve under governmental supervision.
I think there is a need to set up a set of criteria in terms of the development of evaluation associations in Taiwan. The criteria are used to examine whether the institutions that take part in the evaluation system include the consideration of fairness, objectiveness, transparency, and professional. If every organization can develop in this way, their professional field evaluation will be worthy of encouragement because they will demonstrate great effectiveness. It is hoped that the criteria may enable professional evaluation associations obtain recognition and allow them to establish the self improvement mechanism. (G)
Some scholars propose different viewpoints. They think that professional evaluation associations abroad are independently operated, they will not adopt the regulations set by our government. If HEEACT do not have governmental support, it cannot operate continuously. The viewpoints are indeed the challenges encountered by the Ministry of education when approves foreign associations. The following is the opinion of an interviewed scholar.
Many accreditation or evaluation group abroad have their different requirement. I think they may not adopt the guidelines that you established.
Currently, we have only HEEACT, and of course, we hope to see the emergence of another organization. However, Taiwan market is very small, and other
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
commissioned by the Ministry of Education, it may not exist now because it is very costly to operate such an organization. (C)
The Ministry of Education once supported the establishment of IEET, and for becoming the professional accreditation agency that gears into international connections. However, the Ministry of Education cannot possibly make the budget to establish professional accreditation association for each academic field. One interviewed officials in this study indicates that professional accreditation association must be financially independent. They can then perform the task fairly. The opinion of this official is shown below.
Should the Ministry of Education support the establishment of professional accreditation group? IEET has clear linkage to engineering education, thus, it was being supported. Regarding to other fields, such as Architecture Association, Chinese Chemical Society, and Chinese Physical Society, if they conduct their own accreditation or evaluation process, they can charge the accreditation fee and they need to be responsible for their own expenses of survival. The concept of self-financing should be emphasized. Otherwise, government cannot afford to support each of them. If the association meets our funding criteria or regulations for gearing into international conferences or organizations, we certainly consider offering funding. However, the funding percentage is very low. The accreditation association should be financially independent in addition to receiving governmental funding grants. They can then have positive attitude of fairness and objectiveness. Otherwise, they will easily be led by governmental guidelines. (M)
An interviewed senior executive as well as senior professor also thinks higher accreditation in the future is an open market and allows the fair competition of
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
professional accreditation association both local and international. University departments can freely choose which one to take part in.
Taiwan has developed its accreditation agencies, such as IEET in Engineering, and Medical Evaluation Board in Medical field. The question is they cannot establish for the purpose of protecting themselves. Instead, they need to open and receive inspection by others. For instance, the judges of U.S. Federal Courts are required to be questioned in the Congress, and they are willing to receive the external inspection. On the contrary, our system is nearly a close one and we do not receive external inspection. Open means the permit of competition, and then obtain the external recognition. (D)
Another executive expert of the Ministry of Education indicates in the interview that government will approve the domestic professional evaluation agencies that have public credibility. They can take initial move of recognizing international organization. The interview result is given below.
Professional evaluation agency must have a set of procedures and compositions. We need to inspect your outcomes since the establishment, whether your process is strict and has public credibility. Domestic professional evaluation agency can apply for the approval by the Ministry of Education while we take initiative move to recognize international associations. International recognition is not limited in certain areas. As long as it is international, we will recognize it.
Those professional agencies include IEET in Taiwan, and AACSB in American.
We hope HEEACT in Taiwan can reach to the same ranking as CHEA in American. (N)
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Because globalization is developed rapidly, international advanced countries have great ambitions in the education market in emerging countries in Asia and South America. Following the law, our government recognizes domestic and international professional accreditation and evaluation agencies. Although it does not protect domestic association, it cannot loosen the requirement of international association.
Domestic professors being interviewed in this study provides the following opinion over the issue that the Ministry of Education initiates the move of recognizing international association.
In the future, government will need to conduct recognition process for various and emerging domestic and international professional accreditation and evaluation associations. Therefore, government must establish a clear set of criteria. However, government cannot take strict criteria to treat domestic associations but loose requirements for international agencies. The recognition criteria must be the same. That is, when government makes guidelines, it requires global views. Government will need to face the market of international professional accreditation and evaluation associations in the future. (H)
One elected official interviewed in this study has some questions about the effectiveness of current higher education evaluation. The official thinks that the Ministry of Education need to supervise the professional evaluation agency more specifically. Moreover, the Ministry of Education not only can reach the aim of improving educational quality through higher education evaluation, but can adopt alternative funding grant project to enable university to improve. The interview content is given here.
Every of the meetings before commencing evaluation is called by the official
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
of the Ministry of Education. The Minister and Secretary of the Ministry of Education are there. They will brief with the evaluators about the main direction and things that need to avoid. Even if they do so, there are still many controversies. It means that the mechanism needs improvement. Currently, the Ministry of Education supervises HEEACT in form but not in practical. How to avoid the over large power of evaluators is also what we need to consider. In
of the Ministry of Education. The Minister and Secretary of the Ministry of Education are there. They will brief with the evaluators about the main direction and things that need to avoid. Even if they do so, there are still many controversies. It means that the mechanism needs improvement. Currently, the Ministry of Education supervises HEEACT in form but not in practical. How to avoid the over large power of evaluators is also what we need to consider. In