Chapter 3 Design of This Research
3.1 Structure of this research
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chapter 3 Design of This Research
3.1 Structure of this research
3.1-1 Investigation through interview
There are six models of evaluation as described by Dr. Pan（潘慧玲，2002). The first one is the objected-directed evaluation that emphasizes the formulation of objectives and the extent of achievement. The applicant should define the targets or objectives, find the scenarios to display the targets, collect data to compare whether the targets have been fulfilled. The second is the CIPP (context, input, process, and product) model. This is the management-oriented model that provides necessary information for the decision makers. The third is the consumer-oriented product evaluation. The purpose of the product evaluation is to supply information for consumers in selecting desired products. The fourth one is the defense-oriented evaluation. The evaluators present opinions to support and against a specific subject.
The evaluation is a debate process and the final result is determined by an arbiter. The fifth type is the responsive or naturalistic evaluation. This model considers the participation of the applicants and judges the evaluation scenario as a whole.
The most important model in this research is the professional comments oriented evaluation. The accreditation system is the most common one belonging to this model.
Kells (1983, also translated by 王保進，2002) defined this model as: It is a voluntary process operated by a non-governmental organization. It applies the peer evaluation to examine the self-study report, and determine if the applicant has matched his objective and the standards of evaluation. The non-governmental organization has to
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
announce the evaluation standards or criteria. The applicants submit their self evaluation reports to the evaluators. There is on-site evaluation process and the final result is decided by a recognized meeting. The CHEA (Council on Higher Education Accreditation) is executing the accreditation in USA. The HEEACT in Taiwan is working on the similar mission under the commission of Ministry of Education. The Institute of Engineering Education Taiwan (IEET), a non-governmental organization, is conducting the accreditation of engineering programs in Taiwan. The accreditation conducted by IEET belongs to the type of professional comments oriented evaluation.
The essential steps described by Kells have all been included in the process of IEET.
It is also known that programs accredited by IEET can be waived from the evaluation by HEEACT.
The structure of this research intends to interview scholars and professionals about the most appropriate evaluation model in Taiwan. The Deputy Minister of Education （呂木琳次長）believed that higher education evaluation should be combined with the enrollment and recession processes (陳曼玲，2007). On the other hand, some scholars concluded that higher education should not be tightened with financial support from the government (戴曉霞，2004). Research is part of the function of university. There was argument (嚴昆陽，2004), however, quantitative numbers of publications and research budgets would be ridiculous for higher education evaluation. The accreditation model has been applied in United States for more than one hundred years (戴曉霞，2004). It might also be used in Taiwan for evaluations of various academic fields. There are three kinds of evaluation presented by Dr. Stella who is the chairman of the Australia Universities Quality Agency, AUQA (邱于貞，2009). They are the popularity approach, quality assurance approach, and responsible approach. The first one is based on the ranking of institutions which
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
is generally believed to be lack of confidence. The second one is based on the measurements of the outcomes of students and graduate attributes. The third one opens the learning achievements of students to the public. Is there one single model or various models that are suitable for the higher education in Taiwan? This question has the first priority to be clarified in this study. Through the interviews of this research, the advantages and drawbacks of the present higher education quality control systems will be summarized and analyzed.
3.1-2 Glossary of items for the evaluation/accreditation
In order to fulfill the successful interview process in this research, some important terminologies have to be identified. A glossary of items related to evaluation or accreditation is listed below that will be used in the interview and analysis of interview results.
Outcomes based evaluation（成果導向之評鑑）: Outcomes based evaluation examines the impacts, benefits, and changes of a program after its participation into the evaluation. Those changes are usually expressed by the knowledge, skill, and behavior of graduates.
Assessment（評量）: It is a process of documentation, usually carried out in measurable terms. Typical assessment methods include standardized measurements (in-class examinations), survey, focus group, and portfolio.
Accreditation（認證）: It is a voluntary, non-governmental, peer-review process to reach high standard declared by a commission.
Criteria of evaluation（評鑑規範）: They are the quality standards set by
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
professionals and technical societies. The evaluation/accreditation process is based on those standards.
Graduate attributes （畢業生核心能力）: These are the skills and understandings that the students should develop upon graduation for the social goods in unknown future. The graduate attributes are also known as core competencies.
Self-evaluation report（自評報告）: This report, prepared by the educational program, describes its educational objectives, practices, procedures and outcomes.
Continuous improvement（持續改善）: It is a process of data collection and interpretation of the facts in order to achieve the outcomes continuously.
Professional mobility （ 專 業 人 員 流 動 ） : This is a mission to facilitate professionals to across international borders through mutual recognition of credentials.
Substantial equivalence（實質相當）: It means that the education programs have comparable contents and experiences. It implies that all graduates have reasonable confidence as entry level professionals.
Program evaluators （ 評 審 委 員 ） : The evaluators are well qualified and committed professionals to conduct evaluation or accreditation and to give balanced professional judgments.