Creating the aspired intelligent assessment systems for teaching materials

(1)

Creating the aspired intelligent assessment systems for teaching materials

Cheng-Hsiung Chen

a

, Gwo-Hshiung Tzeng

b,c,⇑

a

Department of Information Management, Kainan University, No. 1, Kainan Road, Shinshing Tsuen, Luchu Shiang, Taoyuan 338, Taiwan b

Distinguished Chair Professor, Kainan University, No. 1, Kainan Road, Shinshing Tsuen, Luchu Shiang, Taoyuan 338, Taiwan c

Institute of Management of Technology, National Chiao-Tung University, 1001, Ta-Hsueh Road, Hsin-Chu 300, Taiwan

a r t i c l e

i n f o

Keywords:

Intelligent assessment system Teaching materials

MCDM (Multiple Criteria Decision Making) DEMATEL

ANP (Analysis Network Process) VIKOR

Core competence (CC)

a b s t r a c t

The core objective of the Nine-Year Integrated Curriculum for primary schools is to ‘‘enable students to demonstrate their talents instead of just scoring high on exams’’ around the world. The determining fac-tor in education reform is to declare the competence indicafac-tors of necessary educational behavior in pri-mary and junior high school. In the reform process, for all domains, the enriched rate of competence indicators for educational materials and methods is very meaningful. Because educational materials and methods in different domains have their own style, we should evaluate these teaching materials sep-arately. Thus, in this research we propose a novel MCDM (Multiple Criteria Decision Making) framework for evaluating, comparing, and improving the effectiveness of competence indicators in the various pub-lications for teaching materials in primary school based on different viewpoints. The ANP (Analysis Net-work Process) weights are based on the DEMATEL technique with the MCDM method for resolving the problems of dependence and feedback among criteria. Then, a VIKOR technique with ANP weights is proposed for addressing and reducing the performance gaps for each criterion, thus hopefully improving, re-conﬁguring and selecting the aspired Intelligent Assessment Systems (IAS) for teaching materials. An empirical study of Mandarin Chinese based on this system design of three publishers is illustrated to verify the effectiveness of the proposed method. The results can improve the efﬁciency and quality of the authored Mandarin Chinese teaching materials and may extend to other Learning Areas.

1. Introduction

Every country has made the cultivation of human talent a priority in the 21st century. As other advanced countries propose education reform in different forms throughout the world, Taiwan also recognizes education as the bedrock of national development, implementing various education reforms such as pre-school edu-cation reform, grade 1–9 curriculum reform, the restructuring of secondary education, the enhancement of higher education, and lifelong learning projects (MOE, 2008). Therefore, the purpose of this research is to propose a novel technique and evaluation method that can improve, re-conﬁgure and select the aspired Intel-ligent Assessment Systems (IAS) for teaching materials to promote education levels.

Since 1990, the four British educational reform contexts have been: (a) the right to national centralization reform; (b) the priority of compulsory education; (c) vocational education mainstreaming; and (d) the ability to pursue educational evaluation (DfE, 1991).

The report of the Mayer Committee proposed to advise the Austra-lian Education Council (AEC) and the Ministers for Vocational Educa-tion, Employment and Training (MOVEET) on employment-related key competencies for post-compulsory education and training (Mayer, 1992). The Education Commission of Hong Kong proposed the following: ‘‘Learning is the key to one’s future, and education is the gateway to our society’s tomorrow’’. Education enables indi-viduals to develop their potential, construct knowledge and enhance the quality of their personal lives (EC, 2000).

In general, no matter what style of education reform is used, the key competencies are the major concern for national education re-form at the beginning of the 21st century.

The core objective of Nine-Year Integrated Curriculum for pri-mary schools in Taiwan is to ‘‘enable students to demonstrate their talents instead of just scoring high on exams’’. The determining factor in education reform is to declare the competence indicators for necessary educational behavior in primary school (MOE, 2002). In the reform process, for all domains, the enriched rate of compe-tence indicators for educational materials and methods is very meaningful. Because educational materials and methods in differ-ent domains have their own style, we should evaluate these teach-ing materials separately.

doi:10.1016/j.eswa.2011.03.050

⇑Corresponding author at: Distinguished Chair Professor, Kainan University, No. 1, Kainan Road, Shinshing Tsuen, Luchu Shiang, Taoyuan 338, Taiwan. Tel.: +886 937 541 184.

E-mail addresses: hsiung.chen@msa.hinet.net(C.-H. Chen), ghtzeng@cc.nctu. edu.tw,ghtzeng@mail.knu.edu.tw,arthur@mail.knu.edu.tw(G.-H. Tzeng).

Contents lists available atScienceDirect

Expert Systems with Applications

(2)

(Decision Making Trial and Evaluation Laboratory) technique for evaluating, comparing, and improving the effectiveness of compe-tence indicators in the various publications for teaching materials in primary school based on different viewpoints. The ANP (Analysis Network Process) weights are based on the DEMATEL technique with the novel MCDM method for resolving the problems of depen-dence and feedback among criteria. Then, a VIKOR technique with ANP weights is proposed for addressing and reducing the perfor-mance gaps for each criterion, thus hopefully improving, re-conﬁg-uring and selecting the aspired Intelligent Assessment Systems for teaching materials. An empirical study of Mandarin Chinese teach-ing materials in Grade 1 of primary school based on this system de-sign of three publishers is illustrated to verify the effectiveness of the proposed method. The results can improve the efﬁciency and quality of the authored Mandarin Chinese teaching materials and may extend to other Learning Areas.

The remainder of this paper is organized as follows. In Section2, the aspired intelligent assessment systems for teaching materials

with MCDM are introduced. In Section3, a novel MCDM method

based on the DEMATEL technique is proposed. In Section 4, an

empirical study for the aspired IAS for Mandarin Chinese teaching materials is presented to show the process that our proposed method entails, and discussions are conducted. Finally, in Section5, concluding remarks are presented.

2. Intelligent assessment systems for teaching materials with MCDM method

In recent decades, competence-based education has become the mega trend impacting the education reform strategies of the majority number of governments throughout the world. In the fol-lowing section, the literatures related to core competence (CC), the intertwined effects of an assessment system for teaching materials, will be reviewed as a foundation for the development of the theo-retical framework of this paper. We also give an example of Taiwan to explain the basic concepts of Educational Reform for teaching materials.

2.1. Educational reform of Taiwan (MOE, 2002)

In keeping with the progress of the 21st century and the global trends of educational reform, Taiwan must engage in educational reform in order to foster national competitiveness and boost the overall quality of our citizen’s lives.

The Ministry of Education (MOE) of Taiwan, therefore, has initi-ated curricular and instructional reforms in primary and junior high school education. These reforms have been based on the

Action Plan for Educational Reform approved by the Executive Yuan of Taiwan. Because the curriculum is not only the core of schooling but also the foundation on which teachers plan learning activities, the MOE places top priority on the development and implementation of the Grade 1–9 Curriculum. Curricular reforms are necessary and will be timely for the following reasons: (a) meeting national development needs; and (b) meeting public expectations.

2.2. Assessment system for mandarin chinese teaching materials Mandarin Chinese is one of the curricula included in the Lan-guage Arts. It is divided into 3 stages: Grades 1–3, Grades 4–6, and Grades 7–9.

2.2.1. Mandarin Chinese’s curriculum goal

Based on the Curriculum Goals and CCs of the Grade 1–9 Curric-ulum, a more detailed description is given in sub Sections4.1.2 and 4.1.3, with the Mandarin Chinese Curriculum Goal deﬁned as fol-lows: (1) to utilize language to inspire individual potential and to cultivate learning; (2) to cultivate interest in writing and enhance ability to appreciate literature; (3) to equip with the self-learning ability for language learning and lay the foundation for lifelong learning; (4) to utilize language and words for expressions of emotion, experience-sharing and communications; (5) to utilize language expression to adapt to one’s environment and to demon-strate appropriate behavior; (6) to understand and recognize the Chinese, Taiwanese and foreign cultures and rituals through lan-guage learning; (7) to utilize the power of lanlan-guage to develop and implement plans effectively; (8) to combine the information of lan-guage and technology to enhance learning and to expand ﬁelds of study; (9) to cultivate an interest in language exploration and a pro-active attitude towards language-learning; (10) to utilize language for independent thinking and problem-solving.

2.2.2. Competence indicators of Mandarin Chinese’s curriculum of grade 1–9 curriculum

Based on the requirements for children’s intellectual develop-ment, the numbers for the competence indicators of the Mandarin Chinese Curriculum for each stage are deﬁned (seeTable 1).

In general, this includes listening, speaking, reading and writing of languages, and developing basic communication competencies. From the academic point of view, six terms were developed as fol-lows: (1) phonetic symbol applications; (2) listening skills; (3) the ability to speak; (4) literacy and writing ability; (5) reading skills; and (6) writing skills.

Table 1

Numbers for competence indicators for the mandarin Chinese curriculum for each stage.

Criterion of CC Stage1 Stage2 Stage3 Sum

D1: Physical, mental, and spiritual mold

C1: Self-understanding and exploration of potential 24 19 18 61

C2: Appreciation, representation, and creativity 20 17 17 54

C3: Career planning and lifelong learning 9 13 10 32

D2: Interpersonal and social relations

C4: Expression, communication, and sharing 8 9 12 29

C5: Respect, care and teamwork 9 9 13 31

C6: Cultural learning and international understanding 6 8 6 20

C7: Planning, organizing and putting plans into practice 7 7 6 20

D3: The use of Life Science and Technology

C8: Utilization of technology and information 4 11 8 23

D4: Logical thinking and reasoning

C9: Active exploration and study 8 7 9 24

C10: Independent thinking and problem-solving 9 8 7 24

Sum 104 108 106 318

(3)

3. A novel MCDM method based on the DEMATEL technique with ANP

The structure of the MCDM problem will be derived using the DEMATEL method. The priorities of every determinant are based on the structure derived by using the ANP. The VIKOR technique will be leveraged to calculate the compromise ranking of the alter-natives. Finally, the assessment system for Mandarin Chinese teaching materials will be derived. In summary, this evaluation framework consists of ﬁve main phases: (1) establishing determi-nants using the questionnaire survey method; (2) building the structure of network relation map (NRM) for the determinants by using DEMATEL; (3) calculating the priorities of every determinant using the ANP based on the structure of NRM derived by using DEMATEL in (2); (4) ranking the priorities of the assessment sys-tem for teaching materials using the VIKOR technique; and ﬁnally (5) establishing the assessment system for teaching materials and achieving the aspired levels (seeFig. 1).

3.1. DEMATEL Technique with ANP

The DEMATEL technique was developed by the Battelle Geneva Institute: (1) to analyze complex ‘world problems’ dealing mainly with interactive man-model techniques; and (2) to evaluate qual-itative and factor-linked aspects of societal problems (Gabus & Fontela, 1972). The applicability of the method is widespread, with applications ranging from industrial planning and decision-making to urban planning and design, regional environmental assessment, the analysis of world problems, and so forth. It has also been suc-cessfully applied in many situations and fields, such as those of marketing strategies, control systems, safety problems, developing the competencies of global managers, group decision-making and so on (Chen, Lien, & Tzeng, 2010; Chiu, Chen, Tzeng, & Shyu, 2006; Huang, Shyu, & Tzeng, 2007; Lee, Tzeng, Hsu, & Huang, 2009; Li & Tzeng, 2009a, 2009b; Lin & Tzeng, 2009; Lin & Wu, 2008; Liou, Tzeng, & Chang, 2007; Ou Yang, Leu, & Tzeng, 2009; Ou Yang, Shieh, Leu, & Tzeng, 2008; Tzeng, Chen, Yu, & Shih, 2010; Wu & Lee, 2007). Furthermore, a hybrid model combining the two methods has been widely used in various fields – for example, e-learning evaluation (Tzeng, Chiang, & Li, 2007), teach-ing materials assessment (Chen and Tzeng, 2009), airline safety measurement (Liou et al., 2007), and innovation policy portfolios for Taiwan’s SIP Mall (Huang et al., 2007). Therefore, in this paper we use DEMATEL not only to detect complex relationships and build a NRM for the criteria but also to obtain the influence levels of each element over others; we then adopt these influence level values as the basis of the normalization supermatrix for determin-ing ANP weights to obtain the information about relative impor-tance. To apply the DEMATEL method smoothly, the authors refined the definitions based on the above authors and produced the essential definitions indicated below. The DEMATEL method is based upon graph theory, enabling us to plan and solve problems visually, so that we may divide multiple criteria into a relationship of cause and effect to better understand causal relationships. Direc-ted graphs (also called digraphs) are more useful than directionless graphs because digraphs will demonstrate the directed

relation-ships of sub-systems. A digraph typically represents a communica-tion network or a dominacommunica-tion relacommunica-tionship between individuals. Suppose a system contains a set of elements, S = {s1, s2, . . . , sn},

and particular pair-wise relationships are determined for modeling with respect to a mathematical relationship, MR. Next, portray the relationship MR as a direct-relation matrix that is indexed equally in both dimensions by elements from the set S. Then, extract the case for which the number 0 appears in the cell (i, j) if the entry is a positive integral that has the following meaning: the ordered pair (si, sj) is in the relationship MR; and has the kind of

relation-ship regarding that element such that sicauses element sj. The

di-graph portrays a contextual relationship between the elements of the system, in which a numeral represents the strength of influ-ence (Fig. 2). The number between factors is the influence or influ-enced degree. For example, an arrow from s1to s2represents the

fact that s1 inﬂuences s2 and that its inﬂuenced degree is two.

The DEMATEL method can convert the relationship between the causes and effects of criteria into an intelligible structural model of the system (Chiu et al., 2006)..

Definition 1. The pair-wise comparison scale may be designated as five levels, where the scores 0, 1, 2, . . . , 4 represent the range from ‘no influence’ to ‘very high influence’.

Definition 2. The initial direct relation/influence matrix A is an n n matrix obtained by pair-wise comparisons in terms of influ-ences and directions of influence between the determinants, in which aijis denoted as the degree to which the ith determinant

affects the jth determinant.

A ¼ a11 a12 a1n a21 a22 a2n .. . .. . . . . .. . an1 an2 ann 2 6 6 6 6 4 3 7 7 7 7 5

Deﬁnition 3. The normalized direct relation/inﬂuence matrix N can be obtained through Eqs.(1) and (2), in which all principal diagonal elements are equal to zero.

N ¼ zA ð1Þ

Fig. 1. An analytical framework for aspired assessment system for teaching materials.

s2 2 3 1

3 s

₁

s

₄

s

₃ s2 2 3 1

3 s

₁

s

₄

s

₃

(4)

where z ¼ 1 max max 16i6n Xn j¼1 aij;max 16j6n Xn i¼1 aij ! , : ð2Þ

In this case, N is called the normalized matrix. Because

limq!1Nq¼ ½0nn

Deﬁnition 4. Then, the total relationship matrix T can be obtained using Eq.(3), where I stands for the identity matrix.

T ¼ N þ N2

þ þ Nq¼ NðI NÞ1 ð3Þ ½Explain

T ¼ N þ N2þ þ Nq

¼ NðI þ N þ N2þ þ Nq1ÞðI NÞðI NÞ1 ¼ NðI NqÞðI NÞ1

¼ NðI NÞ1; when limq!1Nq¼ ½0nn

where q ? 1 and T is a total inﬂuence-related matrix; N is a direct inﬂuence matrix and N = [xij]nn;limq?1(N2+ + Nq) stands for a

indirect inﬂuence matrix, 0 6Pn

j¼1xij<1 and 0 6Pni¼1xij<1, and

only onePn_j¼1xijorPni¼1xij is equal to 1 for "i, j, but not all. So,

limq?1Nq= [0]nn. The (i, j) element tijof matrix T denotes the

di-rect and indidi-rect inﬂuences of factor i on factor j.

Deﬁnition 5. The row and column sums are separately denoted as r and c within the total-relation matrix T through Eqs.(4) and (5).

T ¼ ½tij i; j 2 f1; 2; . . . ; ng ð4Þ r ¼ ½rin1¼ Xn j¼1 tij " # n1 and c ¼ ½cjn1¼ Xn i¼1 tij " #0 1n ð5Þ

where the vectors r and c denote the sums of the rows and columns, respectively.

Deﬁnition 6. Suppose that ridenotes the row sum of the ith row of

matrix T. Then, riis the sum of the inﬂuences of factor i on the

other factors, both directly and indirectly. Suppose that cjdenotes

the column sum of the jth column of matrix T. Then, cjis the sum of

the inﬂuences that factor j is receiving from the other factors. Fur-thermore, when i = j (i.e., the sum of the row sum and the column sum (ri+ ci) represents the index indicating the strength of the

inﬂuence, both dispatching and receiving), (ri ci) is the degree

of the central role that factor i plays in the problem. If (ri ci) is

positive, then factor i primarily is inﬂuencing the strength of the other factors; and if (ri ci) is negative, then factor i primarily is

receiving inﬂuence from other factors (Huang et al., 2007; Liou et al., 2007; Tamura, Nagata, & Akazawa, 2002).

3.2. The ANP method

The ANP method, a multi-criteria theory of measurement

devel-oped bySaaty (1996), provides a general framework for dealing

with decisions without making assumptions about the indepen-dence of higher-level elements from lower-level elements and about the independence of the elements within a level as in a

hier-archy. Compared with traditional MCDM methods (Saaty, 2005) –

e.g. AHP (Analytic Hierarchy Process), TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution), ELECTRE (ELimina-tion Et Choix Traduisant la REalité), etc. (Lee et al., 2007) – which usually assume independence between criteria, ANP, a new theory that extends AHP to deal with dependence in feedback and utilizes

the supermatrix approach (Saaty, 2003), is a more reasonable tool for dealing with complex MCDM problems in the real world. In this section, the concepts of the ANP are summarized based on Saaty’s earlier works (Saaty, 1996, 1999, 2005).

The ANP is a coupling of two parts. The first consists of a control hierarchy or network of criteria and subcriteria that control the interactions. The second is a network of influences among the ele-ments and clusters. The network varies from criterion to criterion, and a different supermatrix of limiting influence is computed for each control criterion. Finally, each of these supermatrices is weighted based on the priority of its control criterion, and the re-sults are synthesized through addition for all the control criteria (Saaty, 2004). A control hierarchy is a hierarchy of criteria and sub-criteria for which priorities are derived in the usual way with re-spect to the goal of the system being considered.

The criteria are used to compare the components of a system, and the subcriteria are used to compare the elements. The criteria with respect to which influence is presented in individual super-matrices are called control criteria. Because all such influences ob-tained from the limits of the several supermatrices will be combined to obtain a measure of the priority of overall influence, the control criteria should be grouped in a structure to be used to derive priorities for them. These priorities will be used to weight the corresponding individual supermatrix limits and sum up. The analysis of priorities in a system can be thought of in terms of a control hierarchy with dependence among its bottom-level

alter-natives arranged as a network as shown in Fig. 3. Dependence

can occur within the components and between them.

A control hierarchy at the top may be replaced by a control net-work with dependence among its components, which are collec-tions of elements whose funccollec-tions derive from the synergy of their interaction and that hence have a higher-order function not found in any single element. The criteria in the control hierarchy that are used for comparing the components are usually the major parent criteria whose subcriteria used to compare the elements need to be more general than those of the elements because of the greater complexity of the components.

A network connects the components of a decision system. According to size, there will be a system made up of subsystems, with each subsystem made up of components and each component made up of elements. The elements in each component interact or have an inﬂuence on some or all of the elements of another com-ponent with respect to a property governing the interactions of the entire system, such as energy, capital, or political inﬂuence. Fig. 4demonstrates a typical network. Those components that no

Goal Criteria

Subcriteria Control Criteria

A possible different network under each subcriterion of the control hierarchy Goal

Criteria

Subcriteria

Control Criteria

A possible different network under each subcriterionof the control hierarchy

Source: (Saaty, 1996) Fig. 3. The control hierarchy.

(5)

arrow enters are known as source components, such as C1and C2.

Those from which no arrow leaves are known as sink components,

such as C5. Those components which arrows both enter and exit

leave are known as transient components such as C3 and C4. In

addition, C3and C4form a cycle of two components because they

feed back and forth into each other. C2and C4have loops that

nect them to themselves and are inner-dependent. All other con-nections represent dependence between components, which are thus known to be outer-dependent.

A component (dimension) of a decision network that was

de-rived by the DEMATEL method in sub Section3.1will be denoted

by Ch, h = 1, . . . , m;assume that it has nhelements (determinants),

which we denote by eh1;eh2; . . . ;ehnh. The inﬂuence of a given set of elements (determinants) in a component (dimension) on any element in the decision system is represented by a ratio scale pri-ority vector derived from paired comparisons of the comparative importance of one criterion and another criterion with respect to the interests or preferences of the decision-makers. This relative importance value can be determined using a scale of 1–9 to repre-sent equal importance to extreme importance (Saaty, 1996). The inﬂuence of elements (determinants) in the network on other ele-ments (determinants) in that network can be represented using the following supermatrix:

A typical entry Wij in the supermatrix is called a block of the

supermatrix in the following form, where each column of Wijis a

principal eigenvector of the inﬂuence of the elements nants) of the ith component of the network on an element (determi-nant) in the jth component. Some of its entries may be zero

corresponding to those elements (determinants) that have no inﬂuence. Wij¼ wi1j1 wi1j2 wi1j_nj wi2j2 wi2j2 wi2j_nj .. . .. . . . . .. . wi_nij1 winij2 winijnj 2 6 6 6 6 6 4 3 7 7 7 7 7 5

After forming the supermatrix, the weighted supermatrix is derived by transforming all columns sum to unity exactly. This step is very much similar to the concept of the Markov chain in terms of ensur-ing that the sum of these probabilities of all states equals 1. Next, the weighted supermatrix is raised to limiting powers, such as limh?1Whto get the global priority vector or called weights (Huang,

Tzeng, & Ong, 2005). 3.3. VIKOR

The compromise ranking method (VIKOR) was proposed by Opricovic (Opricovic, 1998) as one applicable technique to imple-ment within MCDM. We assume that the alternatives are denoted as A1, A2, . . . , Ak, . . . , Am. The rating (performance score) of the jth

criterion is denoted by fkj for alternative Ak, wjis the weight of

the jth criterion, expressing the relative importance of the criteria, where j = 1, 2, . . . , n, and n is the number of criteria. The VIKOR method began with the following form of theLp metric:

Lpk¼ Xn j¼1 wj fj fkj h . f j f j #p ( )1=p

where 1 6 p 6 1, k = 1, 2, . . . , m; weight wjis derived using the ANP

according to the NRM based on the DEMATEL method. The VIKOR method also uses Lp¼1

k (as Sk) and Lp¼1k (as Qk) to formulate the

rank-ing measure (Opricovic, 1998; Opricovic & Tzeng, 2002, 2004, 2007; Tzeng, Teng, Chen, & Opricovic, 2002, 2005).

Sk¼ Lp¼1k ¼ Pn j¼1 wj fj fkj = f j fj h i Qk¼ Lp¼1k ¼ maxj wj fj fkj = f j fj jj ¼ 1; 2; . . . ; n n o

The compromise solution minkLpkwill be chosen because its value is

closest to the ideal/aspired level. In addition, when pis small, group utility is emphasized (such as p = 1), and as pincreases to p = 1, the individual maximal regrets/gaps receive more importance, as shown by Yu (Freimer & Yu, 1976; Yu, 1973). Therefore, miniSi

emphasizes the maximum group utility, whereas miniQi

empha-sizes selecting the minimum of the maximum individual regrets. Based on the above concepts, the compromise ranking algorithm VI-KOR has the following steps.

Step 1. Normalize the original rating matrix. In this step, we deter-mine the best f

j and the worst fj values of all criterion

functions, j = 1, 2, . . . , n. If we assume that the jth function represents a beneﬁt, f

j ¼ maxifkj (or setting an aspired

level) and f

j ¼ minifkj(or setting a tolerable level).

Alter-natively, if we assume that the jth function represents a cost, f

j ¼ minifkj (or setting an aspired level) and

f

j ¼ maxifkj(or setting a tolerable level). Moreover, an

ori-ginal rating matrix is transformed into a normalized weight-rating matrix with the following formula:

rkj¼ fj fkj = f j fj

Step 2. Compute the values Sk, Qk, when k = 1, 2, . . . , m, using the

rela-tions (Ou Yang et al., 2008, Ou Yang, Leu, & Tzeng, 2009)

Source Component Source Component Source Component (Feedback loop) Source Component (Feedback loop) Intermediate Component (Transient State) Intermediate Component (Transient State) Sink Component (Absorbing State) Sink Component (Absorbing State) Intermediate Component (Recurrent State) Intermediate Component (Recurrent State) Outerdependence Innerdependence loop C₁ C3 C₄ C5 C2 Source Component Source Component Source Component (Feedback loop) Source Component (Feedback loop) Intermediate Component (Transient State) Intermediate Component (Transient State) Sink Component (Absorbing State) Sink Component (Absorbing State) Intermediate Component (Recurrent State) Intermediate Component (Recurrent State) Source Component Source Component Source Component (Feedback loop) Source Component (Feedback loop) Intermediate Component (Transient State) Intermediate Component (Transient State) Sink Component (Absorbing State) Sink Component (Absorbing State) Intermediate Component (Recurrent State) Intermediate Component (Recurrent State) Outerdependence Innerdependence loop C₁ C3 C₄ C5 C2 Source: (Satty, 1996)

(6)

Sk¼ Xn j¼1 wjrkj and Qk¼ maxjfrkjj ¼ 1; 2; . . . ; ng

where Skand Qkshow the mean of group utility and maximal regret,

respectively. Using the traditional VIKOR method, Qkis represented

as maxj{wjrkjjj = 1, 2, . . . , n}, which implies that group utility is more

important than maximal regret. Because Qkis only a part of Sk, Skis

unquestionably more than Qk. Therefore, Skis emphasized more

than Qkusing the traditional VIKOR method. However, maximal

re-gret is also very important in practice and is usually taken into

ac-count to improve it. Therefore, to balance Sk and Qk,

Qk= maxj{rkjjj = 1, 2, . . . , n} is used instead of the traditional VIKOR

Qk.

Step 3. Compute the index values Rk, i = 1, 2, . . . , m,using the relation

Rk¼

v

ðSk SÞ=ðS SÞ þ ð1

v

ÞðQk QÞ=ðQ QÞ

where

S_{¼ min}

kSk; S¼ maxkSk; Q¼ minkQk; Q¼ maxkQk;

(here, we can also set the best value to 0 and the worst value to 1) and 0 6

v

6_{1, where}

v

_{is introduced as a weight for the strategy of} maximum group utility, whereas 1

v

is the weight of the individ-ual regret.

Step 4. Rank the alternatives, sorting by the value of Sk, Qk, and Rk,

for k = 1, 2, . . . , m, in decreasing order. Propose as a compro-mise the alternative (A(1)), which is ranked ﬁrst by the measure min{Rkjk = 1, 2, . . . , m} if the following two

condi-tions are satisﬁed:

C1. Acceptable advantage: R(A(2)_{) R(A}(1)_{) P 1/(m 1),}

where A(2)_{is the alternative with the second position}

on the ranking list by R; m is the number of alternatives.

C2. Acceptable stability in decision making: Alternative A(1)

must also be the best ranked by Sk or/and Qk,

k = 1, 2, . . . , m.

A set of compromise solutions is proposed if one of the condi-tions is not satisﬁed. The set of compromise solucondi-tions consists of the following: (1) alternatives A(1)and A(2)if only condition C2 is not satisﬁed; (2) alternatives A(1)_{, A}(2)_{, . . . , A}(M)_{if condition C1 is}

not satisﬁed. A(M)_{is determined by the relation R(A}(M)_{) R(A}(1)_{) <}

1/(m 1) for maximum M (the positions of these alternatives are close).

The compromise-ranking method (VIKOR method) determines the compromise solution; the obtained compromise solution is acceptable to the decision-makers because it provides the maxi-mum group utility of the majority (represented by minS) and a minimum of individual maximal regret for the opponent (repre-sented by minQ). The model uses the DEMATEL and ANP

proce-dures in sub Sections 3.1 and 3.2 to obtain the weights of the

criteria with dependence and feedback; it uses the VIKOR method to obtain the compromise solution.

4. An empirical study of the aspired assessment systems for mandarin chinese teaching materials

In this section, an example being modiﬁed from a real case will be presented to demonstrate the effectiveness of the proposed novel MCDM framework using the DEMATEL technique. One empirical study example will be based on an example modiﬁed from three leading publishers.

In this case study, Mandarin Chinese teaching materials (six books) for Grade 1 in primary school in Taiwan are selected; these materials were edited by those publishers.

4.1. Background description

In the twenty-ﬁrst century, major changes have taken place around the world in social, political, economic and cultural arenas. These changes are not only global but also national. In a drastic change, the most countries have become aware of the importance of education and culture. Education reform in these countries has been carried out to stimulate personal potential, to overtake the ﬁne culture, and to promote social progress.

After six decades of postwar development, Taiwan has transi-tioned from a traditional agricultural society into a modern indus-trial society. Political, economic, and cultural environments are faced with modernization, industrialization and the technological inﬂuences of structural adjustment and reconstruction. The impact of education reform is one of the most far-reaching and extensive implications of a major reform. It affects national self-positioning, slushing the social consciousness, establishing a new culture and developing national competitiveness for the new century.

The effectiveness of economic development and of the develop-ment of democracy in Taiwan have aroused praise from more developed countries. It is a fact that the main contributing factor is the spread of education and enhancement of people’s quality of life. Taiwan is becoming an educational community. However, in the development of education over the years, many problems have emerged, and delays in solving the problems have made them even more complicated.

In light of this, the Council on Education Reform was established in September, 1994. It was responsible for educational reform and the educational development of research and consideration, the Commission ‘‘Education overall reform Consultation Report’’ was presented in December of 1996.

Taiwan should actively engage in reform. Issues that need to be addressed include the following: establishing a life-long learning society; enhancing equal opportunities in education; guiding examination biased towards intellectual culture; improving course materials and assessments; improve the multi-teacher education system; and improving the efficiency of the use of educational re-sources. The influence of education reform on the social and per-sonal is significant. We must ponder social change movements and carefully consider how the value of benchmarks of socio-cul-tural development is determined, when they occur in the context of education reform, rather than acting rashly.

The Council has developed a comprehensive education reform proposal. It is divided into ﬁve dimensions as follows: (1) educa-tion deregulaeduca-tion, (2) good educaeduca-tional care for every student, (3) smoother avenues to higher education, (4) improved education quality, and (5) developing a life-long learning society.

Speaking of the need to provide proper educational care for every student, it is clear that schools have not given adequate attention to students, especially disadvantaged students. This has been mainly due to the rigidity of primary school and junior high school education (compulsory education) in Taiwan, with a uni-form system and curriculum, coupled with a lack of long-term investment in resources, the unusual effect of teaching, and a sin-gle mode for entrance writing examinations at senior high schools and universities. Early on in the education of those disadvantaged students, they were not able to develop a good foundation for learning, and they were then exposed to large class size and a lack of timely and adequate care in general, so that their performance was different from that of the other students and were relatively vulnerable at their schools.

(7)

The Council proposed the following speciﬁc recommendations: (1) to reform the curriculum and instruction; (2) to reduce school size and class size; (3) to open autonomous schools; (4) to stimu-late schools to use the inner strength of their own power; (5) to as-sist each student with basic competence; (6) to establish a remedial teaching system; (7) to strengthen career counseling and provide multiple approaches; (8) to renew a system of student behavioral counseling; (9) to weaken the physical and psychologi-cal barriers to education; (10) to attach importance to aboriginal education; (11) to implement gender equality in education; and (12) to protect the quality of early childhood education.

In the meantime, the Education Reform Committee, Taiwan, MOE proposed the curricular reforms for the Grades 1–9 (compul-sory education) Curricula.

The current Curriculum Frameworks for Primary Schools and Junior High Schools were revised and promulgated in 1993 and 1994, respectively. Although the current Curriculum Frameworks have been gradually and properly implemented, the MOE believes that innovative thinking and practice in education are the prereq-uisites for success in the new century. Therefore, the MOE has launched plans for another curricular reform to build up consensus and integrate efforts at education reform to create a new and bet-ter environment for school education. The development of a new curriculum was divided into three stages. The duration and major tasks for each stage are shown in detail as follows: (1) Stage 1 (from April 1997 to September 1998): Establishing the Special Panel on the Development of Primary and Junior High Schools’ Cur-ricula; (2) Stage 2 (from October 1998 to November 1999): Estab-lishing the Panel on Researching and Formulating the Guidelines of Each Learning Area in the Grade 1–9 Curriculum, which covered the primary school and junior high school levels of education; and (3) Stage 3 (from December 1999 to August 2002): Establish-ing the Review Committee on the Revision and Formulation of Ele-mentary and Junior High School Curricula.

4.1.1. Core rationale

The aim of education is to foster sound minds and character in students. Students should be taught democratic values, the Rule of Law, and humanitarian ideals; they should develop strong and healthy physiques, learn how to think for themselves and be crea-tive. Every government hopes that the school system will produce outstanding citizens with a sense of patriotism and the ability to adopt a global perspective. In essence, education is a learning pro-cess to help students explore their potential and develop their capacity to adapt and make the necessary efforts to improve their living environment. Given that, the following ﬁve basic aspects are emphasized and included in Grade 1–9 Curricula designed for the new century: developing humanitarian attitudes, enhancing the ability to integrate, cultivating democratic literacy, fostering both indigenous awareness and a global perspective, and building up the capacity for lifelong learning. The core components of each as-pect are as follows: (A) humanitarian attitude; (B) integration abil-ity; (C) democratic literacy; (D) native awareness and a global perspective; and (E) capacity for lifelong learning.

4.1.2. Curriculum goals

The curricula for primary and junior high schools will adopt the following principles: (A) to involve all aspects of daily life that are related to students’ mental and physical development; (B) to encourage the development of individuality and the exploration of one’s potential; (C) to foster democratic literacy and respect for different cultures; and (D) to develop scientiﬁc understanding and competencies to meet the needs of modern life.

The aim of national education is to teach students to obtain ba-sic knowledge and to develop the capacity for lifelong learning, thus cultivating able citizens who are mentally and physically

healthy, vigorous and optimistic, gregarious and helpful to the community, intellectually curious and reﬂective, tolerant, and cre-ative, with a positive attitude and a global perspective. Schools will achieve such ideals through the promotion of educational learning activities that emphasize humanity, practicality, individuality, comprehensiveness, and modernity. Such activities include inter-actions between oneself and others, individuals and the commu-nity, and humans and nature. Regarding this aspect of national education, we must guide our students to achieve the following curriculum goals: (A) to enhance their self-understanding and ex-plore their individual potential; (B) to develop creativity and the ability to appreciate beauty and present their own talents; (C) to promote abilities related to career planning and lifelong learning; (D) to cultivate knowledge and skills related to expression, com-munication, and sharing; (E) to learn to respect others, care for the community, and facilitate teamwork; (F) to promote further cultural learning and international understanding; (G) to strength-en knowledge and skills related to planning, organizing, and imple-mentation; (H) to acquire the ability to utilize technology and information; (I) to encourage an attitude of active learning and studying; and (J) to develop abilities related to independent think-ing and problem-solvthink-ing.

4.1.3. Core competencies

To achieve the aforementioned goals, the curricular design of primary and junior high school education shall focus on the needs and experiences of students and aim to develop the core compe-tencies that a modern citizen should possess. Such CCs (core com-petencies) refer to the key competencies, which deﬁned by the Mayer Committee are: (A) collect, analyze and organize informa-tion; (B) communicate ideas and informainforma-tion; (C) plan and orga-nize activities; (D) co-operate with others and help sustain the group’s ability to work; (E) use mathematical concepts and tech-nologies; (F) solve problems; (G) use technology (Mayer, 1992).

Thus, the CCs for the Grade 1–9 curriculum reform in Taiwan may be categorized as follows (MOE, 2002): (A) self-understanding and exploration of potential; (B) appreciation, representation, and creativity; (C) career planning and lifelong learning; (D) expres-sion, communication, and sharing; (E) respect, care and teamwork; (F) cultural learning and international understanding; (G) plan-ning, organizing and putting plans into practice; (H) utilization of technology and information; (I) active exploration and study; and (J) independent thinking and problem-solving.

With reference to curricular principles (see Section4.1.2), the CCs may be divided into 4 dimensions: (1) the physical, mental, and spiritual mold (A-C); (2) interpersonal and social relations (D-G); (3) the use of Life Sciences and Technology (H); and (4) log-ical thinking and reasoning (I-J).

4.1.4. Learning areas

To foster the CCs in citizens, the curricula for primary and junior high school education should emphasize three dimensions: indi-vidual development, community and culture, and natural environ-ment. Thus, the Grade 1–9 Curriculum encompasses seven major learning areas: (A) Language Arts, including Mandarin and English; (B) Health and Physical Education; (C) Social Studies, including his-tory and culture, geography, social institutions, morals and norms, etc., and the incorporation of the aforementioned subjects into one’s daily life; (D) Arts and Humanities, including music instruc-tion and instrucinstruc-tion in the visual and performing arts; (E) Science and Technology, including learning about substances and energy, nature, the environment, ecological conservation, and information technology; (F) Mathematics, including acquiring the basic con-cepts of ﬁgures, shapes, and quantity; the ability to calculate and organize; and the ability to apply such knowledge and skills to dai-ly life; and (G) Integrative Activities, referring to activities that may

(8)

guide learners to practice, experience, and reﬂect upon the learning process as well as to testify to and apply what they have learned in real situations.

Based on the analytical frame for expanding them, CCs were first selected as determinants. Then, the structure of the assessment system for the Mandarin Chinese teaching materials definition problem was established using DEMATEL. After that, the weight of each determinant for the decision structure would be decided by using the ANP. The determinants are CCs. The crite-ria were confirmed as Competence Indicators and as determinants for editing Mandarin Chinese teaching material. Meanwhile, the relationships between the determinants and the ANP derivations of the weights of each determinant would also be derived for the case study.

The relationships between the determinants for the assessment system for Mandarin Chinese teaching materials were surveyed based on the opinions of teacher(s) who are editing Mandarin Chi-nese teaching materials or have been teaching Mandarin ChiChi-nese in primary school, Taiwan. The teachers are familiar with the assessment of Mandarin Chinese teaching materials.

With the understanding of the determinants of the assessment system for Mandarin Chinese teaching materials, appropriate assessment strategies will also be proposed to shorten the gap be-tween the level of the selected current teaching material and the

aspired levels of the determinants. Meanwhile, the proposed assessment system for Mandarin Chinese teaching materials will assist the publishers in surpassing the publisher leaders. Detailed procedures and results are illustrated below.

4.2. Decision problem network relation map structuring by DEMATEL The inter-relationships between the ten determinants will be deduced using the DEMATEL method introduced in sub Section3.1. First, the direct relation/influence matrix A is introduced (see Ta-ble 2). After that, the direct relation/influence matrix A is normal-ized based on Eq.(1). Then, the total relationship matrix is deduced based on Eq.(3)(seeTable 3). Finally, the strength of the influence for each determinant is deduced based on Eq.(5)(seeTable 4) (see Fig. 6).

The sequence for the strength of the inﬂuence of the determi-nants is as follows: (1) the use of Life Science and Technology (D3),

which includes the utilization of technology and information (C8);

(2) logical thinking and reasoning (D4), which is sequent as (a)

inde-pendent thinking and problem-solving (C10) and (b) active

explora-tion and study (C9); (3) physical, mental, and spiritual mold (D1),

which is sequent as (a) career planning and lifelong learning (C3).

(b) self-understanding and exploration of potential (C1) and (c)

appreciation, representation, and creativity (C2); (4) interpersonal

Table 2

Rating the CCs relationships/inﬂuences for grades 1–3 mandarin curricula.

D1 D2 D3 D4 C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 D1 C1 0.000 3.500 3.375 3.625 3.250 2.250 2.750 2.375 2.875 3.000 C2 3.250 0.000 2.875 3.000 2.625 2.875 2.750 2.250 2.500 2.375 C3 3.750 2.375 0.000 2.875 2.625 2.500 3.000 2.750 3.000 3.125 D2 C4 3.750 3.250 3.000 0.000 3.250 3.125 3.250 2.750 3.125 3.250 C5 2.375 3.000 2.125 3.500 0.000 3.500 3.375 2.375 2.750 3.000 C6 2.625 3.000 2.500 3.250 3.125 0.000 2.875 2.375 2.625 2.625 C7 2.625 2.625 3.375 3.375 3.375 2.875 0.000 2.750 3.000 3.125 D3 C8 2.375 2.500 2.875 3.125 2.375 2.500 3.000 0.000 3.250 3.250 D4 C9 3.000 3.000 3.375 3.250 3.000 2.875 3.500 3.125 0.000 3.750 C10 3.375 3.375 3.125 3.000 2.875 2.625 3.250 2.875 3.875 0.000

Remark: Surveyed based on the opinions of teacher(s) who are editing Mandarin Chinese teaching materials or have been teaching Mandarin Chinese teaching in primary school, Taiwan. Core Competencies (CCs) D1: Physical, mental, and spiritual mold D2: Interpersonal and social relations D3: The use of Life Science and Technology D4: Logical thinking and reasoning C1: Self-understanding and exploration of potential; C2: Appreciation, representation, and creativity; C3: Career planning and lifelong learning; C4: Expression, communication, and sharing; C5: Respect, care and teamwork; C6: Cultural learning and international understanding; C7: Planning, organizing and putting plans into practice; C8: Utilization of technology and information; C9: Active exploration and study; C10: Independent thinking and problem-solving.

Table 3

Result of the DEMATEL analysis of CC Relationships/Inﬂuences for Grades 1–3 Mandarin Curricula.

D1 D2 D3 D4 C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 D1 C1 1.1989 1.2839 1.2798 1.3786 1.2721 1.1868 1.3055 1.1287 1.2787 1.3022 C2 1.1962 1.0744 1.1645 1.2523 1.1539 1.1083 1.2002 1.0342 1.1654 1.1802 C3 1.2752 1.2146 1.1384 1.3175 1.2173 1.1574 1.2736 1.1063 1.2450 1.2678 D2 C4 1.3819 1.3446 1.3369 1.3402 1.3398 1.2754 1.3896 1.1999 1.3542 1.3787 C5 1.2329 1.2298 1.2038 1.3322 1.1317 1.1851 1.2818 1.0928 1.2346 1.2607 C6 1.1988 1.1890 1.1737 1.2813 1.1885 1.0379 1.2253 1.0562 1.1897 1.2083 C7 1.2867 1.2638 1.2847 1.3773 1.2810 1.2097 1.2242 1.1445 1.2880 1.3116 D3 C8 1.2067 1.1886 1.1998 1.2927 1.1809 1.1305 1.2441 0.9939 1.2232 1.2419 D4 C9 1.3678 1.3436 1.3544 1.4480 1.3390 1.2745 1.4038 1.2172 1.2641 1.4001 C10 1.3587 1.3351 1.3284 1.4204 1.3162 1.2490 1.3767 1.1927 1.3628 1.2651

Remark: Core Competencies (CCs) D1: Physical, mental, and spiritual mold D2: Interpersonal and social relations D3: The use of Life Science and Technology D4: Logical thinking and reasoning C1: Self-understanding and exploration of potential; C2: Appreciation, representation, and creativity; C3: Career planning and lifelong learning; C5: Respect, care and teamwork; C6: Cultural learning and international understanding; C7: Planning, organizing and putting plans into practice; C8: Utilization of technology and information; C9: Active exploration and study; C10: Independent thinking and problem-solving.

(9)

and social relations (D2), which are sequent as (a) respect, caring and

teamwork (C5), (b) cultural learning and international

understand-ing (C6), (c) expression, communication, and sharing (C4) and (d)

planning, organizing and putting plans into practice (C7).

4.3. Calculating the weights of determinants by ANP

Setting an appropriate assessment system as the goal, pairwise comparisons of the determinants were executed based on the total relationship matrix as deduced by DEMATEL. The inter-relation-ships between the goal and the dimensions of determinants of an assessment system are illustrated inFig. 5where the direction of the arrows indicates the direction of the inﬂuences. Finally, the to-tal relationship matrix serves as inputs for the ANP. By implement-ing the ANP, the limit super matrix W is calculated. Weights corresponding to each determinant (Table 5) are derived accord-ingly that will be used for the calculation of weighted averages and VIKOR scores.

4.4. Compromise ranking by VIKOR

The VIKOR technique being introduced for compromise ranking was applied after the determinants’ weight calculations by ANP in

sub Section 4.3. Meanwhile, the weighted averages for the

Mandarin Chinese teaching materials (six books) for Grade 1 in primary school, which were edited by three leading publishers, are also calculated for comparison. In general, the calculation results (Table 6–8) demonstrate that both global and local reached the same conclusions: publisher A publisher C publisher B.

4.5. Discussions and implications

Authoring teaching material is not an easy task. Meanwhile, there are no straightforward answers to the question of how teach-ing material should be authored to meet particular criteria. Not only criteria but also determinant, how teaching material should be authoring by considering factors being related to the authored curriculum. Also, from the perspective of MCDM, very little re-search has addressed the assessment system for teaching material, not to mention Mandarin Chinese teaching materials. In this research, a novel MCDM framework combining the DEMATEL tech-nique, ANP and VIKOR was proposed to address the above-mentioned problems; using this technique yielded satisfactory results.

The novel MCDM model consisting of DEMATEL, ANP and VI-KOR was formed as follows: (1) overcome the issue of deﬁning the assessment system for teaching material; (2) use innovative traditional MCDM approaches to resolve the problem of deﬁning the assessment system for teaching material; (3) clarify the vague correlations between the determinants of teaching material; and (4) create the aspired priorities for the authored teaching material.

For the assessment system for teaching materials, CCs were se-lected as determinants of the assessment, and categorized into 4 groups as the criteria of the assessment. These groups were as follows;

C1: Self-understanding and exploration of potential;

C2: Appreciation, representation, and creativity;

C3: Career planning and lifelong learning;

C4: Expression, communication, and sharing;

C5: Respect, care and teamwork;

C6: Cultural learning and international understanding;

C7: Planning, organizing and putting plans into practice;

C8: Utilization of technology and information;

C9: Active exploration and study;

C10:

In-dependent thinking and problem-solving.

Through application studies, we found the novel MCDM model to be applicable. DEMATEL establishes a reasonable assessment structure for dealing with criteria inﬂuence. Basically, the inﬂuence relationships of the results (seeFig. 6) were quite reasonable: Table 4

ri+ ciand ri ciof the DEMATEL analysis of CC Relationships/Inﬂuences for Grades 1–3 Mandarin Curricula.

CC ri+ ci ri ci

C1: Self-understanding and exploration of potential 25.3189 (5) 0.0886 (5)

C2: Appreciation, representation, and creativity 23.9970 (8) 0.9377 (10)

C3: Career planning and lifelong learning 24.6777 (6) 0.2515 (8)

C4: Expression, communication, and sharing 26.7817 (1) 0.0992 (6)

C5: Respect, care and teamwork 24.6058 (7) 0.2351 (7)

C6: Cultural learning and international understanding 23.5633 (9) 0.0659 (4)

C7: Planning, organizing and putting plans into practice 25.5963 (4) 0.2534 (9)

C8: Utilization of technology and information 23.0685 (10) 0.7357 (2)

C9: Active exploration and study 26.0180 (3) 0.8069 (1)

C10: Independent thinking and problem-solving 26.0219 (2) 0.3886 (3)

(10)

(a) The aim of education is to foster students’ sound mind and char-acter – ﬁrst, enhancing ‘‘D3: The use of Life Science and

Technol-ogy’’ may strengthen ‘‘D4: Logical thinking and reasoning’’ for

students; then, improving ‘‘D2: Interpersonal and social

rela-tions’’ becomes the goal; ﬁnally, the curriculum goal gives stu-dents a ‘‘D1: Physical, mental, and spiritual mold’’.

Table 5

Weights of the determinants being derived by ANP.

CC Local weights Globe weights (ANP)

D1: Physical, mental, and spiritual mold 0.3014

C1: Self-understanding and exploration of potential 0.3377 0.1018

C2: Appreciation, representation, and creativity 0.3312 0.0998

C3: Career planning and lifelong learning 0.3312 0.0998

D2: Interpersonal and social relations 0.4054

C4: Expression, communication, and sharing 0.2656 0.1077

C5: Respect, care and teamwork 0.2455 0.0995

C6: Cultural learning and international understanding 0.2335 0.0947

C7: Planning, organizing and putting plans into practice 0.2554 0.1035

D3: The use of Life Science and Technology 0.0895

C8: Utilization of technology and information 1.0000 0.0895

D4: Logical thinking and reasoning 0.2037

C9: Active exploration and study 0.4959 0.1010

C10: Independent thinking and problem-solving 0.5041 0.1027

Table 6

Satisfaction of Grade 1 Curriculum.

CC A Publisher B Publisher C Publisher

11 12 11 12 11 12

C1: Self-understanding and exploration of potential 6.0000 5.7500 5.5000 5.2500 6.2500 6.2500

C2: Appreciation, representation, and creativity 6.5000 6.7500 5.5000 7.0000 5.7500 7.2500

C3: Career planning and lifelong learning 5.7500 4.7500 4.2500 4.5000 6.2500 4.7500

C4: Expression, communication, and sharing 6.7500 7.5000 6.0000 6.5000 6.2500 7.2500

C5: Respect, care and teamwork 6.7500 6.2500 6.2500 6.5000 6.5000 6.5000

C6: Cultural learning and international understanding 5.2500 4.0000 4.2500 4.2500 4.0000 4.0000

C7: Planning, organizing and putting plans into practice 5.0000 5.0000 4.5000 4.7500 4.5000 4.2500

C8: Utilization of technology and information 3.7500 3.7500 4.0000 4.7500 4.5000 4.5000

C9: Active exploration and study 5.0000 5.2500 4.2500 4.2500 4.7500 4.5000

C10: Independent thinking and problem-solving 5.2500 4.7500 4.7500 4.5000 5.0000 6.0000

Remark: 11: the 1st Semester of Grade 1; 12: the 2nd Semester of Grade 1.

Table 7

Aspired level for Grade 1 Curriculum.

Criterion of CC Local weights Global weights (ANP) A Publisher B Publisher C Publisher

11 12 11 12 11 12

D1: Physical, mental, and spiritual mold 0.3014 Si 0.3917 0.4250 0.4914 0.4419 0.3916 0.3916

Qi 0.4250 0.5250 0.5750 0.5500 0.4250 0.5250

C1: Self-understanding and exploration of potential 0.3377 0.1018 0.4000 0.4250 0.4500 0.4750 0.3750 0.3750

C2: Appreciation, representation, and creativity 0.3312 0.0998 0.3500 0.3250 0.4500 0.3000 0.4250 0.2750

C3: Career planning and lifelong learning 0.3312 0.0998 0.4250 0.5250 0.5750 0.5500 0.3750 0.5250

D2: Interpersonal and social relations 0.4054 Si 0.4047 0.4263 0.4730 0.4472 0.4661 0.4459

Qi 0.5000 0.6000 0.5750 0.5750 0.6000 0.6000

C4: Expression, communication, and sharing 0.2656 0.1077 0.3250 0.2500 0.4000 0.3500 0.3750 0.2750

C5: Respect, care and teamwork 0.2455 0.0995 0.3250 0.3750 0.3750 0.3500 0.3500 0.3500

C6: Cultural learning and international understanding 0.2335 0.0947 0.4750 0.6000 0.5750 0.5750 0.6000 0.6000

C7: Planning, organizing and putting plans into practice 0.2554 0.1035 0.5000 0.5000 0.5500 0.5250 0.5500 0.5750

D3: The use of Life Science and Technology 0.0895 Si 0.6250 0.6250 0.6000 0.5250 0.5500 0.5500

Qi 0.6250 0.6250 0.6000 0.5250 0.5500 0.5500

C8: Utilization of technology and information 1.0000 0.0895 0.6250 0.6250 0.6000 0.5250 0.5500 0.5500

D4: Logical thinking and reasoning 0.2037 Si 0.4874 0.5002 0.5498 0.5624 0.5124 0.4744

Qi 0.5000 0.5250 0.5750 0.5750 0.5250 0.5500

C9: Active exploration and study 0.4959 0.1010 0.5000 0.4750 0.5750 0.5750 0.5250 0.5500

C10: Independent thinking and problem-solving 0.5041 0.1027 0.4750 0.5250 0.5250 0.5500 0.5000 0.4000

Integration 1.0000 Si 0.4374 0.4587 0.5056 0.4760 0.4606 0.4446

Qi 0.6250 0.6250 0.6000 0.5750 0.6000 0.6000

(11)

(b) For ‘‘D4: Logical thinking and reasoning’’, ‘‘C9: Active

explo-ration and study’’ is better than ‘‘C10: Independent thinking

and problem-solving’’ from the well-known Knowledge Management point of view.

(c) For ‘‘D2: Interpersonal and social relations’’, starting from

‘‘C6: Cultural learning and international understanding’’

may shorten the spiritual distance of the individual; then, there will be improvement in ‘‘C4: Expression,

communica-tion, and sharing’’; naturally, ‘‘C5: Respect, care and

team-work’’ will be consolidated; and ‘‘C7: Planning, organizing

and putting plans into practice’’ will be thoroughly considered.

(d) For ‘‘D1: Physical, mental, and spiritual mold’’, ‘‘C2:

Appreci-ation, representAppreci-ation, and creativity’’ are the benchmark of ’’C1: Self-understanding and exploration of potential’’; in

particular, ’’C3: Career planning and lifelong learning’’ is

bright and healthy.

ANP provides a general framework for dealing with decisions without making assumptions about the independence of higher-le-vel elements from lower-lehigher-le-vel elements and about the indepen-dence of the elements within a level as in a typical hierarchy (Saaty, 2005). Thus, in deﬁning the assessment system for teaching material illustrated in the paper, ANP is apparently a more reason-able tool for analysing the network structure with feedback.

In-deed, the weighting sequence for the CCs (see Table 5) is

seemingly the priority in the practice of CCs. It is contracted with the inﬂuence relationships mentioned above using the DEMATEL technique. For long-term concerns, who is responsible for assess-ing teachassess-ing materials is a serious concern.

The VIKOR method uses an aggregating function R representing ‘‘closeness to the ideal’’. In comparison with the TOPSIS, which determines a solution with the shortest distance from the ideal solution and the farthest distance from the negative-ideal solution (Chen & Hwang, 1992; Tzeng, Shiau, & Teng, 1994), VIKOR can se-Table 8

VIKOR versus weighted average results.

m A Publisher B Publisher C Publisher

11 12 11 12 11 12

D1Physical, mental, and spiritual mold (Si)m= 1 0.0016(4) 0.3350(3) 1.0000(1) 0.5041(2) 0.0000(6) 0.0001(5)

m= 0.5 0.0008(5) 0.5008(3) 1.0000(1) 0.6687(2) 0.0000(6) 0.3334(4)

(Qi)m= 0 0.0000(5) 0.6667(3) 1.0000(1) 0.8333(2) 0.0000(5) 0.6667(3)

D2Interpersonal and social relations (Si)m= 1 0.0000(6) 0.3153(5) 1.0000(1) 0.6223(3) 0.8984(2) 0.6031(4)

m= 0.5 0.0000(6) 0.6577(5) 0.8750(2) 0.6862(4) 0.9492(1) 0.8015(3)

(Qi)m= 0 0.0000(6) 1.0000(1) 0.7500(4) 0.7500(4) 1.0000(1) 1.0000(1)

D3The use of Life Science and Technology (Si)m= 1

m= 0.5 1.0000(1) 1.0000(1) 0.7500(3) 0.0000(6) 0.2500(4) 0.2500(4)

(Qi)m= 0

D4Logical thinking and reasoning (Si)m= 1 0.1478(5) 0.2933(4) 0.8568(2) 1.000(1) 0.4319(3) 0.0000(6)

m= 0.5 0.0739(6) 0.3133(5) 0.9284(2) 1.0000(1) 0.3826(3) 0.3333(4)

(Qi)m= 0 0.0000(6) 0.3333(4) 1.0000(1) 1.0000(1) 0.3333(4) 0.6667(3)

Integration (Si)m= 1 0.0000(6) 0.3133(4) 1.0000(1) 0.5671(2) 0.3403(3) 0.1069(5)

m= 0.5 0.5000(3) 0.6567(2) 0.7500(1) 0.2836(6) 0.4201(4) 0.3034(5)

(Qi)m= 0 1.0000(1) 1.0000(1) 0.5000(3) 0.0000(6) 0.5000(3) 0.5000(3)

Remark: 11: the 1st Semester of Grade 1; 12: the 2nd Semester of Grade 1.

(12)

lect the real ‘‘closest to the ideal’’ solution. On the other hand, a solution by TOPSIS is not always the closest to the ideal. A detailed comparison of TOPSIS and VIKOR has already been presented in the article by Opricovic and Tzeng (Opricovic & Tzeng, 2004, 2007).

Furthermore, the Lp-metric of VIKOR represents the grouping

le-vel in power p; in other words, p = 1 is the most emphasized group utility; p = 1 represents the most emphasized individuals. For

example (seeTable 8), regarding Book 11 from A Publisher, one

can say that when

m

¼ 1; RD1¼ 0:0016; RD2¼ 0:0000; RD3¼ 1:0000; RD4¼ 0:1478; RA11¼ 0:0000, and the sequences are, respectively, 4, 6, 1, 5, and 6. However, when

m

= 0, RD1¼ 0:0000; RD2¼ 0:0000; RD3¼ 1:0000; RD4¼ 0:0000; RA11 ¼ 1:0000, and the sequences are, respectively, 5, 6, 1, 6, and 1. In this case, Book 11 from A Publisher

has good behavior on average, but D3is the worst. Thus, when

m

= 0.5, the integration sequence is 3. This means that improving D3will achieve higher performance. In contrast, Book 12 from B

Pub-lisher may indicate the reverse conclusion. 5. Concluding remarks

Based on effective concern, begin from Mandarin Chinese. This paper would have mainly advanced work in the ﬁeld of the assess-ment system for teaching materials.

First, a novel MCDM framework was proposed to deﬁne the determinants of teaching materials (not exclusively for Mandarin Chinese).

Second, the traditional problem of the difficulty of defining the assessment system for teaching materials was resolved based on the novel MCDM approach being proposed. An important reason-ing conclusion was obtained: the more important determinant, the less influent determinant. The influent sequence of determi-nants is as follows: The use of Life Science and Technology (D3) Logical thinking and reasoning (D4) Interpersonal and

so-cial relations (D2) Physical, mental, and spiritual mold (D1) (see

Fig. 5). However, the weighting sequence of determinants is as fol-lows: Interpersonal and social relations (D2, 0.4054) Physical,

mental, and spiritual mold (D1, 0.3014) Logical thinking and

rea-soning (D4, 0.2037) The use of Life Science and Technology

(D3, 0.0895) (seeTable 5).

Finally, the difﬁculty of the traditional MCDM approach in selecting ‘‘rotten apple(s)’’ was also resolved based on the concep-tual advance in achieving the aspired level of criteria.

Acknowledgment

Over 20 teachers provided feedback on the questionnaire. Although I do not know who are you, I sincerely appreciate your kindness.

References

Chen, C.H., & Tzeng, G.H. (2009a). Combined DEMATEL technique with a novel MCDM method for creating the aspired intelligent assessment systems for mandarin Chinese teaching materials. In The 10th Asia Paciﬁc industrial engineering and management systems conference (APIEMS2009), Kitakyushu, Japan (pp. 2050–2061).

Chen, C. H., & Tzeng, G.H. (2009b). Creating the aspired intelligent fuzzy assessment systems for mandarin Chinese teaching materials. In The 17th national conference on fuzzy theory and its application, Kaohsiung, Taiwan (pp. 787–792). Chen, Y. C., Lien, H. P., & Tzeng, G. H. (2010). Measures and evaluation for environment watershed plan using a novel hybrid MCDM model. Expert Systems with Applications, 37(2), 926–938.

Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making: Methods and applications. Berlin: Springer.

Chiu, Y. J., Chen, H. C., Tzeng, G. H., & Shyu, J. Z. (2006). Marketing strategy based on customer behavior for the LCD-TV. International Journal of Management and Decision Making, 7(2/3), 143–165.

DfE (1991). Education and training for the 21st century. London: HMSO.

Education Commission (2000). Reform proposal for the education system in Hong Kong. Hong Kong: Education Bureau.

Freimer, M., & Yu, P. L. (1976). Some new results on compromise solutions for group decision problems. Management Science, 22(6), 688–693.

Gabus, A., & Fontela, E. (1972). World Problems, an invitation to further thought within the framework of DEMATEL. Geneva, Switzerland: Battelle Geneva Research Centre.

Huang, C. Y., Shyu, J. Z., & Tzeng, G. H. (2007). Reconﬁguring the innovation policy portfolios for Taiwan’s SIP mall industry. Technovation, 27(12), 744–765. Huang, J. J., Tzeng, G. H., & Ong, C. S. (2005). Multidimensional data in

multidimensional scaling using the analytic network process. Pattern Recognition Letters, 26(6), 755–767.

Lee, W. S., Tzeng, G. H., Hsu, C. Y., & Huang, J. M. (2009). Combined MCDM techniques for exploring stock selection based on Gordon model. Expert Systems with Applications, 36(3/2), 6421–6430.

Li, C. W., & Tzeng, G. H. (2009a). Identiﬁcation of a threshold value for the DEMATEL method using the maximum mean de-entropy algorithm to ﬁnd critical services provided by a semiconductor intellectual property mall. Expert Systems with Applications, 36(6), 9891–9898.

Li, C. W., & Tzeng, G. H. (2009b). Identiﬁcation of interrelationship of key customers’ needs based on structural model for services/capabilities provided by a semiconductor-intellectual property mall. Applied Mathematics and Computation, 215(6), 2001–2010.

Lin, C. L., & Tzeng, G. H. (2009). A value-created system of science (technology) park by using DEMATEL. Expert Systems with Applications, 36(6), 9683–9697. Lin, C. J., & Wu, W. W. (2008). A causal analytical method for group decision-making

under fuzzy environment. Expert Systems with Applications, 34(1), 205–213. Liou, J. J. H., Tzeng, G. H., & Chang, H. C. (2007). Airline safety measurement using a

hybrid model. Air Transport Management, 13(4), 243–249.

Mayer, E. (1992). Putting general education to work: The key competencies report. Canberra: Australian Government Publishing Service.

Ministry of Education (2002). General guidelines of grade 1-9 curriculum of elementary and junior high school education. Taipei: Ministry of education. Ministry of Education (2008). Education in Taiwan 2008. Taipei: Ministry of

Education.

Opricovic, S. (1998). Multicriteria optimization of civil engineering systems. Belgrade: Faculty of Civil Engineering.

Opricovic, S., & Tzeng, G. H. (2002). Multicriteria planning of post-earthquake sustainable reconstruction. Computer-Aided Civil and Infrastructure Engineering, 17(3), 211–220.

Opricovic, S., & Tzeng, G. H. (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156(2), 445–455.

Opricovic, S., & Tzeng, G. H. (2007). Extended VIKOR method in comparison with outranking methods. European Journal of Operational Research, 178(2), 514–529. Ou Yang, Y. P., Leu, J. D., & Tzeng, G. H. (2009). A VIKOR-based multiple criteria decision method for improving information security risk. International Journal of Information Technology and Decision Making, 8(2), 1–21.

Ou Yang, Y. P., Shieh, H. M., Leu, J. D., & Tzeng, G. H. (2008). A novel hybrid MCDM model combined with DEMATEL and ANP with applications. International Journal of Operations Research, 5(3), 1–9.

Saaty, T. L. (1996). Decision making with dependence and feedback: The analytic network process. Pittsburgh: RWS Publication.

Saaty, T.L. (1999). Fundamentals of the analytic network process. In Proceedings of international symposium on analytical hierarchy process, Kobe, Japan.

Saaty, R.W. (2003). The analytic hierarchy process (AHP) for decision making and the analytic network process (ANP) for decision making with dependence and feedback. Pittsburgh, PA: Creative Decisions Foundation.

Saaty, T. L. (2004). Fundamentals of the analytic network process – Dependence and feedback in decision-making with a single network. Journal of Systems Science and Systems Engineering, 13(2), 71–91.

Saaty, T. L. (2005). Theory and applications of the analytic network process. Pittsburg, PA: RWS Publications.

Tamura, M., Nagata, H., & Akazawa, K. (2002). Extraction and systems analysis of factors that prevent safety and security by structural models. In Proceedings of the 41st SICE annual conference, 3, Osaka, Japan (pp. 1752–1759).

Tzeng, G. H., Chiang, C. H., & Li, C. W. (2007). Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Systems with Applications, 32(4), 1028–1044.

Tzeng, G. H., Chen, W. H., Yu, R., & Shih, M. L. (2010). Fuzzy Decision Maps–A Generalization of the DEMATEL Methods. Soft Computing, 14(11), 1141–1150. Tzeng, G. H., Lin, C. W., & Opricovic, S. (2005). Multi-criteria analysis of

alternative-fuel buses for public transportation. Energy Policy, 33(1), 1373–1383. Tzeng, G. H., Shiau, T. A., & Teng, J. J. (1994). Multiobjective decision-making

approach to energy supply mix decisions in Taiwan. Energy Sources, 16(3), 301–316.

Tzeng, G. H., Teng, M. H., Chen, J. J., & Opricovic, S. (2002). Multicriteria selection for a restaurant location in Taipei. International Journal of Hospitality Management, 21(2), 171–187.

Wu, W. W., & Lee, Y. T. (2007). Developing global managers’ competencies using the fuzzy DEMATEL method. Expert Systems with Applications, 32(2), 499–507. Yu, P. L. (1973). A class of solutions for group decision problems. Management