Using a novel conjunctive MCDM approach based on DEMATEL, fuzzy ANP, and
TOPSIS as an innovation support system for Taiwanese higher education
Jui-Kuei Chen
a,1, I-Shuo Chen
b,* aGraduate Institute of Futures Studies, Tamkang University, 4F, No. 20, Lane 22, WenZhou Street, Taipei City 10616, Taiwan b
Institute of Business & Management, National Chiao Tung University, 4F, No. 20, Lane 22, WenZhou Street, Taipei City 10616, Taiwan
a r t i c l e
i n f o
Keywords:
Decision-making trial and evaluation laboratory (DEMATEL)
Fuzzy analytic network process (FANP) Technique for order preference by similarity to an ideal solution (TOPSIS)
Innovation support system (ISS)
a b s t r a c t
Increasing numbers of Taiwanese higher education institutes are pursuing innovation operation. How-ever, these institutes generally rely greatly on academic research to evaluate innovation performance. Nevertheless, the performance of innovation may be affected by numerous factors that are often beyond the scope of a single academic study. Thus, to address this concern, this paper constructs an innovation support system (ISS) for Taiwanese higher education institutes to comprehensively evaluate their innovation performance. Previous research often evaluates performance by independently considering a number of criteria. However, this assumption of independence does not model the so-called ‘‘real world”; thus, we present a novel conjunctive multiple criteria decision-making (MCDM) approach that addresses dependent relationships among each measurement criteria. As such, we utilize a decision-making trial and evaluation laboratory (DEMATEL), a fuzzy analytical network process (FANP), and a technique for order preference by similarity to an ideal solution (TOPSIS) forming order to develop an innovation support system (ISS) that considers the interdependence and the relative weights of each measurement criterion.
Ó 2009 Elsevier Ltd. All rights reserved.
1. Introduction
Due to a recent drop in the birthrate, an increase in the number of higher educational institutions, and Taiwan’s recent member-ship in the WTO, higher educational institutions in Taiwan will not have competitive advantages when faced with competitions from the West and Asia (Chen, 2005). Thus, the need to increase innovative operations, improve performance, and develop core competitive abilities is an urgent issue currently faced by higher educational institutions in Taiwan (Chen & Chen, 2008).
The most utilized evaluations used for innovation performance by Taiwanese higher educational institutions emerge from aca-demic research (Chen & Chen, 2008). However, the factors that can affect innovation performance are numerous. One way to over-come the problem of evaluation performance with regard to numerous factors involves the use of multiple criteria decision-making (MCDM), which is often characterized by multiple, con-flicting criteria (Hwang & Yoon, 1981; Liou, Tzeng, & Chang, 2007). Along these lines, various research studies have produced different measurement dimensions, and criteria (Chen & Chen, 2008; Chin & Pu, 2006; Lin, Wang, & Yen, 2006; Tang, 2006). Some
of this research assumes independence of criteria; however, in the real world, most criteria are not mutually independent.
In this paper, a decision-making trial and evaluation laboratory (DEMATEL) method is adapted to model complex interdependent relationships and construct a relation structure using measure-ment criteria for innovation evaluation. A fuzzy analytic network process (FANP) is conducted to address the problem of dependence as well as feedback among each measurement criteria. A technique for order preference by similarity to an ideal solution (TOPSIS) is finally utilized to find optimal alternatives for innovation configu-rations. Here, we combine DEMATEL, fuzzy ANP and TOPSIS ap-proaches to develop a novel innovation support system (ISS).
2. An innovation support system
Some literature has indicated that an organization must contin-ually innovate to avoid failure (Daft, 2004; Krause, 2004). Innova-tion performance evaluaInnova-tions, involve numerous complex factors, including member innovation, administrative innovation, market-ing innovation, and so on. However, an innovation criterion that follows academic research may be imperfect.
Although a large body of academic studies offers numerous in-sights involving innovation performance, evaluation tools devel-oped by these studies do not evaluate innovation performance completely. Recent studies have argued that the factors influencing
0957-4174/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2009.06.079
* Corresponding author. Tel.: +886 911393602.
E-mail addresses:chen3362@ms15.hinet.net (J.-K. Chen), ch655244@yahoo.-com.tw(I-Shuo Chen).
1 Tel.: +886 912272961.
Contents lists available atScienceDirect
Expert Systems with Applications
j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / e s w ainnovation in higher education are numerous (Bantel & Jackson, 1989; Damanpour, 1996; O’Sullivan, 2000; Wolfe, 1994). Thus, after summarizing relevant studies, we introduce a novel conjunctive MCDM approach that combines DEMATEL, Fuzzy ANP, and TOPSIS. In doing so, we consider increasingly complex relationships and uti-lize them to develop an innovation support system (ISS).
3. A novel conjunctive MCDM approach
Quantifying values with precision in a complex measurement system is difficult; nevertheless, such systems can be partitioned into separate subsystems to facilitate the evaluation of each parti-tion. Here, DEMATEL is used to develop interrelations among each measurement criterion. Next, the weights of each criterion are cal-culated using fuzzy ANP. After that, TOPSIS is utilized to rank the alternatives. Finally, we construct an innovation support system (ISS) based on these results.
3.1. Illustrating interrelations among measurement criteria
All factors in a complex system may be either directly or indi-rectly related; therefore, it is difficult for a decision maker to eval-uate a single effect from a single factor while avoiding interference from the rest of the system (Liou et al., 2007). In addition, an inter-dependent system may result in passive positioning; for example, a system with a clear hierarchical structure may give rise to linear activity with no dependence or feedback, which may cause prob-lems distinct from those found in non-hierarchical systems (Tzeng, Chiang, & Li, 2007).
To avoid such problems, the Battelle Geneva Institute created DEMATEL in order to solve difficult problems that mainly involve interactive man-model techniques as well as to measure qualita-tive and factor-linked aspects of societal problems (Gabus & Fon-tela, 1972). In addition, DEMATEL has been utilized in numerous contexts, such as industrial planning, decision-making, regional environmental assessment, and even analysis of world problems (Huang, Shyu, & Tzeng, 2007); in all cases, it has confirmed inter-dependence among criteria and restricted the relations that reflect characteristics within an essential systemic and its developmental trends (Liou et al., 2007).
The foundation of the DEMATEL method is graph theory. It al-lows decision-makers to analyze as well as solve visible problems. In doing so, decision-makers can separate multiple measurement criteria into a cause and effect group to realize causal relationships much more easily. In addition, directed graphs, called digraphs, are much more helpful than directionless graphs since they depict the directed relationships among subsystems. In other words, a di-graph represents a communication network or a domination rela-tionship among entities and their groupings (Huang et al., 2007).
The steps in DEMATEL are as follows (Liou et al., 2007):
Step 1: Calculate the initial average matrix by scores.Sampled experts are asked to point the direct effect based on their perception that each element i exerts on each other ele-ment j, as presented by aij, by utilizing a scale ranging from
0 to 4. No influence is represented by 0, while a very high influence is represented by 4. Based on groups of direct matrices from samples of experts, we can generate an average matrix A in which each element is the mean of the corresponding elements in the experts’ direct matrices.
Step 2: Calculate the initial influence matrix.After normalizing the average matrix A, the initial influence matrix D,½dijnn, is
calculated so that all principal diagonal elements equal zero. In accordance with D, the initial effect that an
ele-ment exerts and/or acquires from each other eleele-ment is given. The map depicts a contextual relationship among the elements within a complex system; each matrix entry can be seen as its strength of influence. This is depicted in
Fig. 1; an arrow from d to g represents the fact that d affects g with an influence score of 1. As a result, we can easily translate the relationship between the causes and effects of various measurement criteria into a comprehen-sible structural model of the system based on influence degree using DEMATEL.
Step 3: Develop the full direct/indirect influence matrix.The indi-rect effects of problems decreases as the powers of D increase, e.g., to D2;D3; . . . ;D1, which guarantees
conver-gent solutions to the matrix inversion. From Fig. 1, we see that the effect of c on d is greater than that of c on g. Therefore, we can generate an infinite series of both direct and indirect effects. Let the ði; jÞ element of matrix A be presented by aij, then the direct/indirect matrix can be
acquired by following Eq.(1)through(4)
D ¼ sA; s > 0 ð1Þ or ½dijnn¼ s½aijnn; s > 0; i; j 2 f1; 2; . . . ; ng ð2Þ where s ¼ Min 1 max 16i6n Pn j¼1jaijj ; 1 max 16i6n Pn i¼1jaijj 2 4 3 5 ð3Þ and lim m!1D m ¼ ½0nnwhere D ¼ ½dijnn; 0 6 dij<1: ð4Þ The total-influence matrix T can be acquired by utilizing Eq.
(5). Here, I is the identity matrix
T ¼ D þ D2
þ þ Dm¼ DðI DÞ1when m ! 1: ð5Þ
If the sum of rows and the sum of columns is represented as vector r and c, respectively, in the total influence matrix T, then T ¼ ½tij; i; j ¼ 1; 2; . . . ; n; ð6Þ r ¼ ½rin1¼ Xn j¼1 tij ! n1 ð7Þ c ¼ ½cj01n¼ Xn i¼1 tij ! 1n ð8Þ
where the superscript apostrophe denotes transposition. If rirepresents the sum of the ith row of matrix T, then
ripresents the sum of both direct and indirect effects of
factor i on all other criteria. In addition, if cj represents
the sum of the jth column of matrix T, then cj presents
the sum of both direct and indirect effects that all other factors have on j. Moreover, note that j ¼ iðriþ ciÞ
demon-strates the degree to which factor i affects or is affected by j. Note that if ðri ciÞ is positive, then factor i affects other
factors, and if it is negative, then factor i is affected by others (Liouet al., 2007; Tzeng et al., 2007).
Step 4: Set the threshold value and generate the impact relations map.
Last, we must develop a threshold value. This value is generated by taking into account the sampled experts’ opinions in order to filter minor effects presented in ma-trix T elements. This is needed to isolate the relation structure of the most relevant factors. In accordance with the matrix T, each factor tijprovides information about
how factor i affects j. In order to decrease the complexity of the impact relations-map, the decision-maker deter-mines a threshold value for the influence degree of each factor. If the influence level of an element in matrix T is higher than the threshold value, which we denote as p, then this element is included in the final impact relations map (IRM) (Liou et al., 2007).
3.2. Fuzzy ANP
3.2.1. Fuzzy set theory
Fuzzy set theory was first developed in 1965 by Zadeh; he was attempting to solve fuzzy phenomenon problems, including prob-lems with uncertain, incomplete, unspecific, or fuzzy situations. Fuz-zy set theory is more advantageous than traditional set theory when describing set concepts in human language. It allows us to address unspecific and fuzzy characteristics by using a membership function that partitions a fuzzy set into subsets of members that ‘‘incom-pletely belong to” or ‘‘incom‘‘incom-pletely do not belong to” a given subset.
3.2.2. Fuzzy numbers
We order the Universe of Discourse such that U is a collection of targets, where each target in the Universe of Discourse is called an element. Fuzzy number eA is mapped onto U such that a random x ! U is appointed a real number,
l
eAðxÞ ! ½0; 1. If another
ele-ment in U is greater than x, we call that eleele-ment under A. The universe of real numbers R is a triangular fuzzy number (TFN) eA, which means that for x 2 R;
l
eAðxÞ 2 ½0; 1, and
l
e AðxÞ ¼ ðx LÞ=ðM LÞ; L 6 x 6 M; ðU xÞ=ðU MÞ; M 6 x 6 U; 0; otherwise; 8 > < > :Note that eA ¼ ðL; M; UÞ, where L and U represent fuzzy probability between the lower and upper boundaries, respectively, as in
Fig. 2. Assume two fuzzy numbers eA1¼ ðL1;M1;U1Þ and
e A2¼ ðL2;M2;U2Þ; then, (1) eA1 eA2¼ ðL1;M1;U1Þ ðL2;M2;U2Þ ¼ ðL1þ L2;M1þ M2;U1þ U2Þ (2) eA1 eA2¼ ðL1;M1;U1Þ ðL2;M2;U2Þ ¼ ðL1L2;M1M2;U1U2Þ; Li>0; Mi>0; Ui>0 (3) eA1 eA2¼ ðL1;M1;U1Þ ðL2;M2;U2Þ ¼ ðL1 L2;M1 M2;U1 U2Þ (4) eA1 eA2¼ ðL1;M1;U1Þ ðL2;M2;U2Þ ¼ ðL1=U2;M1=M2;U1=L2Þ:Li>0; Mi>0; Ui>0 eA1 1 ¼ ðL1;M1;U1Þ1¼ ð1=U1;1=M1;1=L1Þ; Li>0; Mi>0; Ui>0
3.2.3. Fuzzy linguistic variables
The fuzzy linguistic variable is a variable that reflects different aspects of human language. Its value represents the range from natural to artificial language. When the values or meanings of a lin-guistic factor are being reflected, the resulting variable must also reflect appropriate modes of change for that linguistic factor. Moreover, variables describing a human word or sentence can be divided into numerous linguistic criteria, such as equally impor-tant, moderately imporimpor-tant, strongly imporimpor-tant, very strongly important, and extremely important, as shown inFig. 3; definitions and descriptions are shown in Table 1. For the purposes of the present study, the 5-point scale (equally important, moderately important, strongly important, very strongly important and extre-mely important) is used.
3.2.4. Analytic network process (ANP)
The purpose of the ANP approach is to solve problems involving interdependence and feedback among criteria or alternative
Fig. 3. A fuzzy membership function for linguistic variable attributes.
Table 1
Definition and membership function of fuzzy number.
Fuzzy number Linguistic variable Triangular fuzzy number ~
9 Extremely important/preferred (7,9,9) ~
7 Very strongly important/preferred (5,7,9) ~ 5 Strongly important/preferred (3,5,7) ~ 3 Moderately important/preferred (1,3,5) ~ 1 Equally important/preferred (1,1,3)
( )
Ax
µ
L
M
1
0
U
solutions. ANP is the general form of the analytic hierarchy process (AHP), which has been used in multi-criteria decision-making (MCDM) in order to consider non-hierarchical structures. MCDM has been applied to project selection, product planning, and so forth (Ong, Huang, & Tzeng, 2004).
The first phase of ANP compares the measuring criteria in the overall system to form a super matrix. This can be accomplished using pair-wise comparisons. The relative importance-values of pair-wise comparisons can be categorized from 1 to 9 in order to represent pairs of equal importance(1)to extreme inequality in importance(9)(Saaty, 1980). The following is the general form of the super matrix (Liou et al., 2007):
where cmdenotes the mth cluster, emndenotes the mth element in
the mth cluster, and Wijis the principal eigenvector of the influence
of the elements compared in the jth cluster to the ith cluster. In addition, if the jth cluster has no influence to the jth cluster, then Wij¼ 0.
Thus, the form of the super matrix relies on the variety of its structure. There are several structures that were proposed by Saaty including hierarchy, holarchy, suparchy, and so on (Ong et al., 2004). In order to demonstrate how the structure is affected by the super matrix,Ong et al. (2004)offer two simple cases that both involve three clusters to show how to form the super matrix in accordance with different structures (seeFig. 4).
Based onFig. 4, the super matrix can be formed as:
InFig. 5, a case more complex than that depicted inFig. 4is shown. Based onFig. 5, the super matrix can be formed as:
Table 2
The original innovation support system for Taiwanese higher education.
Goal Evaluating dimensions Evaluating criteria
The original innovation support system for Taiwanese higher education
Academic Research (D1) Research Patents (C1)
International Academic Interaction (C2) Number of R&D Members (C3)
Financial Support of National Science Council (C4) Journals accepted and published (C5)
Government Tender Planning (C6) Administrative process (D2) Operation Electrification (C7)
Outsourcing (C8) Affair Rotation (C9)
Faculty and Staff (D3) Information Study Camp (C10) Refresher Classes (C11)
Go Abroad for Further Education (C12) Market Development (D4) Number of Conferences (C13)
Number of International Students in School (C14) Number of Chair Professors (C15)
Organizational Structure (D5) Learning Organization (C16) Specialization Organization (C17) Matrix Organization (C18) Organizational Culture (D6) Result-Oriented (C19)
Employee-Oriented (C20) Parochial-Oriented (C21) Open-Oriented (C22) Loosely Control-Oriented (C23) Leadership Style (D7) Transformational Leadership (C24)
Transactional Leadership (C25) Fig. 4. Case 1 structure.
After forming the super matrix, the weighted super matrix is gener-ated by transforming all column sums to unity (Ong et al., 2004). Then, we use the weighted super matrix to generate a limiting super matrix by using Eq.(9)to calculate global weights.
lim
k!1W k
ð9Þ
In this step, if the super matrix shows signs of cyclicity, then there exists more than one limiting super matrix. That is, there are two or
more limiting super matrices, and the Cesaro sum must be calcu-lated to obtain the priority among these matrices. The Cesaro sum is calculated using Eq.(10).
lim k!1 1 N XN k¼1 Wk ð10Þ
Eq. (10)calculates the average effect of a limiting super matrix; otherwise, the super matrix can be raised to a large power to gen-erate the priority weights.
The steps of the fuzzy ANP calculation are provided as follow:
Step 1: Confirm both dimensions and criteria of the model. Step 2: Develop the ANP model hierarchically using the
dimen-sions, and criteria.
Step 3: Determine the local weights of both dimensions and cri-teria by utilizing pair-wise comparison matrices. Assume that there is no dependence between each. The relative importance-values of pair-wise comparisons is provided inTable 1.
Step 4: Determine the inner dependence matrix of each dimen-sion with respect to other dimendimen-sions. In Step 3, the dependence of local weights in the inner matrix was cal-culated, such that this step is intended to calculate the interdependent weights of the dimensions.
Step 5: Calculate the global weights for the sub-factors. This can be done by multiplying the local weight of each sub-fac-tor with the interdependent weights associated with dimensions where it belongs.
3.3. TOPSIS
The technique for order preference by similarity to an ideal solution (TOPSIS) was first proposed byHwang and Yoon in 1981
and expanded developments by Chen and Hwang in 1992. The foundational principle is that, in a graph, any chosen alternative should have the shortest distance from the ideal solution and the farthest distance from the negative-ideal solution (Opricovic & Tzeng, 2004).
TOPSIS is conducted as follows (Opricovic & Tzeng, 2004):
Step 1: Calculate the normalized decision matrix. The normal-ized value rijis and is calculated as:
rij¼ fij= ffiffiffiffiffiffiffiffiffiffiffiffi XJ j¼1 f2 ij v u u t ; j ¼ 1; . . . ; J; i ¼ 1; . . . ; n; ð11Þ
Step 2: Calculate the weighted normalized decision matrix. The weighted normalized value is
vij
and is calculated as:v
ij¼ wirij; j ¼ 1; . . . ; J; i ¼ 1; . . . ; n ð12Þ where wi is the weight of the ith criterion, andPn j¼1wi¼ 1.
Step 3: Determine the ideal and negative-ideal solutions using Eqs.(13) and (14).
Table 3
The average initial direct-relation 7 7 matrix A.
D1 D2 D3 D4 D5 D6 D7 D1 0 0.12 1.35 1.62 0.27 0.33 0.03 D2 1.24 0 2.33 0.57 1.13 0.06 0.71 D3 3.91 3.76 0 2.97 1.19 0.23 0.04 D4 3.29 0.24 0.26 0 0.30 1.75 1.22 D5 1.07 2.93 3.35 1.10 0 3.63 1.32 D6 3.01 1.25 2.63 2.77 1.29 0 1.10 D7 2.98 3.03 3.42 2.20 3.78 3.89 0 Table 4
Total influence matrix T.
D1 D2 D3 D4 D5 D6 D7 D1 0.05 0.04 0.09 0.11 0.03 0.04 0.01 D2 0.14 0.06 0.17 0.09 0.09 0.04 0.05 D3 0.29 0.24 0.09 0.22 0.10 0.06 0.04 D4 0.24 0.06 0.08 0.06 0.05 0.13 0.08 D5 0.22 0.26 0.29 0.18 0.08 0.25 0.11 D6 0.28 0.15 0.22 0.23 0.12 0.07 0.90 D7 0.36 0.30 0.34 0.27 0.28 0.30 0.07 Table 5
The sum of influences on measurement dimensions.
Measurement dimensions riþ ci ri ci D1 1.95 -1.21 D2 1.74 -0.46 D3 2.32 -0.24 D4 1.87 -0.47 D5 2.13 0.64 D6 2.88 1.09 D7 3.19 0.66
Fig. 6. The impact relations map of this study.
Table 6
The illustration of the local weight of criteria 13 through 15 under the effect of criterion 1.
Measurement Criteria C13 C14 C15 Local Weight
C13 1.00 1.00 3.00 1.91 3.31 4.27 0.14 0.18 0.31 0.22
C14 0.23 0.30 0.52 1.00 1.00 3.00 0.19 0.28 0.49 0.12
Table 7
The unweighted matrix of measurement criteria.
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20 C21 C22 C23 C24 C25 C1 0 0 0 0 0 0 0.13 0.1 0.1 0.06 0.12 0.08 0.1 0.11 0.08 0.06 0.03 0.06 0.08 0.1 0.05 0.04 0.1 0.03 0.06 C2 0 0 0 0 0 0 0.26 0.41 0.33 0.19 0.16 0.23 0.26 0.21 0.26 0.13 0.2 0.13 0.18 0.15 0.17 0.21 0.19 0.14 0.13 C3 0 0 0 0 0 0 0.04 0.06 0.05 0.05 0.06 0.04 0.05 0.07 0.06 0.03 0.02 0.05 0.03 0.03 0.05 0.04 0.03 0.03 0.05 C4 0 0 0 0 0 0 0.12 0.13 0.15 0.1 0.13 0.11 0.09 0.12 0.12 0.08 0.06 0.08 0.09 0.13 0.13 0.09 0.09 0.11 0.06 C5 0 0 0 0 0 0 0.21 0.18 0.24 0.14 0.15 0.18 0.19 0.19 0.21 0.12 0.16 0.12 0.15 0.13 0.14 0.19 0.16 0.07 0.06 C6 0 0 0 0 0 0 0.11 0.04 0.06 0.05 0.05 0.05 0.07 0.05 0.06 0.05 0.04 0.03 0.1 0.05 0.1 0.04 0.08 0.12 0.07 C7 0 0 0 0 0 0 0 0 0 0.003 0.006 0.008 0 0 0 0.0002 0.001 0.003 0.006 0.004 0.002 0.005 0.005 0.004 0.006 C8 0 0 0 0 0 0 0 0 0 0.002 0.004 0.001 0 0 0 0.003 0.003 0.001 0.003 0.002 0.004 0.004 0.006 0.001 0.002 C9 0 0 0 0 0 0 0 0 0 0.03 0.02 0.007 0 0 0 0.002 0.002 0.002 0.001 0.003 0.005 0.001 0.003 0.002 0.003 C10 0 0 0 0 0 0 0.06 0.02 0.02 0 0 0 0 0 0 0.002 0.003 0.001 0.001 0.005 0.001 0.003 0.001 0.003 0.002 C11 0 0 0 0 0 0 0.03 0.03 0.01 0 0 0 0 0 0 0.04 0.03 0.04 0.01 0.03 0.04 0.02 0.02 0.02 0.03 C12 0 0 0 0 0 0 0.04 0.03 0.04 0 0 0 0 0 0 0.03 0.02 0.04 0.02 0.02 0.04 0.03 0.03 0.03 0.04 C13 0.22 0.21 0.27 0.31 0.18 0.33 0 0 0 0.12 0.11 0.12 0 0 0 0.11 0.06 0.05 0.09 0.11 0.05 0.13 0.06 0.06 0.08 C14 0.12 0.03 0.14 0.16 0.12 0.12 0 0 0 0.04 0.07 0.01 0 0 0 0.03 0.02 0.04 0.06 0.03 0.05 0.04 0.03 0.02 0.06 C15 0.66 0.76 0.59 0.53 0.7 0.55 0 0 0 0.17 0.09 0.13 0 0 0 0.12 0.12 0.11 0.12 0.14 0.12 0.11 0.14 0.13 0.12 C16 0 0 0 0 0 0 0 0 0 0.04 0.02 0.03 0 0 0 0 0 0 0.03 0.03 0.002 0.03 0.03 0.04 0.04 C17 0 0 0 0 0 0 0 0 0 0.001 0.004 0.003 0 0 0 0 0 0 0.002 0.004 0.007 0.001 0.003 0.002 3E-04 C18 0 0 0 0 0 0 0 0 0 0.004 0.006 0.001 0 0 0 0 0 0 0.004 0.008 0.006 0.004 0.007 0.004 0.003 C19 0 0 0 0 0 0 0 0 0 0 0 0 0.13 0.13 0.13 0.12 0.14 0.12 0 0 0 0 0 0.12 0.09 C20 0 0 0 0 0 0 0 0 0 0 0 0 0.08 0.08 0.05 0.03 0.06 0.05 0 0 0 0 0 0.05 0.07 C21 0 0 0 0 0 0 0 0 0 0 0 0 0.006 0.008 0.006 0.001 0.003 0.001 0 0 0 0 0 0.001 0.003 C22 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0.03 0.02 0.02 0.01 0.02 0 0 0 0 0 0.01 0.02 C23 0 0 0 0 0 0 0 0 0 0 0 0 0.004 0.002 0.004 0.0008 0.006 0.001 0 0 0 0 0 0.003 7E-04 C24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0.01 0.05 0.02 0.02 0.03 0.01 0.01 0 0 C25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.001 0.002 0.001 0.003 0.004 0.003 0.002 0.005 0 0 Table 8
The weighted matrix of measurement criteria.
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20 C21 C22 C23 C24 C25 C1 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 C2 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 0.1368 C3 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 0.0306 C4 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 0.0634 C5 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 0.1099 C6 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 0.0371 C7 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 C8 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 C9 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 C10 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 C11 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 0.0020 C12 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 0.0025 C13 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 0.1107 C14 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 0.0475 C15 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 0.2997 C16 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 0.0032 C17 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 C18 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 0.0006 C19 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 0.0603 C20 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 0.0279 C21 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 0.0028 C22 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 0.0097 C23 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 0.0017 C24 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 0.0021 C25 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 0.0003 J.-K. Chen, I-Shuo Chen /Expert Systems with Applications 37 (2010) 1981–1990
A ¼
v
1; . . . ;v
n ¼ ðmax jv
ijji 2 I 0 Þ; ðmin jv
ijji 2 I 00 Þ ð13Þ A¼v
1; . . . ;v
n ¼ ðmin jv
ijji 2 I 0Þ; ðmax jv
ijji 2 I 00Þ ð14Þwhere I0is associated with benefit criteria, and I00is
asso-ciated with cost criteria.
Step 4: Calculate the separation measures using the n-dimen-sional Euclidean distance. The separation of each alterna-tive from the ideal solution is:
Dj ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Xn i¼1 ð
v
ijv
iÞ 2 v u u t ; j ¼ 1; . . . ; J ð15ÞSimilarly, the separation from the negative-ideal solution is: D j ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Xn i¼1 ð
v
ijv
iÞ 2 v u u t ; j ¼ 1; . . . ; J ð16ÞStep 5: Calculate the relative closeness to the ideal solution. The relative closeness of alternative aj with respect to Ais
defined as: Cj ¼ D j . Dj þ D j ; j ¼ 1; . . . ; J: ð17Þ
Step 6: Rank the preference order.
4. An empirical study of an innovation support system (ISS)
Owing to increasing domestic and international pressures due to joining the WTO and the drop in the birthrate, universities in Taiwan are innovating in order to gain a sustainable competitive advantage. They evaluate innovation performance mainly by draw-ing on previous academic research, and some studies address this need by specifically searching out measurement criteria for innovation performance. Most criteria have been assumed to be independent of each other; however, in the real world, they may be interdependent. In addition, when applying such research to Taiwanese higher education, we must keep in mind that higher
educational institutions fall into three categories, namely, re-search-intensive universities, teaching-intensive universities, and professional-intensive universities (Li, 2007). The focus of innova-tion improvement and evaluainnova-tion are different among each of these different types of universities. Thus, in order to construct a novel innovation support system (ISS) for Taiwan higher education, we consider the interrelationships for each criterion as well as these different types of universities.
4.1. Developing the original innovation support system
Developing an innovation support system for higher education is complicated, as it contains member, environmental, administra-tive and other factors. It is obvious that the components of an inno-vation support system should be both interdependent as well as being in accordance with real practice. Thus, twenty-five higher educational experts were consulted, including ten from research-intensive universities, seven from professional-research-intensive universi-ties, and eight from teaching-intensive universities. In addition, theNational Science Council (2008), and the R&D departments of thirty universities were consulted to construct a seven-dimen-sional innovation support system based on Academic Research (D1), Administrative Process (D2), Faculty and Staff (D3), Market Development (D4), Organizational Structure (D5), Organizational Culture (D6), and Leadership Style (D7). Each dimension has three to six measurement criteria (Table 2). A questionnaire was given to sixty-six experts, including thirty-six from research-intensive uni-versities, eleven from teaching-intensive uniuni-versities, and nineteen from professional-intensive universities. Their ranking of each measurement innovation criterion was ascertained by adapting a 5-point scale, as shown inTable 3, with respect to the innovation performance of each type of university. Experts were asked to rank their perceptions of innovative performance based on a scale rang-ing from 100 (the best) to 0 (the worst).
4.2. Evaluating the relationships among each dimension
The purpose of this paper is to determine critical innovation cri-teria and evaluate the relationships among such cricri-teria. Sixty-six educational experts were asked to indicate the relationships be-tween seven measurement dimensions. Based on an average of their opinions, we formed an initial direct-relation 7 7 matrix A by using pair-wise comparisons (seeTable 3).
Table 9
The initial value from experts for three types of universities.
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20 C21 C22 C23 C24 C25 RU 90 95 93 96 98 92 96 85 80 95 95 93 97 96 97 93 94 80 90 84 91 92 79 81 82 TU 77 86 81 86 84 84 91 84 81 91 89 90 85 89 91 91 95 79 86 86 84 83 77 86 82 PU 86 80 86 84 82 87 92 86 77 90 89 90 83 86 93 86 91 87 90 83 90 87 79 79 77 W 0.0495 0.1368 0.0306 0.0634 0.1099 0.0371 0.0006 0.0003 0.0003 0.0003 0.0020 0.0025 0.1107 0.0475 0.2997 0.0032 0.0003 0.0006 0.0603 0.0279 0.0028 0.0097 0.0017 0.0021 0.0003 Table 10
The result of universities ranking by TOPSIS.
Wed M C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20 C21 C22 C23 C24 C25 D D- Dþ D Cj Ranking RU 0.0300 0.0837 0.0189 0.0377 0.0669 0.0224 0.0004 0.00018 0.00018 0.00018 0.0012 0.0015 0.0700 0.0295 0.1778 0.0019 0.00018 0.00035 0.0356 0.0160 0.00170 0.0061 0.00101 0.00121 0.00018 0.0016 0.0187 0.0202 0.9231 1 TU 0.0275 0.0729 0.0170 0.0350 0.0603 0.0204 0.0003 0.00017 0.00017 0.00017 0.0011 0.0014 0.0584 0.0255 0.1676 0.0018 0.00017 0.00035 0.0347 0.0162 0.00162 0.0054 0.00097 0.00119 0.00017 0.0206 0.0032 0.0238 0.1357 3 PU 0.0282 0.0800 0.0171 0.0370 0.0630 0.0214 0.0003 0.00017 0.00017 0.00017 0.0012 0.0014 0.0627 0.0271 0.1735 0.0019 0.00017 0.00034 0.0341 0.0161 0.00153 0.0054 0.00097 0.00123 0.00017 0.0106 0.0086 0.0192 0.4464 2 W 0.0495 0.1368 0.0306 0.0634 0.1099 0.0371 0.0006 0.0003 0.0003 0.0003 0.002 0.0025 0.1107 0.0475 0.2997 0.0032 0.0003 0.0006 0.0603 0.0279 0.0028 0.0097 0.0017 0.0021 0.0003 Aþ 0.0303 0.0829 0.0188 0.0377 0.0678 0.0223 0.0004 0.00018 0.00018 0.00018 0.0012 0.0015 0.0696 0.0293 0.1778 0.0019 0.00018 0.00036 0.0354 0.0168 0.0017 0.006 0.00102 0.00127 0.00018 A 0.0272 0.0747 0.017 0.0346 0.0598 0.0203 0.0003 0.00017 0.00017 0.00017 0.0011 0.0014 0.0593 0.0263 0.1698 0.0017 0.00016 0.00033 0.0345 0.0155 0.00157 0.0052 0.00095 0.00116 0.00016 J.-K. Chen, I-Shuo Chen /Expert Systems with Applications 37 (2010) 1981–1990
In accordance with Eq.(1)through(3), we next generated the normalized direct-relation matrix D from A. After that, Eq.(5)is used to calculate the total influence matrix T, as show inTable 4. Finally, Eqs.(7) and (8)are utilized to calculate total influences gi-ven and received along each of these measurement dimensions; the result of these calculations is given inTable 5.
For the purposes of this paper, we use a threshold value of 0.1; we only consider influence values above this threshold, otherwise our system becomes intractably complex. We adopted a threshold value of 0.1 after consultation with educational experts. The result-ing impact relations map (IRM) is given inFig. 6.
4.3. Calculating weights of criteria in the innovation support system
In this stage, we used fuzzy ANP to calculate the weights of measurement criteria after illustrating the relationship structure of the innovation support system. At first, the relative importance of relationships among measurement criteria resembles the impact
relations map. Note again that pair-wise comparisons were con-ducted according to Table 4above. Table 6 illustrates the local weight, which is acquired using the principle eigenvector of com-parison between criterion 1 and criteria 13 through 15; and the re-sults of other relationships are addressed as an unweighted super matrix inTable 7.
From the Eq.(9), we calculated the limiting power of the un-weighted matrix until it reached stability; the results are provided inTable 8. The entries in the same row are the global weights of each measurement criterion. Finally, using above results, the im-pact-direction map that depicts the importance of each measure-ment criterion is shown inFig. 7.
4.4. Ranking alternatives in order to develop a novel innovation support system (ISS)
As mentioned before, universities in Taiwan can be categorized into three main types; namely, research-intensive universities,
Table 11
The overall result of this study.
Goal Evaluating
Dimensions
Evaluating Criteria (After considered interrelationships)
Global Weights
University Type Overall Ranking The original innovation support system for
Taiwanese higher education
Academic Research (D1)
Research Patents (C1) 0.0495 Research- Intensive University (RU)
1 International Academic Interaction
(C2)
0.1368 Number of R&D Members (C3) 0.0306 Financial Support of National Science Council (C4)
0.0634 Journals Accepted and Published (C5) 0.1099 Government Tender Planning (C6) 0.0371 Administrative
Process (D2)
Operation Electrification (C7) 0.0006
Outsourcing (C8) 0.0003
Affair Rotation (C9) 0.0003 Teaching- Intensive University
3 Faculty and Staff
(D3)
Information Study Camp (C10) 0.0003 Refresher Classes (C11) 0.0020 Go Abroad for Further Education (C12) 0.0025 Market
Development (D4)
Number of Conferences (C13) 0.1107 Number of International Students in
School (C14)
0.0475 Number of Chair Professors (C15) 0.2997 Organizational
Structure (D5)
Learning Organization (C16) 0.0032
Specialization Organization (C17) 0.0003 Professional- Intensive University (PU) 2 Matrix Organization (C18) 0.0006 Organizational Culture (D6) Result-Oriented (C19) 0.0603 Employee-Oriented (C20) 0.0279 Parochial-Oriented (C21) 0.0028 Open-Oriented (C22) 0.0097 Loosely Control-Oriented (C23) 0.0017 Leadership Style (D7) Transformational Leadership (C24) 0.0021 Transactional Leadership (C25) 0.0003 Table 12
A novel innovation support system (ISS).
IS System IS Dimension IS Criteria Optimal IS Type
A novel innovation support system (ISS) Academic Research International Academic Interaction Research-Intensive University (RU) Financial Support of NSC
Journals Accepted and Published External Academic Support Number of Conferences
Number of Chair Professors Organizational Culture Result-Oriented
teaching-intensive universities, and professional-intensive univer-sities. As a result, notions of innovation improvement as well as evaluative focuses differ among these universities. Ranking these types of universities is thus useful in determining an optimal inno-vation system for non-optimal, existing universities. This allows us to develop a benchmark for existing universities as well as to yield insights to newly built universities so that they are better equipped to make key choices early in their development. Note that we use the insights of eight of the sixty-six educational experts who either have served in all three types of universities since entering acade-mia or have served on an academic performance measurement committee.
Based on the responses of these eight educational experts, as shown inTable 9, and the global weights of measurement criteria, as shown inTable 8, we utilized TOPSIS to rank the three types of universities, which can be considered alternative solutions for our purposes here. Following the steps of TOPSIS, we generated val-ues necessary to rank these types of universities, as shown in Ta-ble 10. We also present the overall results of this study inTable 11. Following the construction of innovation measurement crite-ria, the calculation of weights, and the generation of university rankings, we finally propose a novel innovation support system in which measurement criteria are extracted from top six weights among all criteria. The reason for this is that we believe that focusing an evaluation of innovation performance on more influ-ential criteria is better than basing it on whole measurement cri-teria. In addition, the transformational leadership criterion is included due to its large influence on innovation promotion (seeTable 12).
5. Conclusions
Given a recent drop in birthrates, an increase in the number of higher educational institutions, and a new membership in the WTO, Taiwanese higher educational institutions are facing in-creased competition. As such, they have recently tried to upgrade their innovation capabilities and innovation performance by using various evaluative tools. In doing so, they mainly focus on aca-demic research. However, the factors influencing innovation in higher education are various. In accordance with the potentially numerous criteria useful in evaluating innovation performance in higher educational institutions, we have combined DEMATEL, fuzzy ANP and TOPSIS approaches to develop an innovation sup-port system (ISS) that considers the interdependence and relative weights of each measurement criterion and different types of
uni-versities. As a result, we hope that ISS will help future innovation improvements to be more practical, efficient and efficacious.
References
Bantel, K., & Jackson, S. (1989). Top management and innovations in banking: does the composition of the top team make a difference. Strategic Management Journal, 10, 107–124.
Chen, M. C. (2005). Creativity comes from living field. Journal of Shih-Yu, 456(5), 13–19.
Chen, J. K., & Chen, I. S. (2008). Select innovative index of higher educational institutions by FAHP. Journal of American Academy of Business, Cambridge, 13(1), 151–157.
Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making: Methods and applications. Berlin: Springer- Verlag.
Chin, J. M., & Pu, S. W. (2006). The concepts and implementations of school innovation management. Journal of Educational Research and Development, 2(3), 123–150.
Daft, R. L. (2004). Organization theory & design. NY: West.
Damanpour, F. (1996). Organizational complexity and innovation: Developing and testing multiple contingency models. Management Science, 42, 693–716. Gabus, A., & Fontela, E (1972). World problems, an invitation to further thought
within the framework of DEMATEL. Geneva, Switzerland: Battelle Geneva Research Center.
Huang, C. Y., Shyu, J. Z., & Tzeng, G. H. (2007). Reconfiguring the innovation policy portfolios for Taiwan’s SIP Mall industry. Technovation, 27(12), 744–765. Hwang, C. L., & Yoon, K. S. (1981). Multiple attribute decision making: Method and
applications. NY: Springer.
Krause, D. E. (2004). Influence-based leadership as a determinant of the inclination to innovative and of innovate-related behavior: An empirical investigation. The Leadership Quarterly, 15, 79–102.
Li, U. C. (2007). Enter university by 2.8 point each subject. Retrieved August 18, 2007, from<http://news.msn.com.tw/print.aspx?id=210245>.
Lin, H. F., Wang, H. L., Wang C. M., & Yen, J. F. (2006). The relationship between elementary principle leadership, continuous knowledge management and innovative operation-take three cities in northern Taiwan as example. In Education Development in Chinese Society Conference, University of Macau, China. Liou, J. H., Tzeng, G. H., & Chang, H. C. (2007). Airline safety measurement using a
hybrid model. Journal of Air Transport Management, 13, 243–249.
National Science Council (2008). Fast connection. Retrieved November 23, 2008, from<http://web1.nsc.gov.tw/mp.aspx>.
Ong, C. S., Huang, J. J., & Tzeng, G. H. (2004). Multidimensional data in multidimensional scaling using the analytic network process. Pattern Recognition Letters, 26(6), 755–767.
Opricovic, S., & Tzeng, G. H. (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156(2004), 445–455.
O’Sullivan, M. A. (2000). Contests for corporate control: Corporate governance and economic performance in the United States and Germany. NY: Oxford University Press.
Saaty, T. L. (1980). The analytic hierarchy process. NY: McGraw-Hill.
Tang, C. M. (2006). School building and campus planning. Taipei: Wu-Nan Press. Tzeng, G. H., Chiang, C. H., & Li, C. W. (2007). Evaluating intertwined effects in
e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Systems with Applications, 32, 1028–1044.
Wolfe, R. A. (1994). Organizational innovation: Review, critique and suggested research directions. Journal of Management Studies, 3(3), 405–430.
