Evaluating IT/IS investments: A fuzzy multi-criteria
decision model approach
Tzy-Yuan Chou
a,1, Seng-cho T. Chou
a,*, Gwo-Hshiung Tzeng
b,c,2aDepartment of Information Management, National Taiwan University, No. 1, Sec. 4, Roosevelt Road, Taipei 106, Taiwan bInstitute of Management of Technology, National Chiao Tung University, 1001, Ta-Hsueh Rd., Hsinchu 300, Taiwan c
Department of Business Administration, Kainan University, No. 1, Kainan Road, Luchu, Toayuan County 338, Taiwan Available online 10 August 2005
Abstract
IT/IS represents a substantial financial investment for many organizations. Whether or not to invest in new IT/IS is, therefore, a difficult decision. Simply considering human resource cost saving criteria is not enough, especially when a corporation has had many information systems. What kind of criteria should we consider when we evaluate IT/IS? How can a new IT/IS project be evaluated in an easy, cost-effective, and collective manner? We need a tool that will help decision-makers evaluate potential new investment projects. Apart from evaluating the new project per se, its com-patibility and ability to integrate with an existing IT portfolio must also be considered. Hence, we propose a new approach based on the fuzzy multi-criteria decision model (FMCDM), featuring a 2-stage evaluation process with 26 criteria for IT/IS investment. All stakeholders in a corporation can decide the relative weights they give to the criteria when they evaluate a new IT/IS project by using linguistic values. Experts can also use linguistic values to evaluate all candidates easily. Only an Excel worksheet is needed to obtain an evaluation result. It is cost-effective and efficient. We conduct a case study to show how this model can be used and discuss the results.
Ó 2005 Elsevier B.V. All rights reserved.
Keywords: Fuzzy theory; IS evaluation; IT evaluation; IT portfolio; Multi-criteria decision making
0377-2217/$ - see front matter Ó 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2005.07.003
* Corresponding author. Tel.: +886 2 33661182; fax: +886 2 23636073.
E-mail addresses:jychou@fdc.gov.tw(T.-Y. Chou),chou@ntu.edu.tw(S.-c.T. Chou),ghtzeng@cc.nctu.edu.tw(G.-H. Tzeng). 1
Tel.: +886 2 2763 1833x4372; fax: +886 2 2761 9217. 2
Tel.: +886 3 5712121x57505; fax: +886 3 5753926.
1. Introduction
Evaluating or justifying investment in IT/IS (information technology/information systems) is problem-atic (Ballantine et al., 1998; Powell, 1992; Irani, 2002). Only 16% of the companies inHochstrasser and Griffiths (1991)used rigorous methods to evaluate the benefits of IT/IS investments.
However, evaluation is important for three reasons: (1) High cost: IT/IS represents substantial financial investment for many organizations (Willcocks, 1992). Computer and telecommunications investments amount to at least half of most large organizationsÕ annual capital expenditure (Willcocks and Lester, 1997; Premkumar and Ramamurthy, 1995). The World Information Technology Services Alliance in 2000 also reported that the global information and communications industry surpassed the US$2 trillion mark in 2000 (Irani and Love, 2002). Even governments spend vast amounts of money on IT/IS. The US PresidentÕs FY 2003 budget requested US$52 billion for investments in Federal Agency information technol-ogy professionals, initiatives, and office operations (Stouffer and Rachlin, 2002). (2) Uncertainty of return:
Strassmann (1997)suggested that IT/IS investment produces negligible benefits and there is no linear rela-tionship between IT/IS expenditure and business success (Earl, 1989).Willcocks (1992)suggested that 30– 40% of projects do not realize any net business benefits and that at least 20% of IT/IS expenditure is wasted.
Hochstrasser and Griffiths (1991)also showed that over 25% of the managers in the organizations they stud-ied did not know if IT/IS investments were better or worse than non-IT/IS investments. (3) A control and management mechanism: Evaluation can provide basic feedback to managers and can form a fundamental component of the organizational learning process. This can be seen as essential for problem diagnosis, plan-ning, reduction of uncertainty, comparing projects, ranking projects in terms of organizational priorities, and as a control mechanism (Smithson and Hirschheim, 1998; Irani and Love, 2002).
Although there are uncertainties of return and vast amounts of money are required, we cannot afford not to invest (Escobar-Perez, 1998) because information systems (IS) can improve organizational efficiency and effectiveness, and provide competitive advantages (Irani, 2002; Powell, 1992). They also offer new ways to manage and organize companies and develop new business (Powell, 1992).
Hence, many kinds of evaluation methods have emerged. Many researchers have made comprehensive and useful reviews of the IS methodologies and approaches that have appeared in recent years (e.g.Powell, 1992; Farbey et al., 1993; Renkema and Berghout, 1997).
However there is increasing debate about the ability of traditional appraisal techniques to measure the ÔsofterÕ benefits of IT/IS due to the difficulty in quantifying those ÔsofterÕ elements in financial terms for deci-sion-makers (Farbey et al., 1993; Lefley and Sarkis, 1997; Irani, 2002), and to the inability to identify Ôhid-denÕ or seldom-considered costs and benefits (Ryan and Harrison, 2000) and organizational problems (Voss, 1986; Irani, 2002).
In addition, traditional appraisal techniques treat the evaluation process in isolation from its human and organizational components and place excessive emphasis on the technological and accounting/financial as-pects (Serafeimidis and Smithson, 2003). They do not consider the influence of social, political, or behav-ioral factors.
The evaluation of IS is moving away from a positivistic approach towards a more interpretive approach among theoreticians (Serafeimidis and Smithson, 2000) and exploring more concerns about social and organizational issues. For example,Mcaulay et al. (2002)focused on discussing attitudes towards the ben-efits and risks of outsourcing, as expressed by different classes of stakeholders. Meanwhile,Serafeimidis and Smithson (2000)suggested that ignoring organizational changes results in a failure to introduce new entre-preneurial evaluation procedures, andSmithson and Hirschheim (1998)highlighted the organizational and political issues that make evaluation difficult.Serafeimidis and Smithson (2003)investigated the organiza-tional roles of the key stakeholders within the particular evaluation context and provide four evaluation orientations: control, sense-making, learning and exploratory to manage the linkage between IS evaluation and organizational change (including political power).Ryan and Harrison (2000)identified a major source
of hidden costs and benefits in social subsystem when introducing new IT/IS. Mirani and Lederer (1998)
developed an instrument for assessing the organizational benefits of IS projects.Irani et al. (1998)also iden-tified a list of sources of indirect costs, including indirect human costs and indirect organization costs.
The above studies highlight the importance of considering social and organizational criteria in the eval-uation process and explain why they are important and what social and organizational criteria should be considered. However, their techniques and constructs have yet to be widely adopted by practitioners (Smithson and Hirschheim, 1998). This may be because there are too many different guidelines that confuse decision-makers, and they are not incorporated into one model for consideration.
Farbey et al. (1993)claimed that the search for a single appraisal technique that addresses all project considerations is very difficult, whileIrani and Love (2002)asserted that the development of an all-embrac-ing generic appraisal technique that takes account of the wide variety of IT/IS-related implications may be too rigid and complex for decision-makers to use.
However, it is necessary to provide the manager with an easy to use evaluation tool because, as Hochst-rasser argues, the high rate of IT/IS failure is partly attributable to a lack of solid—but easy to use—man-agement tools for evaluating, prioritizing, monitoring, and controlling IT/IS investments (Irani, 2002).
Irani (2002)also noted that: ‘‘It is expected that a robust evaluation model will help reduce the time needed to make IT investment decisions’’.
Hence, we propose a model with the following features:
1. Incorporates the opinions of every level of stakeholders in the evaluation process. 2. Integrates the criteria of organization, risk, benefit, and cost into one model.
3. Has the flexibility to allow stakeholders to change the weight of criteria depending on the specific char-acteristics of the new IT/IS.
4. Is suitable for every organization and sensitive to its culture, because the relative weights of the criteria are determined by the stakeholders.
5. Combines quantitative and qualitative decision-making criteria into one model. 6. Is easy and saves time.
The remainder of this paper is organized as follows. Section2presents a review of the literature on IT/IS evaluation criteria, stakeholders, and limitations of AHP (analytical hierarchy process) method. Our pro-posed approach is then detailed in Section3. The use of the proposed method is further illustrated with a case study in Section4. Finally, in Section5, we present our conclusions.
2. Literature review
In this section, we review the IT/IS evaluation criteria, describe the evaluation stakeholders, and discuss the limitations of traditional AHP.
2.1. IT/IS evaluation criteria
The criteria for evaluating IT/IS investments are important because they obviously influence the selec-tion result. Many researches have investigated IT/IS evaluaselec-tion criteria. Bacon (1992)andEscobar-Perez (1998)used the same 15 criteria to investigate the usage of evaluation criteria in American, British, Austra-lian, New Zealand and Spanish firms.
In the evaluation of electronic data interchange (EDI) adoption and integration,Jones and Beatty (1998)
summarized prior studiesÕ measures to identify 16 criteria to evaluate EDI systems and use LISREL (LInear Structural RELations) models to find out a better evaluation set with 13 criteria.
Irani (2002)used a manufacturing company as a case study to classify the MRPII (Manufacturing Re-source Planning II) systemÕs benefits as strategic, tactical, and operational benefits. The criteria are finan-cial, non-finanfinan-cial, and/or intangible.Irani et al. (1998)also proposed direct costs and some indirect costs, including two taxonomies: indirect human costs and indirect organizational costs.
Mirani and Lederer (1998)used three LISREL models to determine the generic criteria for measuring the organizational benefits of IS. The criteria include improving competitive advantage, alignment, cus-tomer relations, information access, information quality, information flexibility, communication efficiency, system development efficiency and business efficiency.
Information systems are socio-technical entities inseparable from the organizational context within which they are situated and interact, as well as being products of history and human agency (Boland and Hirsch-heim, 1987; Orlikowski, 1992; Walsham, 1993).Serafeimidis and Smithson (2000)thought that the Ôorgani-zational changeÕ criteria should be considered; otherwise, the project failure rate would increase.Ryan and Harrison (2000)also identified 11 social subsystem cost and benefit criteria that should be considered and emphasized their importance. AppendixApresents a summary of the criteria suggested in the above studies. We condensed AppendixAinto a set of 31 criteria. In order to understand whether, or not, an organi-zationÕs experience with IT/IS and its Chief Executive OfficerÕs (CEO) preferences influence IT/IS evalua-tion, we added five new criteria to consider. They are: Whether the CEO has IT knowledge, Whether the CEO encourages innovation, Experience in using IS, Failure experience with IS and Maturity of IT. The final worksheet is shown inTable 1.
2.2. IT/IS evaluation stakeholders
For the evaluation process and result to be successful, it is not only important to understand the criteria, but the stakeholders who do the evaluation must also be considered because different constituencies have different perspectives, all of which may be equally important (Mirani and Lederer, 1998). For instance, users, IS professionals and managers may differ about the nature of benefits and costs associated with an IS development project. Also, a projectÕs impact may differ according to which group stakeholders be-long to (Mcaulay et al., 2002). When different stakeholder groups hold conflicting views, evaluation may become a highly political activity (Walsham, 1993; Smithson and Hirschheim, 1998). It is not surprising, therefore, thatIrani (2002) identified ‘‘stakeholdersÕ definitions, analyses, and involvement’’ as important concerns during the IT/IS evaluation process. Therefore, the evaluation team should comprise different stakeholders who will be impacted by the IT/IS.
Mcaulay et al. (2002)classified the stakeholders into three groups: IS management stakeholders, non-IS management stakeholders, and IS-transitioned employee stakeholders.Irani (2002)classified stakeholders into three groups: strategic stakeholders, operational stakeholders, and financial justification stakeholders. In order to collect the opinions of all stakeholders in the organization, we included IS management stake-holders, non-IS management stakestake-holders, strategic stakestake-holders, and financial justification stakeholders in our weighting team.
2.3. Limitations of AHP
Multi-criteria methods are used in many decision-making situations, and are well known for their ability to combine quantitative and qualitative decision-making criteria (Renkema, 2000). Several authors have used the AHP, proposed bySaaty (1980), to evaluate alternate technologies. The method combines all cri-teria in one model and allows the evaluating company to decide the weight of each criterion. Therefore, the AHP method can be used to evaluate the relative importance weight of criteria/alternative IT projects.
However, the application of traditional AHP to evaluate alternative IT/IS projects has some limitations (Prabhu and Vizayakumar, 2001). These include:
Table 1
Criteria worksheet
Criteria Source
1. Ally with partner (Improved trading partner relations) Jones and Beatty (1998), Irani (2002), Mirani and Lederer (1998), Iacovou et al. (1995)
2. Government requirement Escobar-Perez (1998), Bacon (1992)
3. React to competition Escobar-Perez (1998), Bacon (1992), Mirani
and Lederer (1998)
4. Organization learning Escobar-Perez (1998), Bacon (1992), Ryan
and Harrison (2000)
5. Technical/system requirements (UserÕs requirement) Escobar-Perez (1998), Bacon (1992)
6. Whether the CEO has IT knowledge Authors proposed
7. Whether the CEO encourages innovation Authors proposed
8. Experience in using IS Authors proposed
9. Failure experience with IS Authors proposed
10. Compatibility with existing IT/IS portfolio
(Required changes to existing systems/whether duplicate)
Jones and Beatty (1998)
11. Probability of completion Escobar-Perez (1998), Bacon (1992)
12. Manpower (Morale/staff resource/Employee motivation/Staff turnover)
Ryan and Harrison (2000), Irani (2002), Mirani and Lederer (1998)
13. Skill of IT staff (Management/staff resource/ Competence of project leader)
Irani (2002), Iacovou et al. (1995)
14. Maturity of IT Authors proposed
15. Probability of benefit achievement Escobar-Perez (1998), Bacon (1992)
16. Hardware cost Jones and Beatty (1998), Iacovou et al. (1995),
OÕCallaghan et al. (1992)
17. Software cost Jones and Beatty (1998), Mirani and Lederer
(1998), OÕCallaghan et al. (1992) 18. Implementation cost (Decreased productivity at first/disrupted
work at first/Installation cost/Change management/loss of control)
Jones and Beatty (1998), Ryan and Harrison (2000), Iacovou et al. (1995),
OÕCallaghan et al. (1992) 19. Maintenance cost (Management time/the ability to perform
maintenance faster/Ongoing expenses)
Irani (2002), Mirani and Lederer (1998), Iacovou et al. (1995), OÕCallaghan et al. (1992),
Ryan and Harrison (2000)
20. Consultant cost (Increased computer support needs/learning curve and training time)
Jones and Beatty (1998), Irani (2002), Iacovou et al. (1995)
21. Assist to achieve corporation goal (Support explicit business objectives)
Escobar-Perez (1998), Bacon (1992), Mirani and Lederer (1998)
22. Assist planning and control (Support implicit business objectives/data management)
Escobar-Perez (1998), Bacon (1992), Irani (2002), Mirani and Lederer (1998), Banerjee and Golhar (1994)
23. Assist to make a management decision Escobar-Perez (1998), Bacon (1992), Ryan and Harrison (2000), Irani (2002), Mirani and Lederer (1998)
24. Improve competitive advantage (improve customer
satisfaction/reduced inventory/Improved growth and success)
Jones and Beatty (1998), Irani (2002), Mirani and Lederer (1998), Banerjee and Golhar (1994), Iacovou et al. (1995) 25. Improve corporate image (leader in new technology/enhance
the credibility and prestige of the organization/ Improved customer loyalty)
Irani (2002), Orli and Tom, 1987, Banerjee and Golhar (1994)
26. Reduce or avoid operation cost Jones and Beatty (1998), Irani (2002), Mirani and Lederer (1998), Banerjee and Golhar (1994), Iacovou et al. (1995), OÕCallaghan et al. (1992)
(1) AHP uses crisp values to score alternatives. The assessment of alternative projects requires the judge-ment of several experts. Every expertÕs feeling or perception about a score of 80 is not the same, so it is hard and unrealistic to let an expert express his/her own opinion in a crisp value. It would be easier for experts to give their opinions in linguistic variables such as high, medium, or low because a range value can express opinions and feelings more accurately.
(2) Mirani and Lederer (1998)suggested that the evaluation of IS effectiveness is subjective and relative by nature. Hence, it would be unrealistic to assign a crisp value to a subjective judgement, especially when the data is imprecise or fuzzy. Such data might be an estimating error, such as an underesti-mated cost or an indirect cost. Giving a range value to the judgement would be more appropriate. (3) Some criteria in the decision-making process are qualitative, so there cannot be a crisp value for them. (4) AHP only prioritizes the relative importance of all alternative projects and does not indicate the
degree of difference or overlap between the projects in each criterion.
There are many uncertainties in the evaluation of IT investment before it is implemented, so it is difficult, and unwise, to give a crisp value to each criterion. Therefore, we introduce a fuzzy concept to use a range, or a relative degree to represent the value of each criterion in our model. The proposed fuzzy multi-criteria decision model (FMCDM), which uses linguistic variables and range values, not only overcomes the short-comings of AHP, but also enables a decision-maker to determine where and how to improve the final pro-ject, and the benefits from it.
Furthermore, evaluation of a potential IT/IS project must also take the existing IT portfolio into ac-count. This allows us to review the operation flow and information flow of an organization, avoid building redundant systems, and reduce waste as much as possible. It is very important to consider how to integrate the new system into the existing IT portfolio in order to maximize the benefits from the new IT portfolio.
3. FMCDM based procedure for evaluating IT/IS investment
The proposed FMCDM comprises a 2-stage evaluation process with a set of criteria for evaluating IT/IS investment. We briefly introduce the steps of FMCDM in Section3.1. The selection of evaluation criteria is discussed in Section 3.2, followed by a description of the two evaluation teams and their functions in Section3.3.
Table 1 (continued)
Criteria Source
28. Improve user satisfaction (Improved service/Job satisfaction/Work quality) Jones and Beatty (1998), Kaplan (1986), Ryan and Harrison (2000)
29. System flexibility (Improved flexibility/ Improved response to changes)
Irani (2002), Mirani and Lederer (1998)
30. Improve utilization of equipment (Productivity/ Increased plant efficiency/Increased throughput)
Ryan and Harrison (2000), Irani (2002)
31. Improved cash flow Jones and Beatty (1998), Iacovou et al. (1995)
32. Changed operating procedure (Business process reengineering/organizational re-structure)
Jones and Beatty (1998), Irani (2002), Premkumar and Ramamurthy (1995), OÕCallaghan et al. (1992)
33. Improvement in communication (Improved organizational teamwork)
Ryan and Harrison (2000), Irani (2002), Mirani and Lederer (1998),
Banerjee and Golhar (1994);
34. Promotes concept of open culture Irani (2002)
35. Formalized procedures with accountability and responsibility Irani (2002)
3.1. The steps of FMCDM
We follow Prabhu and VizayakumarÕs (2001) evaluation model and modify it for IT/IS evaluation. There are 10 steps in our proposed FMCDM approach. The steps are briefly introduced below and ex-plained in detail in AppendixB.
Step 1: Form two evaluation teams: a weighting team and a scoring team, as described in Section3.3. Step 2: Determine the evaluation criteria. As explained in Section 3.2, given a set of criteria such as
the one shown inTable 1, this step determines which criteria are to be used by the evaluation teams.
Step 3: Identify potential alternative projects and combine them with the existing IT portfolio to form the set of alternative candidates.
Step 4: Set proper linguistic scales (e.g., very high, high, medium, low, and very low) and ask the members of the weighting team to give their opinions on the relative importance of the evaluation criteria by pair-wise comparison.
Step 5: Convert the linguistic variables into triangle fuzzy numbers to get every memberÕs fuzzy reciprocal matrix.
Step 6: Aggregate the weighting team membersÕ fuzzy reciprocal matrices by geometric means and form the final aggregated fuzzy reciprocal matrix.
Step 7: Take the geometric row means of every criterion and normalize it to get its local weight. Then, calculate the global importance weight.
Step 8: Decide the rules for translating linguistic variables of score to a triangle fuzzy scale. As explained in AppendixB, evaluation criteria are divided into groups, and different translation formulas are applied according to the nature of the criteria.
Step 9: Aggregate all the scoring team membersÕ opinions for each criterion and calculate the final score of all the alternative projects. Every alternative project has a score under each criterion. We mul-tiply each score with the corresponding weight of the criterion and sum up the result to get the final score of the alternative.
Step 10: Translate each final triangle fuzzy score of the alternative projects into a crisp value for ranking purposes. There are many de-fuzzification formulas for translating fuzzy values to crisp values. We adoptOpricovic and TzengÕs (2003) formula for its proven superior performance.
3.2. Selection of evaluation criteria
The worksheet shown inTable 1lists the criteria obtained from our review of IT/IS evaluation criteria in the literature. A Delphi approach was used to select which criteria would be used by the evaluation teams. Eight experts were chosen for this purpose. They helped prioritize the criteria and reached a consensus about the important criteria for IT/IS evaluation.
The experts were chosen according to their experience with IT/IS evaluation and formed a group with half of the IT providers and half of the IT customers.Table 2shows the expertsÕ personal profile and the hierarchy of the criteria they chose is shown in Fig. 1.
The experts decided that the organizationÕs previous experience with IT/IS would not influence their evaluation of a new IT/IS project. They only remembered the lessons learned from any failures they may have had. They also agreed that the preferences or encouragement of the CEO were not important criteria in the evaluation process. The project would not be chosen simply to please the CEO. As a result, the total number of evaluation criteria was reduced to 26 from 36 listed in
Table 2
ExpertsÕ personal profile
Number Gender Position Department Industry Age Experience (years)
Academic degree
IT skill IT/IS experience
1 Male Manager IT Public servant
in military
>50 Over 20 Master Sophisticated Much
2 Female Manager IT Bank >50 Over 20 Bachelor Normal Much
3 Male Manager IT Public servant
in military
46–50 Over 20 Bachelor Sophisticated Much
4 Male Manager IT Media 40–45 Over 20 Master Sophisticated Very much
5 Male Staff Other Telecommunication >50 Over 20 Master Normal Much
6 Male Manager Other IT Provider 40–45 16–20 Master Sophisticated Much
7 Male Manager IT IT Provider 46–50 16–20 Master Very
Sophisticated
Very much
8 Male Manager IT IT Provider 40–45 16–20 Master Sophisticated Very much
Hardware costs Software costs Implementation costs (including change of management) Maintenance costs Consultant costs (including training time and learning curve)
Risk criteria Cost criteria
External criteria Internal criteria Benefit criteria
Ally with partner Commit to government requirement Assist to achieve corporation goal Assist to make a management decision Assist planning and control Improve competitive advantage Reduce or avoid operation Improve information quality Improve user’s satisfaction System flexibility Organizational re-structure Organization learning Compatibility or integration ability with existing IT/IS portfolio User’s requirements Probability of completion Probability of benefit achievement Skill of IT staff Manpower Maturity of new technology Security protection React to or compete with other competitions
IT/IS Investment Criteria
3.3. Evaluation teams
Our proposed FMCDM approach calls for two teams, namely, a weighting team and a scoring team, instead of a single evaluation team used in other studies (King and Schrems, 1978; Metaxiotis et al., 1999). As mentioned earlier, the opinions of the stakeholders are important to the IT/IS evaluation process. The weighting team comprised different stakeholders, including an executive manager, the IT department manager, one or two IT staff, the accounting department manager, a couple of users whose departments or jobs would be affected by the new IT/IS, and the manager who proposed the new IT/IS project. Because user resistance is one of the main reasons for IT/IS project failures, we proposed the inclusion of more than one user in the team so users would know their opinions were valued and, they would share responsibility for making the project a success.
The mission of the weighting team is to give relative weights to all evaluation criteria. Team members only need to use linguistic variables to express their opinions, which is easier than giving an exact crisp number. The weighting result, representing the opinions of all stakeholders, can reflect the culture of orga-nization, the characteristics of the IT/IS project, and its suitability for the companyÕs requirements. Because different IT/IS users have their own special evaluation concerns, every new evaluation team may change the weighting of the criteria to reflect those concerns. This provides the organization with more flexibility.
The mission of the scoring team, on the other hand, is to evaluate alternatives to the existing IT/IS port-folio based on the criteria selected. The team consists of IT experts, accounting experts, an executive man-ager, an auditor and an outside consultant. The reasons for having so many different experts in the team are: (1) by including the executive manager we can ensure that the new project will not conflict with stra-tegic business objectives; (2) including the IT experts ensures that the project is feasible with regard to its planning, design, methodology, technology, and architecture; (3) including the accounting experts ensures that the cost and benefit criteria are evaluated and that the project finally adopted will be financially fea-sible; (4) the auditors will understand the purpose of the project more precisely and assist with its subse-quent implementation and monitor; and (5) external consultants can offer objective opinions.
Staff opinions may be different from those of managers because the staff may not be aware of the com-panyÕs strategic goals. In this regard, the inclusion of the CEO in the scoring team plays a key role because he/she is supposed to fully understand the impact of all investment alternatives. Therefore, the clearer the decision process the better the final decision, as the CEO makes that decision.
Every member of the scoring team uses linguistic variables to score only those criteria he knows about according to his expertise. Through this exercise, the decision-maker can obtain a collective, unambiguous, and synthesized final suggestion easily. When the alternatives are evaluated, the new project is combined with the existing IT portfolio to gain the maximum benefit from the new investment.
4. Case study: Selection of IT/IS investment using the FMCDM and IT portfolio approach for the Lion Travel Service Corporation of Taiwan
To determine how the model could be used, we conducted a case study of the Lion Travel Service Cor-poration of Taiwan. The steps shown above were applied in the case study.
4.1. Background of the Lion Travel Service Corporation
The Lion Travel Service Corporation has the biggest share of the travel agency market in Taiwan. It has a staff of 600, of which 30 are IT staff. The company is fully computerized and there are already many infor-mation systems in the working environment, including a web site with all kinds of inforinfor-mation about per-sonal and group travel where customers can make an order on-line. The current human resource
management system only provides basic management functions. These are not sufficient and the human re-source department staff must sometimes work overtime to provide reports that the manager requests. The company is considering whether, or not, to install a new human resource IS.
4.2. Fuzzy MCDM for human resource IS evaluation
We introduced our approach to the CEO of the Lion Travel Service Corporation who showed interest in our model and agreed to use it for evaluation. He asked one of the vice presidents to help us form the weighting and scoring teams. After a long interview with the vice president, we were sure that he under-stood what the model meant and we knew what the requirements of the new human resource IS were. Then, we identified the people who would be involved in the project and formed the two teams. There were eight persons in the weighting team: the vice president, who would make the final decision about the new system, one IT manager, one IT staff member, one accounting manager, one human resources manager and three users. There were seven experts in scoring team: one IT manager, one IT staff member, one accounting manager, one accounting staff member, the vice president, who would make the final decision about the new system, one auditor, and an external consultant. The personal profile of the members of the two teams is shown inTable 3.
The manager of the human resources department was nominated as the project leader. She told us that they had a number of choices: (i) develop a new IS by themselves, (ii) buy a new IS package, (iii) buy a package and customize it by themselves, (iv) buy a package and have it customized by the supplier, or (v) continue to use the existing IS. Eight human resource IS package providers submitted their proposals to her. None of them sold IS source codes, or provided customized service. Five of the proposals were re-jected because their functions were obviously not suitable. Therefore, they were left with five alternatives to consider: (i) do not have a new IS, (ii) develop a new IS by themselves, (iii) buy a package from provider A, (iv) buy a package from provider B, or (v) buy a package from provider C.
Before evaluation, we held a meeting with all team members to make sure they knew what the model meant and how to evaluate it. Then, every member of the weighting team gave his/her opinions on the
Table 3
Personal profile of the two teams
Number Gender Position Department Age Experience (years)
Academic degree IT skill IT/IS experience The weighting team
1 Male Vice President 40–45 16–20 Master Normal Normal
2 Female Manager Human Resource 36–40 16–20 Bachelor Normal Much
3 Male Manager IT 40–45 16–20 Bachelor Sophisticated Very much
4 Female Manager Accounting 40–45 16–20 Master Normal Much
5 Male Staff IT 30–35 6–10 Master Sophisticated Very much
6 Female Staff Human resource 26–30 6–10 Bachelor Normal Much
7 Female Staff Human resource 46–50 Over 20 High school Normal Normal
8 Male Staff Human resource 40–45 16–20 Bachelor Normal Much
The scoring team
1 Male Vice President 40–45 16–20 Master Normal Normal
2 Female Manager Accounting 40–45 16–20 Master Normal Much
3 Male Manager IT 40–45 16–20 Bachelor Sophisticated Very much
4 Female Staff Accounting 30–35 20–25 Bachelor Normal Much
5 Male Staff IT 30–35 6–10 Master Sophisticated Very much
6 Male Staff Auditor 40–45 16–20 Bachelor Normal Much
relative importance of the criteria. All the membersÕ opinions were aggregated according to Steps 6 and 7 to obtain the geometric row mean values and importance weights of criteria. Those values of Level 1 criteria are shown in Table 4. The global weights of all criteria are shown inTable 5.
For each alternative project, experts on the scoring team gave a score to each criterion in which they had expertise. They also had to use a linguistic variable to score each alternative.
Seven of 26 criteria were cost criteria. They were ÔManpowerÕ, ÔHardware costsÕ, ÔSoftware costsÕ, ÔImple-mentation costsÕ, ÔMaintenance costsÕ, ÔConsultant costsÕ, and ÔOrganizational re-structureÕ. We used the fol-lowing formula (explained in Step 8) to translate linguistic variables into a normalized score between 0 and 100. rij¼ x j xij x j xj 100. Table 4
Geometric row mean values and the importance weights of Level 1 criteria
Geometric row mean values Importance weights External criteria (0.30758, 0.33434, 0.37449) (0.04379, 0.05391, 0.06937) Internal criteria (2.21914, 2.58007, 2.88831) (0.31592, 0.41603, 0.53499) Risk criteria (0.72964, 0.83392, 0.95777) (0.10387, 0.13447, 0.17740) Cost criteria (0.79595, 0.88818, 1.00591) (0.11331, 0.14322, 0.18632) Benefit criteria (1.34653, 1.56512, 1.79781) (0.19170, 0.25237, 0.33300) Table 5
Global weights of all criteria
Criteria Weight
Ally with partner (improve trading partner relations) (0.01103, 0.01710, 0.02816)
Commitment to government requirements (0.01168, 0.01886, 0.03130)
React to or compete with competitors (0.01103, 0.01796, 0.03056)
Organization learning (0.04538, 0.08077, 0.14186)
UsersÕ requirements (0.08148, 0.15065, 0.26686)
Compatibility or integration ability with existing IT/IS portfolio (0.07285, 0.13623, 0.24652)
Organizational re-structure (0.02718, 0.04838, 0.08969)
Probability of completion (0.03969, 0.05389, 0.07502)
Manpower (0.02549, 0.04127, 0.06690)
Skill of IT staff (0.00551, 0.00908, 0.01631)
Maturity of new technology (0.00222, 0.00353, 0.00641)
Probability of benefit achievement (0.05905, 0.08058, 0.11161)
Hardware costs (0.00659, 0.01101, 0.02008)
Software costs (0.01255, 0.02165, 0.03903)
Implementation costs (0.02526, 0.04190, 0.06975)
Maintenance costs (0.02961, 0.04842, 0.08108)
Consultants costs (0.01125, 0.02023, 0.03770)
Assist to achieve corporation goal (0.01079, 0.01932, 0.03465)
Assist planning and control (0.02138, 0.03957, 0.07191)
Assist in making a management decision (0.02324, 0.04177, 0.07463)
Improve competitive advantage (0.00999, 0.01809, 0.03334)
Reduce or avoid operation costs (0.02201, 0.04166, 0.07767)
Improve information quality (0.01594, 0.02950, 0.05554)
Improve user satisfaction (0.01293, 0.02485, 0.04831)
System flexibility (0.01151, 0.02083, 0.03797)
For other criteria we used the triangle fuzzy number conversion shown in AppendixB(Table 11) to trans-late linguistic variables to a score.
After translating all expertsÕ linguistic variables to a score we averaged the score of all experts on each criterion for each alternative. Then, we multiplied each criterion by its corresponding global weight and summed up all values to get a final fuzzy score of each alternative. The results of the final fuzzy score of each criterion for all alternatives are shown in Table 6and the final fuzzy net scores of all alternatives are shown inTable 7.
Finally, we used the formula discussed in Step 10 to get the crisp score value of all alternatives. The final crisp scores of all alternatives are shown inTable 8. Provider A is the recommended choice in this case study.
4.3. Discussions
How long does it take to carry out such an evaluation? From the formation of the two teams to the com-pletion of the evaluation by both teams took only one day. The collected opinions were input to our model and the recommendation was known immediately, so our approach does not require much time.
Table 6
Final fussy score of each criterion for all alternatives
DonÕt invest In-house dev. Provider A Provider B Provider C Ally with partner (0.24, 0.46, 1.36) (0.24, 0.46, 1.36) (0, 0, 0.70) (0, 0, 0.70) (0, 0, 0.70) Government requirement (0.25, 0.50, 1.51) (0.25, 0.50, 1.51) (0.25, 0.50, 1.51) (0.25, 0.50, 1.51) (0.25, 0.50, 1.51) React competitions (0.15, 0.36, 1.27) (0.44, 1.08, 2.29) (0.70, 1.44, 2.75) (0.20, 0.54, 1.58) (0.20, 0.54, 1.58) Organization learning (0.78, 2.01, 7.05) (1.77, 4.68, 11.57) (4.19, 8.69, 16.36) (1.03, 3.01, 8.75) (1.29, 4.01, 10.44) UsersÕrequirements (0.41, 1.63, 9.50) (4.35, 10.32, 20.80) (7.41, 15.21, 26.68) (2.87, 7.60, 18.54) (3.72, 9.23, 20.35) Compatibility or ability to integrate with existing IT/IS portfolio
(1.12, 3.06, 11.31) (5.90, 13.25, 24.87) (2.95, 7.14, 17.64) (2.25, 6.12, 15.83) (2.25, 6.12, 15.83) Organizational re-structure (0, 0, 0.68) (0, 0, 0.68) (0, 0, 0.68) (0, 0, 0.68) (0, 0, 0.68) Probability of completion (0.53, 1.08, 3.13) (2.18, 3.95, 6.25) (3.11, 5.03, 7.38) (0.70, 1.62, 3.88) (0.87, 2.16, 4.63) Manpower (2.15, 3.72, 6.60) (0, 0, 0) (0.95, 2.06, 4.82) (0.95, 2.06, 4.82) (0.95, 2.06, 4.82) Skill of IT staff (0.23, 0.48, 1.09) (0.25, 0.58, 1.25) (0.30, 0.67, 1.36) (0.27, 0.61, 1.33) (0.27, 0.61, 1.33) Maturity of IT (0.07, 0.15, 0.39) (0.10, 0.224, 0.49) (0.12, 0.25, 0.53) (0.12, 0.26, 0.53) (0.07, 0.18, 0.44) Probability of benefit achievement (1.28, 2.15, 5.39) (1.30, 3.22, 6.88) (4.23, 6.98, 10.79) (1.83, 4.03, 7.63) (1.83, 4.03, 7.63) Hardware costs (0.56, 0.99, 1.98) (0.18, 0.40, 1.04) (0.30, 0.66, 1.69) (0.25, 0.55, 1.45) (0.25, 0.55, 1.45) Software costs (0.86, 1.59, 2.95) (0.15, 0.43, 1.13) (0.12, 0.29, 0.78) (0.03, 0.14, 0.35) (0.05, 0.29, 0.69) Implementation costs (1.73, 3.07, 5.27) (0.30, 0.84, 2.02) (0.74, 1.82, 4.25) (0.30, 0.84, 2.02) (0.35, 1.12, 2.64) Maintenance costs (2.26, 3.87, 7.03) (0.58, 1.45, 3.32) (0.87, 2.10, 4.94) (0.41, 1.29, 3.06) (0.87, 2.10, 4.94) Consultants costs (0.86, 1.62, 3.27) (0.95, 1.82, 3.72) (0.60, 1.21, 2.77) (0.49, 0.94, 2.01) (0.33, 0.88, 2.30) Assist to achieve corporation goal (0.23, 0.50, 1.58) (0.52, 1.24, 2.67) (0.84, 1.74, 3.21) (0.42, 1.06, 2.45) (0.42, 1.06, 2.45) Assist planning and control (0.49, 1.08, 3.44) (1.44, 3.23, 6.40) (1.63, 3.49, 6.88) (0.70, 2.02, 4.86) (0.89, 2.28, 5.33) Assist in making a
management decision
(0.54, 1.16, 3.70) (0.98, 2.47, 5.74) (1.79, 3.78, 7.39) (0.76, 2.04, 5.23) (0.76, 2.04, 5.23) Improve competitive advantage (0.22, 0.48, 1.58) (0.58, 1.32, 2.89) (0.73, 1.56, 3.16) (0.31, 0.84, 2.23) (0.31, 0.84, 2.23) Reduce or avoid operation costs (0.59, 1.35, 4.46) (1.48, 3.55, 7.69) (1.55, 3.72, 8.15) (1.07, 2.87, 6.92) (1.07, 2.87, 6.92) Improve information quality (0.37, 0.84, 2.89) (0.82, 2.11, 4.88) (1.22, 2.74, 5.78) (0.52, 1.48, 4.09) (0.81, 2.00, 4.88) Improve user satisfaction (0.30, 0.68, 2.35) (0.55, 1.54, 3.64) (0.99, 2.22, 4.69) (0.18, 0.77, 2.67) (0.30, 1.02, 2.99) System flexibility (0.29, 0.64, 2.06) (0.62, 1.51, 3.27) (0.85, 1.91, 3.84) (0.30, 0.96, 2.63) (0.42, 1.19, 2.92) Security protection (0, 0, 0.22) (0, 0, 0.22) (0, 0, 0.22) (0, 0, 0.22) (0, 0, 0.22)
Taking a closer look at the evaluation results, we found that the opinions of the user departmentÕs staff and the user departmentÕs manager in the weighting team were: Ôorganization learningÕ was the most impor-tant criterion, Ôusers requirementsÕ came second, and the third was Ôprobability of completionÕ. In contrast, the IT departmentÕs staff and the IT departmentÕs manager thought that the ÔCompatibility or ability to inte-grate with the existing IT/IS portfolioÕ was the most important criterion, followed by ÔManpowerÕ, and the ÔProbability of benefit achievementÕ. The results show that the users and IT staff have different priorities.
Our interview with the users revealed that they had had little chance to learn new knowledge about human resource management. If they had a new system, they could learn from it. For this reason, they val-ued ÔOrganization learningÕ the highest. The ÔUsersÕ requirementsÕ criterion showed their dissatisfaction with the existing IS; they often needed to work overtime to compensate for the deficiencies of the existing system. Their strong desire for a new system was reflected by the weight they placed on ÔProbability of com-pletionÕ. Although human resources are important assets to the organization, there is high staff turnover. Users complained about poor training opportunities and unclear goals in career planning. If the new system incorporated functions to help staff with career planning and training, staff turnover rate might be lower and indirectly benefit the organization.
Members of the IT department valued ÔCompatibility or the ability to integrate with the existing IT/IS portfolioÕ and ÔManpowerÕ much more than users because they have a shortage of IT manpower. Most IT manpower is dedicated to developing core travel agency systems, due to the absence of travel IS providers. As the travel agency systems are the companyÕs core business, these stakeholders paid more attention to them. But, since the human resource management system is a supportive system, they preferred to buy it from an IS provider. However, they were concerned about the effort needed to modify and maintain a pur-chased system.
Another reason that IT staff members paid more attention to Ôthe compatibility or ability to integrate with the existing IT/IS portfolio was that they were aware of their responsibility for maintaining the new IT portfolio and ensuring there would be no duplicated systems in the new portfolio.
The overall evaluation results suggest that Ôprovider AÕ, having the highest score, should be the recom-mended alternative. In fact, ÔProvider AÕ is highly valued in many criteria, except for the ÔCompatibility or ability to integrate with existing IT/IS portfolioÕ score, which is lower than the alternative Ôin-house devel-opÕ; ÔHardware costsÕ, ÔSoftware costsÕ, ÔImplementation costsÕ, and ÔMaintenance costsÕ scores are lower than the alternative ÔDonÕt invest in itÕ; the ÔConsultants costsÕ score is lower than the alternative ÔDonÕt in-vest in itÕ and Ôin-house developÕ. These results are understandable. It is unavoidable that ÔHardwareÕ, ÔSoft-wareÕ, ÔImplementationÕ and ÔMaintenanceÕ costs are higher than Ônot to investÕ. But, we could ask Provider A to include a free-education service to lower the consultancy fee, if we decide to buy that alternative. We could also ask the engineers of Provider A to improve the data format layout or transference software to increase the ÔCompatibility or the ability to integrate with the existing IT/IS portfolioÕ.
Table 7
Final fuzzy net scores of all alternatives
DonÕt invest In-house dev. Provider A Provider B Provider C
Sum (15.87, 32.17, 88.64) (23.52, 55.92, 120.30) (33.28, 70.28, 142.25) (14.86, 39.47, 101.22) (16.96, 44.62, 109.99)
Table 8
Crisp net scores of all alternatives
DonÕt invest In-house dev. Provider A Provider B Provider C
From the evaluation results, we not only get a final recommendation, but also know how to improve the project we chose. Before translating the fuzzy scores to crisp scores to rank the alternatives, we know which criteria received better scores and their difference in degree from others in this final suggested project. Meanwhile, the criteria with lower scores can be compared to the scores for the same criteria in other alter-natives to find some suggestions for improving the chosen project. We can tell the difference between them and decide our tolerance degree. We also can decide if there is room for improvement before negotiating with suppliers. If there is, we could get a better contract, asSmithson and Hirschheim (1998)emphasized. Therefore, the model is not only an evaluation tool, but also an analytical and negotiating tool.
The flexibility of our model allows evaluators to change the weights of the criteria to represent the spe-cific characteristics of the IT/IS and reflect the culture of the organization. A new evaluation team can be formed to evaluate another new IS investment. Through the evaluation process, new criteria values can be obtained, which may be different from those of the previous project.
During the evaluation process, representatives of all stakeholders get a chance to present their opinions, so the model is also a discussion and communication tool for stakeholders.
5. Conclusions
In this study, we propose a method based on the fuzzy multi-criteria decision model for the evaluation of new IT/IS investments. We compare prior studies to compile a list of evaluation criteria. The final list of criteria is obtained using a Delphi approach. We form two evaluation teams, members of which include end-users, executive managers, and staff from the IT, accounting, and auditing departments. The evaluation model considers the existing IT portfolio to address possible duplicate investment and compatibility issues. Opinions are expressed with linguistic variables, instead of crisp values to reduce perceptive bias. Only a software worksheet, such as Excel, is required for calculation of the final results. The evaluation process is well documented and, therefore, traceable. This model provides the flexibility to change the weights of the criteria, and uses those weights to reflect special concerns about IT/IS, as well as organizational char-acteristics such as cultural, and contextual issues. It can also be used as an analytical tool to improve the projectÕs contract to attain to the ideal goal. We believe the proposed approach is simple, cost-effective, and versatile and can be used as a decision support tool for any IT/IS investment evaluation.
Acknowledgement
This research has been partially supported by the MOE Program for Promoting Academic Excellence of Universities (91-H-FA08-1-4) of Taiwan.
Appendix A
Table 9.
Appendix B
The proposed FMCDM evaluation method is comprised of the following 10 steps. Step 1: Form two evaluation teams: a weighting team and a scoring team.
Table 9
Summary of criteria
Author(s) Criteria
Bacon (1992), Escobar-Perez (1998) Net present value Internal rate of return Profitability index method Average/accounting rate of return Pay-back method
Budgetary constraints
Support explicit business objectives Support implicit business objectives Response to competitive systems Support management decision making Probability of achieving benefits Legal/government requirements Technical/system requirements Introduce/learn new technology Probability of project completion
Jones and Beatty (1998) Reduced transaction costs Improved cash flow Reduced inventory Improved information flow Improved internal operations Improved service
Improved trading partner relations Improved competitive advantage Disrupted work at first Changed operating procedures Decreased productivity at first Long learning time
Required new hardware/software Increased computer support needs Substantial site preparation Required changes to existing systems
Irani (2002) Improved growth and success Leader in new technology Improved market share Market leadership
Enhanced competitive advantage Improved flexibility
Improved response to changes Improved product quality Improved organizational teamwork Promotes concept of open culture.
Improved integration with other business functions Increased productivity
Increased plant efficiency Reduced delivery lead-times Reduced manufacturing lead-times Improved capacity planning Improved stability of MPS Improved data management Improved manufacturing control Improved accuracy of decisions
Table 9 (continued)
Author(s) Criteria
Reduced raw material inventory Reduced levels of WIP Reduced labor costs Reduced manufacturing costs Increased throughput Improved data availability and reporting structure
Improved communication through Ôon-lineÕ order progressing Improved product tractability
Formalized procedures with accountability and responsibility Improved schedule adherence
Cost of ownership: system support Management/staff resources Management time
Management effort and dedication Employee time
Employee training Employee motivation Changes in salaries Staff turnover
Productivity losses and organizational impact Strain on resources
Business process re-engineering Security software protection Security breaches Organizational re-structuring Ryan and Harrison (2000) Productivity Training Labor savings Work quality Change of management Learning curve Job satisfaction Better decisions Improvement in communications Morale Loss of control Mirani and Lederer (1998)
Enhances competitiveness or create strategic advantage Enables the organization to catch up with competitors Aligns with stated organizational goals
Helps establish useful linkages with other organizations Enables the organization to respond more quickly to change Improves customer relations
Provides new products or services to customers Provides better products or services to customers
Enables faster retrieval or delivery of information or reports Enables easier access to information
Improves management information for strategic planning Improves the accuracy or reliability of information Improves information for operational control
Presents information in a more concise manner or better format
Table 9 (continued)
Author(s) Criteria
Increases the flexibility of information requests Saves money by reducing travel costs Saves money by reducing communication costs Saves money by reducing system modification or enhancement costs
Allows other applications to be developed faster
Allows previously infeasible applications to be implemented Provides the ability to perform maintenance faster
Saves money by avoiding the need to increase the work force Speeds up transactions or shorten product cycles
Increase return on financial assets
Enhances employee productivity or business efficiency
Mcaulay et al. (2002) Enhances competitive/strategic advantage Increases access to expertise
Increases flexibility Reduces uncertainty
Eliminates unnecessary functions
Improves management information for strategic business planning Improves information for management decision making
Enables faster retrieval or delivery of information for reports Improves reliability or accuracy of information
Enables focus on core in-house operations Improves the quality of IT systems
Improves service quality/service delivery to customers Provide new and/or broader ranges of products/services Improves disaster recovery
Increases process efficiency
Reduces technology costs through reduced system enhancements and modifications
Reduces operations costs Reduces workforce costs
Increases access to new technologies
Allows previously unfeasible applications to be implemented Allows faster development of applications
Reduces the risk of technological obsolescence Vendor opportunism
Lack of flexibility (becoming locked into the vendor) Potential loss of secrets and intellectual property Change in commitment or financial stability of a supplier Client engaged in new line of business—requires changes to contract or even termination of contract Lack of active management of the vendor by the client Over-dependence on the vendor
VendorÕs lack of clientÕs enterprise knowledge
Treating IT as an undifferentiated commodity to be outsourced Lack of employee morale leading to poor performance and high staff turnover
Loss of expertise within the company
Vendor fails to provide contracted service to the required level Costs of controlling the vendor very high
Step 2: Pick appropriate evaluation criteria and organized them in an appropriate format.
Step 3: Identify various potential alternative projects and combine them with the existing IT portfolio to form the set of alternative candidates.
Step 4: Set proper linguistic scales (very high, high, medium, low, and very low) and ask the members of the weighting team to give their opinions by pair-wise comparison of evaluation criteria. Step 5: Convert the linguistic variables into triangle fuzzy numbers as shown inTable 1. This gives
every-oneÕs fuzzy reciprocal matrix (Table 10).
Step 6: Aggregate the weighting team membersÕ fuzzy reciprocal matrices by geometric means and form the final aggregated fuzzy reciprocal matrix. In the same way, aggregate the scoring team mem-bersÕ opinions by geometric means.
Step 7: Take the geometric row means of every criterion and normalize it to get its local weight. Then, calculate the global importance weight.Let eA¼ ½~auvkk be the fuzzy reciprocal matrix; k, the
num-ber of criteria; and auv, the relative importance of u criteria over v criteria.
~
ru¼ ð~au1 ~au2 ~au3 ~aukÞ 1=k
;
where k is number of criteria; the symbol denotes fuzzy multiplication; and ~ruis the geometric
row mean of relative importance of the uth criterion over all the other criteria. ~
wu¼ ~ru ð~r1 ~r2 ~rkÞ1;
where the symbol denotes fuzzy addition, and ~wu is the normalized local weight of the uth
criterion.
Normalize the geometric row mean of ~ruget the normalized local weight of every criterion. The
local weight of every criterion is then multiplied by its higher levelÕs weight to get the global weight. For example, inFig. 1, the local weight of ÔManpowerÕ is (0.64, 0.77, 0.89), the local weight of ÔProbability of completionÕ is (0.38, 0.40, 0.42), the local weight of ÔRisk criteriaÕ is (0.10, 0.13, 0.18), so the global weight of ÔManpowerÕ is
ð0.64; 0.77; 0.89Þ ð0.38; 0.40; 0.42Þ ð0.10; 0.13; 0.18Þ
¼ ð0.64 0.38 0.10; 0.77 0.40 0.13; 0.89 0.42 0.18Þ ¼ ð0.03; 0.04; 0.07Þ.
Step 8: Decide the rules for scoring. All criteria are separated into two groups, the cost criteria group and the benefit (or non-cost) criteria group, using different translation formulas.
The scoring formula for the benefit criteria translates linguistic variables into triangle fuzzy numbers directly, as shown inTable 2. Logically, a higher linguistic value means a larger value
Table 10
Linguistic scale and its triangle fuzzy number conversion
Linguistic scale Triangle fuzzy scale
Very high (9:1) (8, 9, 9)
Between very high and high (7:1) (6, 7, 8)
High (5:1) (4, 5, 6)
Between high and medium (3:1) (2, 3, 4)
Medium (2:1) (1/2, 1, 2)
Exactly equal (1:1) (1, 1, 1)
between medium and low (1:3) (1/4, 1/3, 1/2)
Low (1:5) (1/6, 1/5, 1/4)
Between low and very low (1:7) (1/8, 1/7, 1/6)
of triangle fuzzy numbers, which translates into a better score, indicating that the alternative pro-ject will be more beneficial to the company (Table 11).
With regard to the cost criteria, since the highest linguistic value of cost criteria means the worst score, the translation uses the following normalization formula to turn a higher xijlinguistic value
into a lower rijscore:
rij¼ x– j xij x– j xj 100;
where i is the ith alternative project; j is the jth criteria; xijis the score of the ith alternative under
the jth criteria; rijis the normalized score of xij; xj is the lowest score, which means the lowest cost,
under the jth criteria; and x
j, is the highest score under the jth criteria. The range of triangle fuzzy
values is between 0 and 100, so we multiply the normalization value rijby 100.
Step 9: Aggregate all the expertsÕ opinions for each criterion and calculate the final score of all the alter-native projects. Every alteralter-native project has a score for each criterion. We multiply each score with the corresponding weight of the criterion and sum up the result to get the final net score of the alternative. The calculation is as follows:
Let ~wj¼ ðlj; wj; ujÞ be the global weight of the jth criterion; li, the lowest range value; wi, a
median; and ui, the largest range value.
Also, let ~rij¼ ðLEij; MEij; UEijÞ be the score of the ith alternative under the jth criterion; LEij,
the lowest range value; MEij, a median and UEij, the largest range value.
Then, the jth criterion score of ith alternative is
eRij ¼ ðLij; Mij; UijÞ ¼ ~rij ~wj¼ ðLEij lj; MEij mj; UEij ujÞ.
Sum up every score of all criteria for the ith alternative and we get the final score: eRi¼ ðLi; Mi; UiÞ ¼ Xk j¼1 ~rij ~wj¼ Xk j¼1 LEij lj; Xk j¼1 MEij mj; Xk j¼1 UEij uj ! .
Step 10: Translate each final triangle fuzzy score of the alternative projects into a crisp value for ranking purposes.
We useOpricovic and TzengÕs (2003)de-fuzzification formula to prioritize the ranks as follows: Ri¼ Liþ fðMi LiÞ þ ðUi LiÞg=3;
where Riis the crisp final score of the ith alternative project; Liis the lowest range value; Miis a
median; and Uiis the largest range value.
The higher the value of R, the greater the benefit an alternative project would be to the company. Therefore, the project with the highest R value would be the right choice for investment.
Table 11
Linguistic scale and its triangle fuzzy number conversion
Linguistic scale Triangle fuzzy scale
Very high (1) (85, 100, 100)
High (2) (65, 80, 95)
Medium (3) (40, 60, 75)
Low (4) (13, 30, 55)
References
Bacon, C.J., 1992. The use of decision criteria in selecting information systems/technology investments. MIS Quarterly 16 (3), 335–353. Ballantine, J., Levy, M., Powell, P., 1998. Evaluating information systems in small and medium-sized enterprises: Issues and evidence.
European Journal of Information Systems 7 (4), 241–251.
Banerjee, S., Golhar, D.Y., 1994. Electronic data interchange: Characteristics of users and nonusers. Information and Management 26, 65–74.
Boland, R.J., Hirschheim, R.A., 1987. Critical Issues in Information Systems Research. John Wiley & Sons, Chichester. Earl, M.J., 1989. Management Strategies for Information Technology. Prentice Hall, Hemel Hempstead, UK.
Escobar-Perez, B., 1998. Information systems investment decisions in business practice: The Spanish case. European Journal of Information Systems 7 (3), 202–209.
Farbey, B., Land, F., Targett, D., 1993. IT Investment: A Study of Methods and Practices, Management Today. Butterworth-Heinemann Ltd., UK.
Hochstrasser, B., Griffiths, C., 1991. Controlling Information Technology Investments: Strategy and Management. Chapman and Hall, London.
Iacovou, C.L., Benbasat, I., Dexter, A.S., 1995. Electronic data interchange and small organizations: Adoption and impact of technology. MIS Quarterly 19 (4), 465–485.
Irani, Z., 2002. Information systems evaluation: Navigating through the problem domain. Information and Management 40 (1), 11–24. Irani, Z., Love, P.E.D., 2002. Developing a frame of reference for ex-ante IT/IS investment evaluation. European Journal of
Information Systems 11 (l), 74–82.
Irani, Z., Ezingeard, J.-N., Grieve, R.J., 1998. Costing the true costs of IT/IS investments: A focus during management decision making. The Journal of Logistics Information Management 11 (1), 38–43.
Jones, M.C., Beatty, R.C., 1998. Towards the development of measures of perceived benefits and compatibility of EDI: A comparative assessment of competing first order factor models. European Journal of Information Systems 7 (3), 210–220.
Kaplan, R.S., 1986. Must CIM be justified by faith alone? Harvard Business Review 64 (2), 87–95.
King, J.L., Schrems, E.L., 1978. Cost–benefit analysis in information systems development and operation. Computing Surveys 10 (1), 19–34.
Lefley, F., Sarkis, J., 1997. Short-termism and the appraisal of AMT capital projects in the US and UK. International Journal of Production Research 35 (2), 341–368.
Mcaulay, L., Doherty, N., Keval, N., 2002. The stakeholder dimension in information systems evaluation. Journal of Information Technology 17 (4), 241–255.
Metaxiotis, K.S., Papakonstantinou, A.P., Psarras, J.E., 1999. Evaluating the integrated measurement and evaluation system IMES: A success story. Communications of the Association for Information Systems 2. Available from:http://cais.isworld.org/articles/ default.asp?vol=2&art=15. Article 15.
Mirani, R., Lederer, A.L., 1998. An instrument for assessing the organizational benefits of IS projects. Decision Sciences 29 (4), 803– 838.
OÕCallaghan, R., Kaufmann, P.J., Konsynski, B.R., 1992. Adoption correlates and share effects of electronic data interchange systems in marketing channels. Journal of Marketing 56, 45–56.
Opricovic, S., Tzeng, G.H., 2003. Defuzzificaion for a fuzzy multi-criteria decision model. International Journal of Uncertainty, Fuzziness and Knowledge-based Systems 11 (5), 635–652.
Orli, R.J., Tom, J.C., 1987. If itÕs worth more than it costs, buy it! Journal of Information Systems Management 4 (3), 85–89. Orlikowski, W., 1992. The duality of technology: Rethinking the concept of technology in organizations. Organization Science 3 (3),
398–427.
Powell, P., 1992. Information technology evaluation: Is it different? The Journal of the Operational Research Society 43 (l), 29–42. Prabhu, T.R., Vizayakumar, K., 2001. Technology choice using FHDM a case of iron-making technology. IEEE Transactions on
Engineering Management 48 (2), 209–222.
Premkumar, G., Ramamurthy, K., 1995. The role of interorganizational and organizational factors on the decision mode for adoption of interorganizational systems. Decision Sciences 26 (3), 303–336.
Renkema, T.J.W., 2000. The IT Value Quest: How to Capture the Business Value of IT-Based Infrastructure. John Wiley & Sons, Chapter 6.
Renkema, T.J.W., Berghout, E.W., 1997. Methodologies for information systems investment evaluation at the proposal stage: A comparative review. Information and Software Technology 39 (1), 1–13.
Ryan, S.D., Harrison, D.A., 2000. Considering social subsystem costs and benefits in information technology investment decisions: A view from the field on anticipated payoffs. Journal of Management Information Systems 16 (4), 11–40.
Saaty, T.L., 1980. The Analytic Hierarchy Process. McGraw-Hill Inc.
Serafeimidis, V., Smithson, S., 2000. Information systems evaluation in practice: A case study of organizational change. Journal of Information Technology 15 (2), 93–105.
Serafeimidis, V., Smithson, S., 2003. Information systems evaluation as an organizational institution—Experience from a case study. Information Systems Journal 13 (3), 251–274.
Smithson, S., Hirschheim, R., 1998. Analyzing information systems evaluation: Another look at an old problem. European Journal of Information Systems 7 (3), 158–174.
Stouffer, D., Rachlin, S., 2002. A Summary of First Practices and Lessons Learned in Information Technology Portfolio Management. Federal CIO Council Best Practices Committee.
Strassmann, P., 1997. The Squandered Computer. The Information Economics Press, New Canaan.
Vaid-Raizada, V.K., 1983. Incorporation of intangibles in computer selection decisions. Journal of Systems Management 33 (11), 30– 36.
Voss, C.A., 1986. Managing advanced manufacturing technology. International Journal of Operations and Production Management 6 (5), 4–7.
Walsham, G., 1993. Interpreting Information Systems in Organizations. John Wiley & Sons, Chichester.
Willcocks, L., 1992. Evaluating information technology investments: Research findings and reappraisal. Journal of Information Systems 2 (4), 243–268.
Willcocks, L., Lester, S., 1997. Assessing IT productivity: Any way out of The Labyrinth? In: Willcocks, L., Feeny, D., Islei, G. (Eds.), Managing IT As A Strategic Resource. McGraw-Hill, Maidenhead, pp. 64–89.
