http://jis.sagepub.com/content/32/1/17
The online version of this article can be found at:
DOI: 10.1177/0165551506059220
2006 32: 17
Journal of Information Science
Mu-Yen Chen and An-Pin Chen
Knowledge management performance evaluation: a decade review from 1995 to 2004
Published by:
http://www.sagepublications.com
On behalf of:
Chartered Institute of Library and Information Professionals
can be found at:
Journal of Information Science
Additional services and information for
http://jis.sagepub.com/cgi/alerts Email Alerts: http://jis.sagepub.com/subscriptions Subscriptions: http://www.sagepub.com/journalsReprints.nav Reprints: http://www.sagepub.com/journalsPermissions.nav Permissions: http://jis.sagepub.com/content/32/1/17.refs.html Citations:
What is This?
- Feb 13, 2006
Version of Record
>>
performance evaluation: a decade
review from 1995 to 2004
Mu-Yen Chen and An-Pin Chen
Institute of Information Management, National Chiao Tung University, Taiwan
Received 25 March 2005 Revised 9 May and 1 June 2005
Abstract.
In this paper, the development of knowledge management (KM) was surveyed, using a literature review and classifi-cation of articles from 1995 to 2004. With a keyword index and article abstract, we explored how KM performance evaluation has developed during this period. Based on a scope of 108 articles from 80 academic KM journals (retrieved from six online databases), we surveyed and classified methods of KM measurement, using the following eight categories: qualitative analysis, quantitative analysis, financial indicator analysis, non-financial indicator analysis, internal performance analysis, external perform-ance analysis, project-orientated analysis and organization-orientated analysis, together with their measurement matrices for different research and problem domains. Future development directions for KM performance evaluation are presented in our discussion. They include: (1) KM perform-ance measurements have tended towards expertise orientation, while evaluation development is a problem-orientated domain; (2) different information technology methodologies, such as expert systems, knowledge-based
systems and case-based reasoning may be able to evaluate KM as simply another methodology; (3) the ability to con-tinually change and obtain new understanding is the driving power behind KM methodologies, and should be the basis of KM performance evaluations in the future.
Keywords: knowledge management; performance evaluation; literature survey
1. Introduction
A knowledge-based economy is emerging, and knowl-edge management (KM) is being rapidly disseminated in academic circles, as well as in the business world. While an increasing number of companies have launched knowledge management initiatives, a large proportion of these initiatives retain a technical per-spective. The problem with this type of focus is the exclusion and neglect of potential benefits that may be derived from knowledge management. The following types of question are proliferating: is it really worth-while to invest in KM? Has our implementation of KM been a success? Is our KM system productive and effective?
Recent surveys indicate that issues such as ‘measur-ing the value of KM’ and ‘evaluat‘measur-ing KM performance’ are of great importance to managers in places like Asia [1], the United States [2] and the United Kingdom [3]. Given the increasing role of KM in upgrading business competition, the interest of managers, in measuring and evaluating both KM performance and its benefits, is not surprising [4]. This brings up an important research issue: how do most firms that have initiated KM develop appropriate metrics to gauge the effectiveness of their initiative? In other words, there is
Correspondence to: Mu-Yen Chen, Institute of Information
Management, National Chiao Tung University, 1001 Ta Hsueh Road, Hsinchu, Taiwan 30050. E-mail: mychen@ iim.nctu.edu.tw
a need for metrics to justify KM initiatives. Our research objective was therefore to analyze a variety of evaluation perspectives, in order to estimate knowl-edge management performance.
This paper is focused on surveying knowledge management development through a literature review and classification of articles from 1995 to 2004, in order to explore KM performance evaluation during that period. This survey methodology is valuable for a number of reasons. First, it brings an issue into focus by defining and detailing its various characteristics. Second, the results of a survey are typically quantified and therefore amenable to statistical analysis. Third, statistical inference allows one to extend the results obtained from a sample of respondents to a large popu-lation, thereby permitting a wider application. Fourth, the survey methodology is fast and straightforward, compared to many other research methods. In addition, our goal was not only to examine the research trend in KM performance evaluation changes, but also to under-stand the gap between the academic and the business world. For this reason we used a questionnaire survey to investigate high-technology organizations and verify if there really exists a gap in academic research. As a result, the questionnaire survey had a significant relationship with the dependent variables. In other words, it was reasonable to use qualitative, quantita-tive, internal performance, external performance, project-orientated and organization-orientated analyses to evaluate KM performance. Hence, these six perspec-tives will be used in our literature survey methodology to classify and evaluate KM performance.
There are two reasons for choosing this period to survey knowledge management development. First, the knowledge spiral was proposed to corporations and organizations in 1995, and this model plays important roles, not only in conducting academic research studies, but also in creating, exploiting and recycling knowledge within the business environment. Second, there is no doubt that KM existed prior to 1995. KM has been rapidly disseminated in academic circles as well as in the business world because of Nonaka and Takeuchi’s influential book. However, while an increas-ing number of companies launched knowledge man-agement initiatives, KM still remained in the theory, model, and application development phases. Recently, the research trend has moved towards ‘how to measure KM performance’. Therefore, our goal was to examine the research trend in KM performance evaluation changes, using two phases, distinguishing the first five
years (1995–1999) from the second five years (2000–2004). Finally, we hope that the distinction between these two five-year periods will be evident. The history of KM evolution, over the past decade, can be seen in Figure 1.
This literature survey began in January 2005. It was based on a search for ‘knowledge management’ in the keyword index and article abstract within the ISI, Elsevier SDOS, IEEE Xplore, EBSCO, Ingenta and Wiley InterScience online databases, for the period from 1995 to 2004, in which 3699 articles were found. After topic filtering, there remained 108 articles, from 80 journals, related to the keyword ‘knowledge man-agement performance evaluation’. In addition, ISI Web of Knowledge, an integrated Web-based platform, provides high-quality content and the tools to access, analyze, and manage research information. We also used the ‘keyword’ and ‘subject category’ functions to analyze the selected articles. We found that the first five fields were Management (20%), Computer Science and Information Systems (19.2%), Information Science and Library Science (16.2%), Operations Research and Management Science (12.7%), and Computer Science and Artificial Intelligence (10.4%). Based on the scope of these 108 articles, from 80 academic KM journals, this paper surveyed and classified KM measurements into the following eight categories: qualitative analysis, quantitative analysis, financial indicator analysis, non-financial indicator analysis, internal performance analysis, external performance analysis, project-orientated analysis and organization-project-orientated analysis, together with their measurement matrices for different research and problem domains.
The basic underlying assumption is that knowledge may be viewed from a unified perspective; it circulates in the organization creating knowledge assets and influences the performance of the organization. It has multifaceted characteristics, such as: state of mind, object, having access to information, or the potential for influencing future action. We summarized the dis-tinctions between these perspectives about knowledge in Table 1.
The remainder of the paper is organized as follows. Section 2 explains the methodology used to classify KM measurements into the above eight categories. Section 3 presents the survey results of KM perform-ance evaluation, based on the above categories, respec-tively. In Section 4, a discussion offers suggestions for the future development of KM performance evaluation, while in Section 5 we present a brief conclusion.
2. Methodology of the literature review
2.1. MeasuresIn this study, we have examined various KM perform-ance evaluation perspectives, using path analysis with
structural equation modeling (SEM). A survey, con-ducted among existing KM project members, who were corporate employees, was analyzed using LISREL software. Path analysis is an extension of the regression model, used to test the fit of the correlation matrix against two or more causal models, which are being Fig. 1. The KM process [5–12].
Table 1
Diverse perspectives of knowledge and implications for KM [7, 13]
Perspectives Implications for KM
State of mind Knowledge is the state of knowing and KM involves enhancing an individual’s learning
understanding and understanding through the provision of
information
Object Knowledge is an object to be stored and Key KM issue is building and managing
manipulated knowledge stocks
Process Knowledge is a process of applying expertise KM focus is on knowledge flows and the process of creation, conversion, circulation and carrying out
Access to information Knowledge is a condition of access to information KM focus is organized access to, and retrieval of, content
Capability Knowledge is the potential to influence action KM is about building core competencies and understanding strategic know-how
compared by the researcher. A regression is done for each variable in the model that is dependent on others, which the model indicates as causes. The regression weights, predicted by the model, are compared with the observed correlation matrix for the variables, and a goodness-of-fit statistic is calculated. The best fitting, of two or more models, is selected by the researcher as the best model for the advancement of the theory. Path analysis, in the main, has two models: ‘Path Analysis with Observed variables’ (PA-OV) and ‘Path Analysis with Latent Variables’ (PA-LV). In this study, we adopted the PA-OV model to observe each performance evaluation perspective, because there were no latent variables in the PA-OV model; all variables in this model were manifest variables.
Our research has emphasized the evaluation of knowledge management performance. We reviewed the literature on the following categories: qualitative analysis, quantitative analysis, internal performance analysis, external performance analysis, project-orientated analysis and organization-project-orientated analysis. Moreover, we obtained 21 items representing various dimensions underlying KM performance evalu-ation, and these were used to form the initial item pool for the scale in the questionnaire. To make sure that important aspects of KM performance were not omitted, we conducted experience surveys and personal interviews with two professionals, four college teachers, and two CKOs (chief knowledge officers). They were asked to review the initial item list of the scale, and approved the items. Consequently, the 21-item list was considered to constitute a complete domain for KM performance measurement.
In addition, we designed a mediation variable – Knowledge Management Evaluation (KME), which pre-sented the total score from the previous six main items. The higher the KME score, the better it expresses the knowledge management evaluation. Although it is important to understand the score of the KME, our aim is to analyze the KM performance in the business. In other words, KME is just an evaluation tool to present the KM performance. As a result, the variable – Knowl-edge Management Performance (KMP) – is dependent on the KME status, which uses a 10-point scale, scored by the administrator. Again, the higher the score, the more successful the knowledge management perform-ance. Finally, we also designed a moderator – ‘Time’ – to prove that there was a significant relationship between KME and KMP through T-value analysis. Therefore, there was a total of nine observation vari-ables in our model.
For that reason, our exploratory instrument
involv-ing 24 items (as shown in the Appendix), with the one variable – KMP – perceived overall performance and given the success of the knowledge management criterion, the instrument was developed using a 10-point Likert-type scale, with anchors ranging from ‘strongly unimportant’ to ‘strongly important’. For each question, respondents were asked to circle the response which best described their level of agree-ment. The KMP can be used to analyze the criterion-related validity of the instrument and to measure overall significant relationships prior to detailed analysis. After careful examination of the result of experience surveys and interviews, the statements were further adjusted to make their wording as precise as possible.
2.2. Subjects
The data used to test the research model was obtained mainly from five international organizations: the Taiwan Semiconductor Manufacturing Corporation (TSMC), the United Microelectronics Corporation (UMC), the Coretronic Corporation, the Trea Autotech Corporation, and IBM Taiwan. Each company already had KM implemented, and each respondent had experience involving KM projects or in using a KM system. The respondents completed a self-administered, 24-item questionnaire. For each question, respondents were asked to circle the response which best described their level of agreement with the statements. Of the 300 surveyed, 269 useful responses were returned, and thus the response rate was 89.6%. The respondents averaged 34 years of age and had an average of five years experience in KM; the male-to-female ratio was approximately 1.8 to 1. Thirty-nine percent had completed a college or university degree only, while 45 percent had obtained postgraduate degrees. Table 2 gives the detailed information for the questionnaire survey.
2.3. Model assumptions
The KME and KMP in our model were dependent variables. When the KME is a dependent variable, the ‘Qualitative (X1)’, ‘Quantitative (X2)’, ‘Internal Perform-ance (X3)’, ‘External Performance (X4)’, ‘Project-Oriented (X5)’, ‘Organization-oriented (X6)’, and ‘Time (X7)’ are all independent variables. When the KMP is a dependent variable, the ‘Time (X7)’ and ‘KME (X8)’ are independent variables. The regression equation is shown as follows:
Y1= b1X1+ b2X2+ b3X3+ b4X4+ b5X5+ b6X6+
b7X7+ a1 (1)
Y2= b7X7+ b8X8+ a2
Where a is the intercept, and b is the regression coefficient.
Because path analysis is an SEM application, the SEM equation is shown below:
y = α + βy + ΓX+ ζ (2)
where α = intercept, β = regression coefficient, and Γ = regression coefficient between dependent and inde-pendent variable.
The model, shown in Figure 2, has four assumptions, which are described below:
(1) The variables can all be observed.
(2) Each dependent variable has two explained dis-turbances (ζ1, ζ2).
(3) y1has seven structural parameters, from γ1 to γ7. (4) y2has two structural parameters, from β1to β2.
2.4. Path and T-value analysis
The relationships related to the performance evalu-ation perspectives were tested using LISREL path Table 2
Details of the questionnaire survey
TSMC UMC Coretronic Trea Autotech IBM, Taiwan
Industry IC Manuf. IC Manuf. LCD-TFT Automation IT
Survey 60 60 60 60 60
Respondent Deptartment KM KM KM IT KM
Position CKO and CKO and CKO and Manager
Members Members Members and Members Consultants
Average experience 6 4 4 6 5
Average age 38 36 31 31 34
analysis. This provided estimates of the parameters and tests of fit for the linear structural equation model; Figure 3 shows these model relationships as well as the fit measures for the model. The chi-square statistics tested the hypothesis that the proposed model could generate the observed data. What this means is that when the chi-square values are statistically significant, the observed data can be closely patterned by the hypothesized model. Figure 4 shows the T-value analysis between the six performance evaluation per-spectives and KME and KMP. The main findings are described as follows:
(1) Qualitative analysis and quantitative analysis are important in evaluating KM. This is because γ11 and γ12 are 0.25 and 0.21, respectively and simi-larly, γ11 (t = 4.38) and γ12 (t = 3.70) are significant at the 0.001 level, showing that both the quali-tative and quantiquali-tative analyses are observable indicators for KM evaluation. In addition, the γ15 (t = 3.01) and γ16 (t = 2.20) are also significant at the 0.01 and 0.05 levels, respectively.
(2) None of the independent variables had a direct effect on KMP. However, these independent variables may influence the KMP through the mediation variable – KME. As shown in Figure 2, all independent variables had significance in the chi-square statistic tests, especially for X1, X2, X5 and X6.
(3) Time (X7) had a high ‘total effects’ value in this questionnaire analysis. The γ17 and γ27 were 0.13 and 0.2, respectively. In the meantime, X7 had an indirect effect, with KMP (Y2), of 0.06. Therefore, the total effect was 0.26 between Time (X7) and KMP (Y2). The γ27 (t = 4.52) also had significance at the 0.001 level, showing Time as a critical observable indicator for KM evaluation and performance.
2.5. Results
The detailed questionnaire analyses are summarized in Table 3. From these statistics, some interesting facts may be noted:
(1) Qualitative and quantitative analyses had the most significant relationships with KME and KMP. The aim of the quantitative analysis was to present the extent of the impact on both decision making and task performance, using historical data that is easily available, relevant, accurate and timely. This type of evaluation may avoid the drawbacks of qualitative analysis, especially in the subjective judgments of empirical results.
Therefore, KM performance was measured by two quantitative methods: financial and non-financial indicators. Using two indicators to analyze the data was more effective than using quantitative analysis alone.
(2) Project-orientated and organization-orientated analyses had the second most significant relation-ships with KME and KMP.
(3) Internal and external performance analyses had relationships with the lowest significance to KME and KMP. However, we found that γ13 (t = 0.61) and γ14 (t = 0.78) still had positive relationships with KME and KMP.
(4) In this research, we used interview by questionnaire methodology in five famous inter-national organizations. Each company had already implemented KM, and each respondent had experience involving KM projects or in using a KM system. Our questionnaire survey included six independent variables: qualitative analysis, quantitative analysis, internal performance analysis, external performance analysis, project-orientated analysis and organization-project-orientated analysis. In addition, we designed a mediation variable – Knowledge Management Evaluation (KME), since it presented the total score from the previous six independent variables. As shown in Table 3, the independent variables had a signifi-cant relationship with the KME in the chi-square statistic tests. In other words, it was suitable and reasonable to use qualitative, quanti-tative, internal performance, external performance, project-orientated and organization-orientated analyses to evaluate KM performance. Hence, these six perspectives to classify and evaluate KM performance, were used in our research. Most important of all, we use the results from path analysis to ensure our classification of KM per-formance evaluation is compatible with our literature survey. In this way, the gap between the academic and business worlds has been reduced.
3. Knowledge management evaluation
perspective
3.1. Qualitative analysis
A universally accepted definition of KM does not yet exist. While there is debate as to whether knowledge itself is a cognitive state, a process or an object, the
Fig. 4. T-value analysis.
Fig. 3. LISREL path diagram for KM performance evaluation.
(Qual: qualitative; Quan: quantitative; IP: internal performance; EP: external performance; PO: project-orientated; and OO: organization-orientated.)
description of KM as a process is based on understand-ing an organization as a knowledge system [14]. As Polanyi observed, ‘we can know more than we can tell’ [15]. The notion of tacit knowledge was introduced by Polanyi, a philosopher made known to a larger audience by being quoted in the writings of Kuhn in 1962 [16]; this notion has since had a renaissance, due
to the writings of Nonaka [17] and Nonaka and Takeuchi [11].
While referring to and building on the arguments of Polanyi, different scholars have arrived at contradic-tory conclusions. Cook and Brown argue, in what they claim is agreement with Polanyi, that ‘explicit and tacit are two distinct forms of knowledge, and that one form Table 3
Summary of path analysis
Dependent variables
Independent variables Knowledge management evaluation (Y1) Knowledge management performance (Y2)
Standardized t-values Standardized t-values
Qualitative ( X1) Direct effects 0.25 4.38*** – – Indirect effects – – 0.09 3.29*** Total effects 0.25 4.38*** 0.09 3.29*** Quantitative ( X2) Direct effects 0.21 3.70*** – – Indirect effects – – 0.08 3.24** Total effects 0.21 3.70*** 0.08 3.24** Internal performance ( X3) Direct effects 0.04 0.61 – – Indirect effects – – 0.01 0.60 Total effects 0.04 0.61 0.01 0.60 External performance ( X4) Direct effects 0.04 0.78 – – Indirect effects – – 0.02 0.77 Total effects 0.04 0.78 0.02 0.77 Project-orientated ( X5) Direct effects 0.18 3.01** – – Indirect effects – – 0.07 2.75** Total effects 0.18 3.01** 0.07 2.75** Organization-orientated ( X6) Direct effects 0.13 2.20* – – Indirect effects – – 0.05 2.09* Total effects 0.13 2.20* 0.05 2.09* Time ( X7) Direct effects 0.17 3.56*** 0.2 1.37 Indirect effects – – 0.06 3.15** Total effects 0.17 3.56*** 0.26 4.52***
Knowledge management evaluation ( X8)
Direct effects 0.38 6.75*** Indirect effects – – Total effects 0.38 6.75*** *t-value > 1.96, p < 0.05. **t-value > 2.58, p < 0.01. ***t-value > 3.29, p < 0.001.
cannot be made out of, or changed into, the other’ [18]. In contrast, Tsoukas, also building on Polanyi, claims that tacit and explicit knowledge are mutually consti-tuted and should not be viewed as two separate types of knowledge [19]. In a critique of Nonaka, Tsoukas further argues that tacit knowledge is not explicit knowledge internalized. In fact, tacit knowledge is inseparable from explicit knowledge since ‘tacit knowledge is the necessary component of all knowl-edge’. It seems that most scholars share the opinion of Cook and Brown, that it is useful to treat tacit knowl-edge as separate from explicit knowlknowl-edge.
Consequently, because of the methodology, quali-tative means are suitable to measure tacit knowledge. Qualitative research includes an array of approaches that share some non-quantitative methods. Several have a well-established and widely respected place in the social sciences and social work. The qualitative research approach has been refined by using the outcomes of pilot studies and reviews by researchers in organizational learning. For example, the success of knowledge sharing in organizational culture is not only technological but also related to behavioral factors [20–22]. Besides, expert interviews, critical success factors (CSFs) and questionnaires are used to imple-ment qualitative methods in exploring specific human problems.
From the organizational perspective, attention to an organization’s internal controls has increased signifi-cantly since the 1990s. Although management is ulti-mately responsible for ensuring that internal controls are adequate, managers often lack knowledge of internal control concepts. Changchit et al. used a ques-tionnaire in examining an expert system which could facilitate the transfer of internal control knowledge to management [23]. The results indicated that expert systems are viable aids for transferring internal control knowledge to managers, whose work experience is outside the field of accounting and control systems. Longbottom and Chourides reported on interviews with key staff within organizations, at various stages of approaching and deploying KM programs [24, 25]. In a follow-up paper, the research investigated issues concerning the CSFs and measurements of KM, estab-lishing practical and key factors likely to enhance successful implementation. It accessed a range of critical factors and identified appropriate measures over five organizational perspectives: strategy; human resource management; information technology; quality; and marketing [26]. Furthermore, ASSESS is a proto-type decision support system for managing tacit assess-ment knowledge, which uses knowledge manageassess-ment
system techniques [27]. These techniques include databases, internet architecture and artificial intelligence. The qualitative analysis framework methodology is categorized in Table 4.
3.2. Quantitative analysis
Returning to the literature, we learn that Nonaka and Takeuchi defined explicit knowledge or codified knowledge as knowledge that can be articulated through formal language, including grammatical state-ments, mathematical expressions, specifications and manuals [11]. Such explicit knowledge, they con-cluded, can be transmitted easily and formally between individuals. Choo suggested that explicit knowledge is knowledge made manifest through language, symbols, objects and artifacts [28].
The aim of quantitative analysis is to present the extent of the impact on both decision making and task performance, using historical data that is easily avail-able, relevant, accurate and timely. This evaluation can avoid the drawbacks of qualitative analysis, especially in the subjective judgment of empirical results. There-fore, a quantitative research approach is designed to represent a tangible, visible and comparable ‘ratio’. In other words, quantitative analysis can be used to measure the explicit knowledge of an organization or an individual, with both financial and non-financial indicators; this is discussed below. Table 5 shows the benefits and classes of KM, with qualitative and quan-titative indicators.
3.2.1. Financial indicator analysis. Traditional
quan-titative methods focus on well-known financial measures, such as analyses of financial statements, payback periods, return on investment (ROI), net present value (NPV), return of knowledge (ROK), and Tobin’s q. These methods are best suited to measure daily transaction processing system values.
Table 4
Qualitative analysis
Research methodology Authors
Questionnaire Changchit et al. (2001)
Expert Interviews Longbottom and Chourides (2001) Longbottom and Chourides (2002) Critical Success Factors Chourides et al. (2003)
Decision Support System Mitri (2003) See References [23–27].
The goal is to expand business leaders’ knowledge of the key drivers of customer satisfaction and business process excellence, strengthening their skills to develop profitable growth strategies, based on customer value added (CVA) [29]. Laitamaki and Kordupleski used an ROI index to evaluate KM projects and per-formance in CVA. From the managerial perspective, Stein et al. deployed a knowledge-based system, which was designed to automate tasks previously performed manually, train new staff members, and capture knowl-edge, to enable a university organization to improve services. Performance evaluation used NPV to diagnose the project outcome. Finally, the system could be viewed as an estimation tool, giving a competitive advantage to the organization [30]. From an empirical point of view, it is well known that Tobin’s q ignores replacement costs for intangible assets, because of the accounting treatment of intangibles [31]. Tangible assets are capitalized and reported on firms’ balance sheets. In contrast, intangibles are expensed, i.e. written off on the income statement, along with regular expenses such as wages, rents and interest. As a result, the book value of assets does not reflect the stock of intangibles, resulting from cumulative investments; market value does, however. In fact, it is a fairly common practice, in studies using Tobin’s q as a measure of corporate performance, to ‘correct’ the denominator of q for the presence of such intangibles. Examples include knowledge capital [31, 32], or customer assets [33]. Villalonga also used Tobin’s q to test empirically the hypothesis that the greater the intangibility of a firm’s resources, the greater the sus-tainability of its competitive advantage [34]. The results suggest that intangibles can help firms to
maintain a persistent advantage. Stein et al. also pre-sented a knowledge-based system, to assist furnace pro-duction staff in diagnosing and correcting faults in electron beam guns, which are used to melt titanium [35]. This project used payback period analysis to measure future cash flows over a three year period. In financial strategy, bank failure prediction is an import-ant issue for the regulators of the banking industry. Very often, bank failures are due to financial distress. Early Warning Systems (EWS) may be able to identify the inherent traits of financial distress, based on cial covariates derived from publicly available finan-cial statements. An EWS is a knowledge management system, which uses a knowledge base to aid in bank regulation. [36]
Unfortunately, evaluation methods which rely on financial measures are not as well-suited for compli-cated IT applications. These systems typically seek to provide a wide range of benefits, including many that are intangible in nature. For example, it is difficult to quantify the full value of a point-of-sales (POS) system [37] or an enterprise resource planning (ERP) system [38].
A number of researchers have written about the use of option models in IT investment decision making. The pioneering work of Dos Santos [39] employed Margrabe’s exchange option model [40] to value an IS project, using a novel technology for testing. He argued that the option model would be better than NPV to evaluate the new IT project. Similarly, Kambil et al. [41] used the Cox–Rubinstein binomial option pricing model [42], to determine whether or not a pilot project should be undertaken.
For a software platform, several options may be relevant. In an analogy to Kester’s ‘growth options’ for firms [43], Taudes investigated options for evaluating ‘software growth options’ [44], which could bring valuable software platform benefits.
Benaroch and Kauffman [37] investigated the problem of investment timing, using the Black–Scholes model in a real-world case study dealing with the development of point-of-sale (POS) debit service. Their contribution did not ask whether an investment should be undertaken, but when to exercise the option held, i.e. when to implement a particular IT solution. In a follow-up paper [45], they used sensitivity analysis to probe Black–Scholes valuation for IT investment opportunities. Taudes et al. [38] also compared NPV with the Black–Scholes valuation method, which they used to employ SAP R/2 and to switch to SAP R/3. Their results also indicated that, in the absence of a formal evaluation of the time option, traditional Table 5
benefits of qualitative and quantitative indicators Knowledge management benefits
Qualitative index Quantitative index • Improving employees’ • Decreasing operation
skills costs
• Improving quality • Decreasing product cycle
strategies time
• Improving core business • Increasing productivity
processes • Increasing market share
• Developing customer • Increasing shareholder
relationships equity
• Developing supplier • Increasing patent income relationships
approaches to evaluating information technology investments would have produced the wrong recom-mendations. The methodology of financial index analysis is categorized in Table 6.
3.2.2. Non-financial indicator analysis. Measurement
requires a medium- to long-term commitment from both senior management and the entire staff, and potentially offers little impact on financial perform-ance, in the short-term. The drivers underpinning knowledge performance measures, such as teamwork, learning, communication, knowledge processes, tools and techniques, etc., require non-financial performance measures to ensure that progress is being made, as well as to determine where and when to take corrective action. In fact, non-financial methods are quite differ-ent from traditional financial statemdiffer-ent analysis, using non-financial indexes, such as the ‘frequency’ of employee logins to the knowledge base, how many ‘times’ each employee comes up with a proposal, how many ‘topic numbers’ are on the KMS discussion board, the level of customer ‘satisfaction’, the depth of ‘loyalty’ of employees, and the ‘number’ of communi-ties of practice (CoP) within the company. These indexes are all related to behavioral factors and system usage. Moreover, non-financial indexes are as import-ant as financial indexes, and belong to quimport-antitative analysis.
One good thing about KM is that a company can retain the knowledge it has acquired, even after the source of the knowledge (the employee) has moved on. In terms of human resource training, focus must be placed on developing people who are capable of
turning internal knowledge into organizational knowl-edge. Performance appraisal aims at bringing organiz-ational improvement through effectively directing employee behavior. Yahya and Goh investigated per-formance appraisal characteristics and their respective association with KM [46]. The feedback generated was then used for the purpose of promoting or encouraging better KM practices, especially from the knowledge transfer phase to the knowledge application phase.
Communities of practice have begun to play an increasingly important role in modern, knowledge-intensive organizations. CoPs foster knowledge development and creative interaction among highly specialized experts, helping to channel their efforts towards areas of most need. Smits and Moor presented a Knowledge Governance Framework, which focused on how to define, measure, and use performance indi-cators for KM in a CoP. The results were successful and offer useful guidelines for KM procedures [47].
To manage knowledge successfully, it must be measured. It is not always clear how this can be done, however, as proper measurements may not exist and, indeed, knowledge may be immeasurable. To address this issue, Ahn and Chang assessed the contribution of knowledge to business performance, rather than trying to measure the value of knowledge directly [1]. They provided a way to assess the contribution of knowledge to business performance, by employing products and processes as intermediaries. Product knowledge is directly related to a company’s specific product. Process knowledge is associated with the activities per-formed at each stage in a value chain, from inbound logistics to customer care. In the same way, Holt et al. used four metrics to access organizational knowledge, including individual, context, content and process knowledge measures [48]. These approaches enable us to relate knowledge to business performance more explicitly, and provide valuable insight into how knowledge may be strategically managed.
Organizational performance must usually be defined using non-monetary metrics, making it relatively diffi-cult to measure. Though it can be measured indirectly, using ‘intermediate’ measures, such as the number of new ideas, the number of new products, job satis-faction levels, and the contribution of knowledge man-agement activities to organizational performance, these are difficult to translate into tangible benefits. Organiz-ational performance is as important as financial per-formance; organizational quality can indirectly influence financial performance, serving as a moderat-ing factor. The methodology of non-financial index analysis is categorized in Table 7.
Table 6
Financial indicator analysis Research methodology Authors
Return on investment Laitamaki and Kordupleski (1997) Net present value Stein et al. (2001)
Tobin’s q Ittner and Larcker (1998) Hall et al. (2000)
Lev (2001) Villalonga (2004) Payback period Stein et al. (2003) Financial statements Tung et al. (2004)
Options Benaroch and Kauffman (2000) Taudes et al. (2000)
3.3. Internal performance analysis
Internal performance measurement methods focus on process efficiency and goal achievement efficiency. These methods evaluate KM performance through the gap between target and current value. Well-known methods include ROI, NPV, balanced scorecard (BSC), performance-based evaluation, activity-based evalu-ation and other models.
Underlying Kaplan and Norton’s concept of BSC was that all aspects of measurement have their drawbacks; however, if companies offset some of the drawbacks of one measure with the advantages of another, the net effect can lead to decisions resulting in both short-term profitability and long-term success [49–51]. As a result, they suggested that financial measures be supple-mented with additional ones, reflecting customer satis-faction, internal business processes and the ability to learn and grow. Many scholars have discussed the use of a Balanced Scorecard approach in determining a business-orientated relationship between strategic KM usage and IT strategy and implementation [52–54]. They have applied an IT investment to KM, by creating a KM scorecard focusing on both the current financial impact of intellectual capital on core processes, and future earnings capabilities in structural or human capital.
Most research on KM has been limited to individual levels or knowledge transfer within organizations. However, firm innovation capability is the most important determinant of product performance and competitive ability. Competitive advantage has a sig-nificant positive economic value for a firm, and the main purpose of KM is to retain sustainability. Cavusgil et al. used three items to measure innovation perform-ance [55]. They measured whether the innovation project had succeeded in achieving its main objectives: financial and ROI. The contribution lay in its
perform-ance-based measures and the values of inter-firm relationships in tacit knowledge transfer, as well as innovation capability.
As mentioned earlier, valuable knowledge resides within individual employees and is critical to an organization’s ability to solve problems and create new knowledge. In a sense, KM can be viewed as an activity which acts as a constituent of a community, perform-ing one’s task by usperform-ing tools or technology [56, 57]. Some KM studies have taken an IT perspective, where enterprise information portals (EIPs) are gateways that streamline access to information, thereby easing the task of transforming data into knowledge, thus increas-ing KM efficiency. Kim et al. stressed the importance of activity theory, by using it to evaluate EIP systems in the context of knowledge integration or application [58]. The analysis revealed that EIP functions, from a KM activity perspective, remain underdeveloped.
Many measurement systems have failed to be effec-tive, because they are disparate, often measuring activi-ties that are of local or individual interest to a manager, rather than a key activity for the business. Pervaiz et al. proposed a model founded upon a continuous improvement methodology. This model utilized a Deming type PDCA (Plan-Do-Check-Act) cycle [59]. The proposed measurement framework enabled the effective and efficient leveraging of knowledge assets. The methodology of internal performance analysis is categorized in Table 8.
3.4. External performance analysis
External performance measurement methods always compare a firm with benchmark companies, primary Table 7
Non-financial indicator analysis
Research methodology Authors
Human resource training Yahya and Goh (2002) Communities of practice Smits and Moor (2004) Product and process knowledge Ahn and Chang (2004) assessment
Individual, context, content and Holt et al. (2004) process knowledge assessment
See References [46–48].
Table 8
Internal performance analysis
Research methodology Authors
Balanced scorecard Van Grembergen and Vander Borght (1997)
Martinsons et al. (1999) Fairchild (2002) Performance-based evaluation Cavusgil et al. (2003) Activity-based evaluation Kuutti (1996)
Hasan and Gould (2001) Kim et al. (2002) Plan-do-check-act (PDCA) Pervaiz et al. (1999) cycle
competitors or the industry average. For example, benchmarking is the process of determining who is the very best, who sets the standard, and what that standard is. When we apply the benchmarking concept to business, the following types of questions are asked: ‘Which company has the best manufacturing opera-tion?’ and ‘How do we quantify that standard?’ With benchmarking or best practice methodologies, firms can understand their KM performance by comparison with competitors. Thus, firms can retain a competition advantage and expand the gap between themselves and competitors.
Traditionally, benchmarking has been described as a practice that promotes imitation. However, according to a more recent approach, benchmarking has looked outside a firm’s boundaries, to enable comparison with others, in terms of both practice and performance, in order to acquire both explicit and tacit knowledge [60–62]. Such newly acquired knowledge, once inte-grated with a firm’s prior internal knowledge, may create new knowledge that can give rise to improve-ments and innovations. Benchmarking is also seen as a tool for identifying, understanding and adopting best practices, in order to increase the operational perform-ance of intellectual capital (IC) [63, 64]. From an organizational learning perspective, benchmarking is concerned with enhancing organizational performance, by establishing standards against which processes, products and performance can be compared and con-sequently improved [65]. Furthermore, all the organiz-ational factors examined in both sectors proved to be statistically significant, when comparing world-class and potentially winning companies with their competi-tors; this adds weight to the argument that the exist-ence of organizational learning, within a company, is an essential ingredient in the quest for superior per-formance.
In the endlessly hyped knowledge age of the new millennium, evaluators are being asked to generate lessons learned and best practices. Lessons learned (local knowledge about what works) can be converted to best practices (universal knowledge about what works, at least by implication of being best). Lessons learned represent principles extrapolated from multiple sources, and increase transferability in the form of cumulative knowledge that can be adapted and applied to new situations. The internal validity of any single source of knowledge must be judged by the criteria appropriate for that type of knowledge. Thus, practitioner wisdom and evaluation studies may be internally validated in different ways [66]. On the other hand, the ‘Best Practice’ approach is an essential
com-ponent of KM. It provides an opportunity to retain and use knowledge, even when an expert has left the organization. Asoh et al. investigated how governments could deliver more innovative services to a demanding public [67]. They felt that governments must be involved in the deployment of new services, such as e-Government and e-Commerce. Active management of knowledge assets is mandatory for success. A suggested implementation approach highlights leadership, culture, technology and best practice measurements as critical success factors. The methodology of external performance analysis is categorized in Table 9.
3.5. Project-orientated analysis
Since projects characteristically involve the develop-ment of new products and new processes, obvious opportunities may present themselves for novel ideas to emerge and for cross-functional learning to occur, thereby enhancing the organization’s innovative capacity and potential. On the other hand, recent studies of knowledge management and organizational learning, in project environments, have emphasized instead the difficulties of learning from projects – not only within individual projects, but also across and between projects [68].
Some articles have set out to examine the significance of social factors in enhancing knowledge management capabilities in the construction industry [69, 70]. Bresnen et al. revealed that processes of the capture, transfer and learning of knowledge, in project settings, rely very heavily upon social patterns, practices and processes, in ways which emphasize the value and importance of adopting a community-based approach to managing knowledge [69]. Bresnen et al.’s paper made a contribution to the development of knowledge man-agement theory, within project environments.
Table 9
External performance analysis
Research methodology Authors
Benchmarking Pemberton et al. (2001) Chai et al. (2003) Leung et al. (2004) Massa and Testa (2004) Carrillo (2004)
Marr (2004)
Best practices Patton (2001)
Asoh et al. (2002) See References [60–67].
In recent years, after the term was proposed, numerous individuals and organizations have been trying to put more ‘science’ behind the ‘art’ of ‘knowledge manage-ment’. Rubenstein-Montano et al. found that current project management frameworks did not typically employ a systems approach [71]. For this reason, they suggested that frameworks should be developed within a systems context. With this in mind, Liebowitz provided some useful frameworks to help project managers and others to conceptualize and implement knowledge man-agement initiatives [72]. In the strategy approach, Kamara et al. described a framework for selecting a KM strategy that is appropriate to the organizational and cultural context in KM projects [73]. This approach underscores the fact that knowledge management is not an end in itself, but a means towards the solution of business problems that mitigate inefficiencies and improve the innovative capacity of a company.
Nevertheless, project organizations require particu-larly systematic and effective knowledge management, if they are to avoid knowledge fragmentation and loss of organizational learning [74]. Kasvi et al. dealt with knowledge management and knowledge competence in project organizations, particularly from a programmer’s perspective [75]. Finally, they made a contribution by presenting the Learning Programme Model. In order to systematically manage the knowledge created within a project, the project itself must be systematically managed by the model. The methodology of project-orientated analysis is categorized in Table 10.
3.6. Organization-orientated analysis
With the increasing importance of effective knowledge management in organizations, it has become increasingly
important for organizations to be able to measure their ‘state of the art’ on this subject. Organization-orientated analysis is focused on the entire organization, on the multi-dimensional and multi-layering aspects of the firm. In the horizontal perspectives, KM performance evaluation is focused on leadership, and cultural and technological as well as process dimensions. In the vertical perspectives, KM performance evaluation is focused on strategy, management, and implementation layers. The primary objective is to estimate the level of KM performance from the perspective of the whole organization. KM performance evaluation is carried out using the Skandia AFS (Assurance and Financial Services) model, technology tools and all four perspec-tives in BSC.
Most organizations have only a vague understanding of how much they have invested in intellectual capital (IC), let alone what they may receive from those invest-ments. Standard financial accounting systems do not allow for the easy estimation of intellectual capital investments. Without methods to measure intellectual capital, many firms are ignorant of its full potential. Among the most widely used approaches for IC man-agement and reporting are the so-called Intangible Asset Monitor by Sveiby and the IC approach by Edvinsson and Van Buren, originally introduced by the insurance company Skandia [76–78]. These models are designed to measure human, innovation, process, and customer capital, and represent a major step toward providing precisely the information that firms and their stakeholders need to foresee the future. Thus, these IC models can help visualize the knowledge-production process of research organizations [79]. In addition, some firms have also used BSC – originally developed for strategic management, control, and performance measurement – for IC management and reporting [80–82].
Knowledge management tools can support the per-formance of applications, activities or actions, such as knowledge generation, codification or transfer, and also promote and enable the knowledge process, in order to improve decision making. Ruggles claimed that Knowl-edge Codification is the capture and representation of knowledge, so that it can be accessed, reused and trans-ferred, either by an individual or by an organization [83]. Jackson investigated 59 Knowledge Management tools and examined both the software and technology approaches for knowledge management [84]. Wensley simply discounted any tool that was not web-based, believing that KM tools would only be utilized in an internet environment [85]. Tyndale also evaluated a wide variety of such tools, by examining the literature Table 10
Project-orientated analysis
Research methodology Authors
Social patterns Edelman et al. (2001) Bresnena et al. (2003) KM project management Rubenstein-Montano et al.
framework (2001)
Kamara et al. (2002) Liebowitz and Megbolugbe (2003)
KM project management model Vartiainen et al. (1999) Kasvi et al. (2003) See References [70–75].
related to the selection and evaluation of the KM tools available in the software market [86].
Van Den Hooff et al. presented the Knowledge Man-agement Scan, which is an adequate instrument for diagnosing organizations, with the results providing sufficient insight into the organization’s knowledge processes [87]. In practice, the scan is repeatedly translated into concrete activities, which improves such processes. The methodological reflection, which is important in the execution of the scan, will lead to an instrument that is practically and scientifically valid; moreover, it will give much valuable insight into the subject of knowledge management. The methodol-ogy of organization-orientated analysis is categorized in Table 11.
4. Discussion, limitations and suggestions
4.1. DiscussionKPMG reported that the reasons for the creation of knowledge management initiatives, cited by most
companies, are to facilitate better decision making, increase profits and reduce costs [88]. However, KM suffers from the same challenges as many other man-agement issues: it assumes that knowledge is a ‘thing’, which is amenable to being ‘managed’ by a ‘manager’. It must first be determined which KM process is key to achieving a competitive advantage, and second, which measurement method is the most appropriate to appraise KM performance.
KM performance measurement methods comprise broad categories of research issues. Method develop-ment has been diverse, due to researchers’ back-grounds, expertise and problem domains [89]. On the other hand, some means of analysis have common measurement concepts and methodologies. For example, the NPV measurement method is used in both financial and internal performance analysis. In addition, the BSC measurement method is used in internal performance and organization-orientated analysis. This indicates that the development trend in evaluation methods is also diverse, due to the author’s research interests and ability in the methodology and problem domains. This directs the development of KM performance measurement towards an expertise orientation.
Furthermore, some evaluation methodologies overlap, to a high degree. For example, financial state-ment analysis, ROI, ROK, payback period and option evaluation methods are all quantitative methods, with different concepts and methodologies, which evaluate KM within a common problem domain. This indicates that these evaluation methods are the major trend for KM development, and that many methodologies are focused on these problems. This can direct the development of KM evaluation towards a problem domain orientation.
As shown in Table 12, we gathered statistics for this KM performance evaluation survey research from 1995 to 2004. We divided this data into eight categories of KM performance evaluation methodologies. Our goal Table 12
A review of articles evaluating KM performance: 1995–2004
Approach Paper amount 1995–1999 2000–2004
Qualitative 14 4 10 Quantitative 26 10 16 Internal performance 16 6 10 External performance 15 4 11 Project-orientated 17 5 12 Organization-orientated 20 9 11 Total 108 38 70 Table 11 Organizational-orientated analysis Research methodology Authors
Technology Ruggles (1997)
Jackson (1999) Wensley (2000) Tyndale (2002)
Process Van Den Hooff et al. (2003) intellectual capital Edvinsson (1997)
Sveiby (1998) Van Buren (1999)
Leitner and Warden (2004)
BSC De Gooijer (2000)
Johanson et al. (2001) Bukh et al. (2002) See References [76–87].
was to examine the research trend in KM performance evaluation changes, using two phases, to distinguish the first five years (1995–99) from the second five years (2000–2004). In Figure 5, we can see the change between these first and second five-year periods. The main findings can be described as follows:
(1) KM performance evaluation is becoming more important. Articles published over the last five years are almost double the five years previous to that. This shows that research topics have changed from KM creation, transformation and implementation to the evaluation of KM perform-ance.
(2) Quantitative analysis is the primary methodology used to evaluate KM performance; indeed, most research articles, in the last five years, have involved quantitative analysis. Traditionally, most scholars have suggested financial indicators to display the value of KM; now, more and more scholars are insisting on evaluating KM perform-ance using non-financial indicators, in a social and behavioral sciences approach.
(3) Firms are now highlighting the KM performance of competitors, through benchmarking or best practices, rather than internally auditing KM per-formance via BSC. In Table 12, we can see that articles outlining the external performance approach have grown quite substantially. These
results allow us to infer that, in the future, firms will carefully consider their own KM perform-ance, as well as that of their competitors. For this reason, firms are now using an external perform-ance approach to replace the original BSC frame-work, using benchmarking or best practices to integrate the four perspectives of BSC activities. It is now evident that past, present and future KM performance can be measured by an external per-formance approach.
(4) Firms may begin to focus more on project manage-ment measuremanage-ment, than on the entire organiz-ation. In Table 12, it can be seen that project-orientated articles have grown consider-ably, showing that measurement and control of the achieved percentage of scheduled progress in KM project management is becoming a major focus. Measurement of the entire organization’s KM performance is very difficult from process, leadership, culture or technology perspectives; it is obvious that better efficiency and effectiveness in KM performance can be reached through a project-orientated approach.
In this paper, most of the articles discussed came from management science and social science journals, found on the five online databases, while a few came from computer and information science journals. It is hoped that different research fields will begin to Fig. 5. KM development trend analysis.
publish KM performance evaluation articles, in order to broaden the horizon of academic and practical KM studies.
4.2. Limitations
This analysis and research into KM performance evalu-ation has several limitevalu-ations. First, a literature review of this broad category is difficult, due to the extensive background knowledge required to study, classify and compare these articles. Although limited in back-ground knowledge, this paper has presented a brief literature review of KM from 1995 to 2004, in order to explore how KM performance evaluations developed throughout this period. Thus, the first limitation of this article is its scope, in that surveys that focused solely on a specialty domain, or which were proprietary, and therefore inaccessible, were excluded.
Second, the scope of our investigation was further diminished by utilizing only surveys for which we were able to obtain the original documents. Some academic journals listed in the science citation index (SCI) and the social science citation index (SSCI), as well as other practical reports, were not included in this survey. This weakens our conclusions, somewhat, and notably our proposition that the epistemological foundation of survey research in KM performance evaluation turns around eight categories.
A third constraint, constituting possible limitations in the analysis of these eight categories, may require submission to other researchers for further validation.
Fourth, non-English publications were not con-sidered in this survey, so the effects of different cultures on the development of KM performance evalu-ations were not determined. Many other KM perform-ance evaluations have been published and developed, in addition to those discussed in this article.
4.3. Suggestions
(1) Integration of cultural perspective. In this survey, we obtained questionnaire data from five inter-national high-technology organizations. As such we infer that KM performance evaluation development is closer to that of the information technologies. However, confusion between knowledge and information underlies many of the problems caused by information technology. As Brown and Duguid note, knowledge entails a ‘knower’, but people treat information as inde-pendent and self-sufficient [90]. They argue it is difficult to separate knowledge from information.
In addition, Nardi and O’Day define information ecology as ‘a system of people, practices, values, and technologies in a particular local environ-ment’ [91]. Their goal is to change the way people look at information technology. A key to thought-ful action is to ask more ‘know-why’ questions, before jumping to the more straightforward ‘know-how’ questions. Since we are heading into a totally technology-dominated world, it is very important that we not only know ‘how’ to use a certain technology, but ‘why’ we use a certain technology. Therefore, by trying to understand technology this way, we will then be able to com-municate our thoughts to others and find ways to use technology much more effectively.
(2) Integration of information technologies. KM is an interdisciplinary research issue. Thus, future KM performance evaluation developments will be integrated with information technologies, especi-ally for high-technology organizations; cross- or inter-disciplinary research may offer more methodologies to investigate the challenges of KM performance evaluation.
(3) Integration of options. A major challenge lies in designing models and theories to evaluate the per-formance and value of KM. Traditional financial analysis indicators have long relied on NPV, simple cost–benefit analysis, critical success factors and other less-structured techniques, to make their assessments. Thus, our literature survey has critically reviewed the case for using option pricing as a basis for KM performance analysis, evaluating its merits in a real-word business setting.
(4) Other computer science methodologies. The defi-nition of KM performance evaluation is not complete in this survey, because other method-ologies, such as artificial intelligence, were not included. Data mining and soft computing methods are other research technologies that may be used to solve problems in social studies. Thus, computer science methodologies may include a KM performance evaluation category in future. (5) Evolution as a source of development. Social and
technical evolution may empower the develop-ment of KM performance evaluation. To continue creating, converting, circulating and implement-ing KM processes may be the factors necessary for the successful development of KM. Most importantly, the more KM development is encour-aged, the better KM performance evaluation will be.
5. Conclusions
This paper was based on a literature review of Knowl-edge Management performance evaluation from 1995 to 2004, using a keyword index search. Development of KM performance measurements have tended towards expert orientation, while KM evaluation development is a problem-orientated domain. Different information technology methodologies, such as artificial intelli-gence, may be another way of implementing KM per-formance evaluation. Integration of knowledge-based systems, expert systems and data mining technologies may also increase our understanding of this subject. The ability to continually change, and gain new insights into the power of effective KM performance evaluation, will be the core of future KM research.
Acknowledgement
The authors gratefully thank the Editor and anonymous reviewers for their valuable comments and construc-tive suggestions.
References
[1] J.H. Ahn and S.G. Chang, Assessing the contribution of knowledge to business performance: the KP3 methodol-ogy, Decision Support Systems 36 (2004) 403–16. [2] U. Schultze and D.E. Leidner, Studying knowledge
man-agement in information systems research: discourses and theoretical assumptions, MIS Quarterly 26(3) (2002) 213–42.
[3] M. Shin, T. Holden and R.A. Schmidt, From knowledge theory to management practice: towards an integrated approach, Information Processing and Management 37 (2001) 335–55.
[4] E. Brynjolfsson, A.A. Renshaw and M.V. Alstyne, The matrix of change, Sloan Management Review 38(2) (1997) 37–54.
[5] M. Alavi, KPMG Peat Marwick U.S.: one giant brain,
Case 9–397–108 (Harvard Business School, Boston, MA,
1997).
[6] T. Beckman, A methodology for knowledge manage-ment. In: M.H. Hamza (ed.), Proceedings of the IASTED
International Conference on AI and Soft Computing, Banff, Canada, 1997 (ACTA Press, Calgary, 1997) 29–32.
[7] A.P. Chen and M.Y. Chen, Integrating option model and knowledge management performance measures: an empirical study, Journal of Information Science 31(5) (2005) 381–93.
[8] M.Y. Chen, M.J. Tsai and H.R. Wu, The research of KM
operation module in science and technology industry – case study of TSMC. In: S-C.T. Chou (ed.), Proceedings
of the 12th International Information Management Con-ference (CSIM Press, Taipei, 2001) A-75.
[9] T.H. Davenport, D.W. Long and M.C. Beers, Successful knowledge management projects, Sloan Management
Review 39(2) (1998) 43–57.
[10] J. Liebowitz, Key ingredients to the success of an organization’s knowledge management strategy,
Knowl-edge and Process Management 6(1) (1999) 37–40.
[11] I. Nonaka and H. Takeuchi, The Knowledge Creating
Company (Oxford University Press, New York, 1995).
[12] K. Wiig, Knowledge management: where did it come from and where will it go? Expert Systems with
Appli-cations 13(1) (1997) 1–14.
[13] M. Alavi and D.E. Leidner, Review: knowledge manage-ment and knowledge managemanage-ment systems: conceptual foundations and research issues, MIS Quarterly 25(1) (2001) 107–36.
[14] R.M. Grant, Prospering in dynamically-competitive environments: organizational capability as knowledge integration, Organization Science 7(4) (1996) 375–87. [15] M. Polanyi, The Tacit Dimension, Knowledge in
Organizations (Butterworth-Heinemann, Newton, MA,
1996).
[16] T.S. Kuhn, The Structure of Scientific Revolutions (University of Chicago Press, Chicago, 1962).
[17] I. Nonaka, A dynamic theory of organizational knowledge creation, Organization Science 5(1) (1994) 14–37.
[18] S.D.N. Cook and J.S. Brown, Bridging epistemologies: the generative dance between organizational knowledge and organizational knowing, Organization Science 10(4) (1999) 381–400.
[19] H. Tsoukas, The firm as a distributed knowledge system: a constructionist approach, Strategic Management
Journal 17 (1996) 11–25.
[20] R.J. Calantone, S.T. Cavusgil and Y. Zhao, Learning orientation, firm innovation capability, and firm per-formance, Industrial Marketing Management 31 (2002) 515–24.
[21] M. Hertzum, The importance of trust in software engi-neers assessment and choice of information sources,
Information and Organization 12 (2002) 1–18.
[22] G. Walsham, Knowledge management: the benefits and limitations of computer systems, European
Manage-ment Journal 19(6) (2002) 599–608.
[23] C. Changchit, C.W. Holsapple and R.E. Viator, Transfer-ring auditors’ internal control evaluation knowledge to management, Expert Systems with Applications 20 (2001) 275–91.
[24] D. Longbottom and P. Chourides, Knowledge manage-ment: a survey of leading UK companies. In: P. Hermel (ed.), Proceedings of the 2nd MAAOE International
Conference, Versailles, France, 2001 (UVSQ Press,
[25] D. Longbottom and P. Chourides, Climbing new heights: conquering K2, Knowledge Management Magazine 17 June (2002).
[26] P. Chourides, D. Longbottom and W. Murphy, Excellence in Knowledge Management: an empirical study to identify critical factors and performance measures,
Measuring Business Excellence 7(2) (2003) 29–45.
[27] M. Mitri, Applying tacit knowledge management techniques for performance assessment, Computers &
Education 41 (2003) 173–89.
[28] C.W. Choo, The Knowing Organization (Oxford Uni-versity Press, New York, 1998).
[29] J. Laitamaki and R. Kordupleski, Building and deploy-ing profitable growth strategies based on the waterfall of customer value added, European Management Journal 15(2) (1997) 158–66.
[30] E.W. Stein, M.P. Manco, S.A. Manco, A knowledge-based system to assist university administrators in meeting disability act requirements, Expert Systems
with Applications 21(2) (2001) 65–74.
[31] B. Lev, Intangibles: Management, Measurement, and
Reporting (Brookings Institution Press, Washington, DC,
2001).
[32] B.H. Hall, A. Jaffé and M. Trajtenberg, Market Value and
Patent Citations: A First Look. Working Paper No. 7741
(National Bureau of Economic Research, Cambridge, MA, 2000).
[33] C.D. Ittner and D.F. Larcker, Are non-financial measures leading indicators of financial performance? An analysis of customer satisfaction, Journal of Accounting
Research 36 (1998) 1–35.
[34] B. Villalonga, Intangible resources, Tobin’s q, and sus-tainability of performance differences, Journal of
Economic Behavior and Organization 54(2) (2004)
205–30.
[35] E.W. Stein, M.C. Pauster and D. May, A knowledge-based system to improve the quality and efficiency of titanium melting, Expert Systems with Applications 24(2) (2003) 239–46.
[36] W.L. Tung, C. Quek and P. Cheng, GenSo-EWS: a novel neural-fuzzy based early warning system for predicting bank failures, Neural Networks 17(4) (2004) 567–87.
[37] M. Benaroch and R.J. Kauffman, A case for using real options pricing analysis to evaluate information tech-nology project investments, Information Systems
Research 10(1) (1999) 70–86.
[38] A. Taudes, M. Feurstein and A. Mild, Options analysis of software platform decisions: a case study, MIS
Quar-terly 24(2) (2000) 227–43.
[39] B.L. Dos Santos, Justifying investments in new infor-mation technologies, Journal of Management
Infor-mation Systems 7(4) (1991) 71–90.
[40] W. Margrabe, The value of an option to exchange one asset for another, Journal of Finance 33(1) (1978) 177–86.
[41] A. Kambil, J. Henderson and H. Mohsenzaden, Strategic management of information technology investments: an option perspective. In: R.D. Banker et al. (eds), Strategic
Information Technology Management: Perspectives on Organization Growth and Competitive Advantage (Idea
Publishing Group, Hershey, 1993) 161–78.
[42] J. Cox, S. Ross and M. Rubinstein, Option pricing: a sim-plified approach, Journal of Financial Economics 6 (1979) 229–63.
[43] W.C. Kester, Today’s options for tomorrow’s growth,
Harvard Business Review 62 (1984) 153–61.
[44] A. Taudes, Software growth options, Journal of
Manage-ment Information Systems 15(1) (1998) 165–85.
[45] M. Benaroch and R.J. Kauffman, Justifying electronic banking network expansion using real options analysis,
MIS Quarterly 24(2) (2000) 197–225.
[46] S. Yahya and W.K. Goh, Managing human resources toward achieving knowledge management, Journal of
Knowledge Management 6(5) (2002) 457–68.
[47] M. Smits and A.D. Moor, Measuring knowledge manage-ment effectiveness in communities of practice. In: R. Sprague (ed.), Proceedings of the 37th Hawaii
Inter-national Conference on System Sciences, 2004 (IEEE
Press, Big Island, 2004) 80236b [Abstract only].
[48] D.T. Holt, S.E. Bartczak, S.W. Clark and M.R. Trent, The development of an instrument to measure readiness for knowledge management. In: R. Sprague (ed.),
Proceed-ings of the 37th Hawaii International Conference on System Sciences (IEEE Press, Big Island, 2004) 80238b
[Abstract only].
[49] R. Kaplan and D. Norton, The balanced scorecard: measures that drive performance, Harvard Business
Review 70(1) (1992) 71–9.
[50] R. Kaplan and D. Norton, Putting the balanced scorecard to work, Harvard Business Review 71(5) (1993) 134–42. [51] R. Kaplan and D. Norton, Using the balanced scorecard as a strategic management system, Harvard Business
Review 74(1) (1996) 75–85.
[52] A.M. Fairchild, Knowledge management metrics via a balanced scorecard methodology. In: R. Sprague (ed.),
Proceedings of the 35th Hawaii International Confer-ence on System SciConfer-ences 2002 (IEEE Press, Big Island,
2002) 243 [Abstract only].
[53] M. Martinsons, R. Davison and D. Tse, The balanced scorecard: a foundation for the strategic management of information systems, Decision Support Systems 25 (1999) 71–88.
[54] W. Van Grembergen and D. Vander Borght, Audit guide-lines for IT outsourcing, EDP Auditing (1997) 1–8. [55] S.T. Cavusgil, R.J. Calantone and Y. Zhao, Tacit
knowl-edge transfer and firm innovation capability, The
Journal of Business & Industrial Marketing 18(1) (2003)
6–21.
[56] H. Hasan and E.Gould, Support for the sense-making activity of managers, Decision Support Systems 31(1) (2001) 71–86.
