Design of a knowledge-based performance evaluation system: A case of high-tech state-owned enterprises in an emerging economy

Design of a knowledge-based performance evaluation system: A case

of high-tech state-owned enterprises in an emerging economy

Wei-Kang Wang

, Hao-Chen Huang

, Mei-Chi Lai

a_{Department of Accounting, College of Management, Yuan-Ze University, Taiwan, ROC} b_{Graduate Institute of Business Administration, National Taiwan University, Taiwan, ROC} c

Graduate Institute of Health Care Organization Administration, National Taiwan University, 6F., No. 17, Shiujou Rd., Taipei 100, Taiwan, ROC

Abstract

This paper describes an intelligent decision support system for evaluating state-owned enterprises (SOEs) using DEA models. This decision support system converts numerical data into information that can be used to evaluate state-owned enterprises (SOEs). In this paper, we use DEA models, including CCR, BCC and FDH, to shed new light on the operational efficiency and determinants of state-owned enterprises in Beijing. We propose an evaluation framework for evaluating state-state-owned enterprises (SOEs), including a database management subsystem, a model base subsystem, a knowledge acquisition subsystem, and a dialogue subsystem. Finally, we adopt the logistic regression model to analyze the effect factors of inefficient state-owned enterprises and provide optimum decision-making for state-owned enterprises (SOEs).

Ó 2007 Published by Elsevier Ltd.

Keywords: Decision support system; Knowledge-based system; Performance evaluation; Data envelopment analysis

1. Introduction

In the past, state-owned enterprises (SOEs) contributed greatly to Chinese society. In recent years, state-owned enterprises (SOEs) in China have become more and more competitive. Thus, it is important for these state-owned enterprises (SOEs) to be operated efficiently. How can enter-prises improve in terms of operational efficiency through performance evaluation? How can enterprises strengthen links between their institutions in order to increase their resources and improve their competitiveness? When analyz-ing the operational performance of SOEs, we seek to answer the following research questions: Does the SOEs invest too much or few? Which SOE performs better in terms of

pro-ductivity? What suggestions can we give to inefficient SOEs to improve their efficiency?

In recent years, a few DEA-based operational perfor-mance studies have been published, and several practical implementations have been reported in journals. DEA has been used to determine the relative efficiency of enter-prises in several studies. DEA is a mathematical program-ming approach developed to measure the relative efficiency of units in an observed group of similar units. DEA pro-vides a relative efficiency measure for each unit based on a set of similar units or on best performers operating on the frontier. DEA is a mathematical programming tool that is well suited to this type of research for several rea-sons. DEA is a linear programming-based technique that converts multiple input and output measures into a single comprehensive measure of productivity efficiency (Epstein & Henderson, 1989). One of its most important features is its ability to handle multidimensional inputs and out-puts, unlike traditional performance indicators that gener-ally use one input–one output measures. Moreover, it is 0957-4174/$ - see front matter Ó 2007 Published by Elsevier Ltd.

doi:10.1016/j.eswa.2007.01.032

*

Corresponding author. Tel.: +886 2 33228071; fax: +886 2 23434201. E-mail addresses: jameswang@saturn.yzu.edu.tw (W.-K. Wang),

d94741004@ntu.edu.tw (H.-C. Huang), d94843004@ntu.edu.tw (M.-C. Lai).

www.elsevier.com/locate/eswa

Available online at www.sciencedirect.com

Expert Systems with Applications 34 (2008) 1795–1803

Expert Systems with Applications

non-parametric, and no preconceived relationships need to be established in order to construct a model of the enterprise. Unlike other frontier approaches that define input–output relationships by estimating a true production frontier, DEA uses actual input–output data to construct a best practice frontier. Once the models are determined, the application of DEA in industry is relatively simple.

Because of the complexity and importance of perfor-mance evaluation, decision support systems are frequently used as tools in support of decision-making. Once, a decision on an investment has been made, it may generate a huge profit or lead to terrible loss. Therefore, decision makers in a company must be careful to conduct a detailed survey and analysis before making an investment. Decision support systems are computer-based tools that help managerial deci-sion-making by presenting various effective alternatives. Since the 1990s, knowledge-based intelligent systems have been playing an important role in new decision support tools. McGregor and Schiefer (2004) introduced an approach to reclaiming and improving knowledge for an organization by establishing a framework that enables the definition of web services from a performance measurement perspective, together with the logging and analysis of web services. This framework utilizes web service concepts, DSS principles, and agent technologies to obtain feedback on an organization’s performance measures through the analysis of web services.Tunali and Ozmehmet (2003) pre-sented a decision support system for forming manufacturing cells. The system consists of four modules: the User Interface Module, Cell Formation Module, Performance Evaluation Module, and Multi-Criteria Decision Analysis Module. Kuo and Soflarsky (2003) dealt with the development of an automated procedure for selecting motor carriers to min-imize the transportation cost prepaid by a shipper.Cassie (1997)discussed the application of decision support systems (DSS) to assist in solving marketing decisions. Raggad (1997)presented a decision support system (DSS) equipped with a knowledge-based consequential model aimed at improving the efficiency and effectiveness of decision mak-ing. Kaula (1994) discussed combining different decision support systems (DSS).Jacobs and Lauer (1994)described a new micro-computer interactive decision support system (DSS) for operational enhancement in a job shop environ-ment.Wen, Wang, and Wang (2005)presented a new frame-work for knowledge-based intelligent decision support systems for developing a national defense budget planning. Various researchers have used decision support systems for performance evaluation, including McGregor and Schiefer (2004), Tunali and Ozmehmet (2003), Kuo and Soflarsky (2003), Watson, Shi, and Chen (1999), and Who-ley (1998). But few similar studies on state-owned enter-prises (SOEs) have been done. Thus, in this study, a state-owned enterprise (SOE) evaluation decision support system was developed to provide various useful decision-making information and applied to selected state-owned enterprises (SOEs) in China. This decision support system can be used as a strategic planning tool for evaluating

the efficiency and performance based on decision-making information. The objective of DSS is to help decision mak-ers make correct decisions in complex situations using com-plex information. The results of this research can help the enterprises studied here understand the relative operating performance of high-tech state-owned enterprises (SOEs) in an emerging economy.

The rest of the paper is organized as follows. Section 2 presents a brief review of the DEA theorem and related research. Section3 describes the construct prototype and the architecture of the state-owned enterprise (SOE) deci-sion support system. Section 4 presents the system imple-mentation and results of empirical analysis, including system implementation, system tests, industry analysis, effi-ciency analysis, and reference set analysis. Finally, some concluding remarks and a summary are given in Section5.

2. Background and efficiency evaluation of state-owned enterprises

In order to reform state-owned enterprises effectively, Beijing has promoted the indirect foreign investment of SOEs. This has included a total of 104 SOEs, of which 69 are large and middle scale enterprises. The total assets of these 104 SOEs amount to approximately 252 hundred million Yuan (RMB), and the main sales proceeds amount to 121 hundred million Yuan (RMB). Foreign investors have mainly been interested in the electronics industry. In this study, we measured the relative efficiency of SOEs in China. The purposes of this study were to measure and evaluate the productive efficiency of SOEs in a developing country. Many similar researches and studies have mea-sured or evaluated the efficiency of the electronics industry in the U.S. and other developed countries, but very few studies have been done on developing countries or econo-mies. Two stage methodologies were used in this research. The methodology used to perform efficiency analysis of SOEs was data envelopment analysis (DEA) coupled with the logistic regression model. DEA is a linear program-ming-based technique that converts multiple input and out-put measures into a single comprehensive measure of productivity efficiency (Epstein & Henderson, 1989). One of its most important features is its ability to handle multi-dimensional inputs and outputs, unlike traditional perfor-mance indicators that generally use one input–one output measures. Logistic regression is a statistical technique used to classify observations, by means of a set of independent variables, into two or more mutually exclusive categories. In this study, we used the logistic regression method to investigate the inefficiency effects of SOEs.

We constructed a state-owned enterprise (SOE) evalua-tion decision support system state-owned enterprises (SOEs) for evaluating as shown inFig. 1.

Step 1: Choosing targets (SOEs): We conduct research analysis with financial and non-financial information through a case study to understand the industry

mental factors, resources, dispositions, value-created activ-ities, and costs involved in choosing evaluation targets.

Step 2: Using DEA analysis to measure efficiency scores: The DEA models include CCR, BCC, SBM, and FDH. Our approach includes more than 46 models and can deal with large- scale problems. We can select a DEA model and input–output variables to measure efficiency scores.

Step 3: Analyzing the effect factors of inefficient state-owned enterprises: We adopt the logistic regression model to analyze the effect factors of inefficient state-owned enterprises.

Step 4: Reference set analysis and slack variable analy-sis: We use reference set analysis and the slack variable method to analyze the usefulness of various inputs in order to find ways to improve SOEs.

Step 5: Decision-making: After DEA analysis, we can perform decision-making for performance evaluation.

Step 6: Re-choosing targets (SOEs): If the authority has not carried on buys or defeat, it may again re-choose tar-gets for tartar-gets.

3. The architecture of the state-owned enterprises (SOEs) evaluation decision support system

This research will use Web-based. A web-based system is an easy-to-use interface, offering support for heterogeneous computing platforms and relatively low costs have been gaining widespread acceptance (Liao, 2002). We associate the database management subsystem, the model base sub-system, the knowledge acquisition subsub-system, and the dia-logue subsystem to construct decision support system for performance evaluation (DSSPE) of the state-owned enter-prises (SOEs). We used the prototype construction in this study, and it uses the Windows system, the main platform is based on Web-base and the World Wide Web (WWW). This system is designed by HTML and ASP, and managed by Access database.

The DSSPE system is composed of four main compo-nents, as shown inFig. 2: the database management

subsys-tem, the model base subsyssubsys-tem, the knowledge acquisition subsystem, and the dialogue subsystem. The database man-agement subsystem mainly contains a relational database which is managed by a software program called the data-base management system, and which provides speed data retrieval, updating, and appending. A DSS database is a collection of current or historical data and financial or non-financial information from a number of applications or units. The data in a DSS database are usually extracts or copies of operational databases, so using a DSS does not interfere with critical operation systems. The model base subsystem includes many DEA models such as CCR, BCC, SBM, and FDH models that offer the system’s analy-tical or forecasting capability to solve future outcomes. There are many types of models. Modeling languages for building adequate models are also included and together are called the model base management system. This paper focuses on the use of DEA models to evaluate the relative efficiency of the state-owned enterprises. As for the knowl-edge-based acquisition subsystem, it can support any of the other subsystems or play as an independent role. It suggests alternatives or actions to decision makers. Additionally, it can be inter-connected with the company’s knowledge base. Finally, the dialogue subsystem supports a friendly environ-ment for communicating with and giving commands to the DSS through this subsystem.

The above four subsystems form the DSS application system, which is connected to the Internet. The control program first gets input data through the friendly interface (i.e., the dialogue subsystem). Next it searches for rules to select a suitable model and to execute the model to get ana-lytical financial results. Additionally, all the parameters values needed by the models are retrieved from the industry analysis system; indictors select system, and financial and non-financial information. After finishing model analysis, the inference engine uses the results of the model analysis to perform a suggested action. Of course, sometimes, the inference engine may independently infer knowledge rules without using any model.

1. Efficiency Analysis 2. Reference Set Analysis DMUs State-Owned Enterprises (SOEs) 1. Industry Analysis 2. Financial and Non-Financial

Information Decision_{(Yes or No)}

DEA Analysis

Analysis Model (CCR, BCC) Input and Output Variable Selection

Measure Efficiency DMUs Measure Inefficiency DMUs Feedback Process flow Data flow 1. Slack Variable Analysis 2. Logit Model

Fig. 1. The process in the model base.

ESE, TA, PPS, and ROA are statistically significant at the 0.1 level. The results show that the ESE, TA, and PPS vari-ables exhibit significant positive relationships between the effect factors and the inefficient SOEs examined in this study, and that ROA exhibits a significant negative relationship.

5. Conclusions

This paper has reported the design and implementation of an intelligent decision support system for mergers and acquisitions targets. In this study, we have used non-para-metric DEA methods to analyze the technical efficiency of SOEs in Beijing. The main findings can be summarized as follows. The CCR efficiency score analysis results show that one SOE is relatively efficient, and the BCC efficiency score analysis results show that five SOEs are relatively effi-cient. All the inefficient SOEs can improve their

perfor-mance by decreasing their inputs. Efficient reform

strategies (i.e., downsizing and privatization) have been proposed for the inefficient SOEs. For them, the most important approaches to reform are downsizing and priv-atization. Through the successful development of this pro-totype system (DSSPE), it has been demonstrated that SOEs can be evaluated and their performance improved.

Although the DSSPE system has been developed, there are still several ways in which we can further improve its functions. In particular, we can extend its functions to assist a province to evaluate its performance by using dif-ferent models, such as Stochastic Frontier Analysis, the Fuzzy DEA Model, and the Malmquist index, to evaluate long term productivity. Furthermore, we also can use the logistic regression method or Probit model to investigate the effect factors of inefficient SOEs.

There is a need to further develop the DEA models, which can more accurately measure SOEs, performance and identify those SOEs with maximal returns. The devel-opment of such models is necessary for success of those models in SOE performance evaluation.

Acknowledgments

The authors would like to thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract No. NSC 94-2416-H-155-020.

References

Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science, 30(9), 1078–1092.

Cassie, C. (1997). Marketing decision support systems. Industrial Man-agement and Data Systems, 97(8), 293–296.

Deprins, D., Simar, I., & Tulkens, H. (1984). Measuring labor efficiency in post offices. In M. Marchand, P. Pestienu, & H. Tulkens (Eds.), The performance of public enterprises: Concepts and measurement (pp. 243–266). Amsterdam: North-Holland.

Epstein, M. K., & Henderson, J. C. (1989). Data envelopment analysis for managerial control and diagnosis. Decision Science, 20, 90–119. Jacobs, L. W., & Lauer, J. (1994). DSS for job shop machine scheduling.

Industrial Management and Data Systems, 94(4), 15–23.

Kaula, R. (1994). Integrating decision support systems in organizations: a three level framework. Industrial Management and Data Systems, 94(4), 8–14.

Kuo, C. C., & Soflarsky, F. (2003). An automated system for motor carrier selection. Industrial Management and Data Systems, 103(7), 533–539.

Lau, H. C. W., Choy, K. L., Lau, P. K. H., Tsui, W. T., & Choy, L. C. (2004). An intelligent logistics support system for enhancing the airfreight forwarding business. Expert Systems, 21(5), 253–268. Liao, S. H. (2002). Problem solving and knowledge inertia. Expert System

with Application(22), 21–30.

McGregor, C., & Schiefer, J. (2004). A web-service based framework for analyzing and measuring business performance. Information Systems and eBusiness Management, 2(1), 89–110.

Raggad, B. G. (1997). Decision support system: use IT or skip IT. Industrial Management and Data Systems, 97(1/2), 43–50.

Sedbrook, T. (2001). Integrating E-business XML business forms and rule-based agent technologies. Expert Systems, 18(5), 250–265. Tofallis, C. (1997). Input efficiency profiling: an application to airlines.

Computer and Operations Research, 24(3), 253–258.

Tone, K. (1993). An epsilon-free DEA and a new measure of efficiency. Journal of the Operational Research Society, 36(3), 167–174. Tone, K. (2001). A slack-based measure of efficiency in data envelopment

analysis. European Journal of Operational Research, 130(3), 498–509. Tunali, S., & Ozmehmet, S. (2003). A decision support system for cell

formation. International Journal of Computer Applications in Technol-ogy, 17(2), 101–109.

Watson, E. F., Shi, Y., & Chen, Y. S. (1999). A user-access model-driven approach to proxy cache performance analysis. Decision Support Systems, 25(4), 309–339.

Wen, W., Wang, W. K., & Wang, C. H. (2005). A knowledge-based intelligent decision support system for national defense budget planning. Expert Systems with Applications, 28(1), 55–66.

Wholey, J. S. (1998). Assessing the quality and usefulness of performance measurement systems. Public Manager, Potomac, 27(3), 23–24. Table 4

Logistic regression analysis result

logit (Y) =36 + 12ESE + 11TA + 5PPS28ROA + 5AT Chi-square = 10.474 df = 5 p = 0.03317 2* log (Likelihood): 10.1227 Inefficient SOEs (Y = 1) b SE t P-level Constant 36*** ₂₃ 2 <0.001 ESE 12*** _{8 2 <0.001} TA 11*** _{7 2 <0.001} PPS 5*** 4 1 <0.001 ROA 28*** 36 1 <0.001 AT 5 7 1 1.000

Note: ESE: employee. TA: total assets.

PSS: the proportion of state-owned shares. ROA: return of assets.

AT: assets turnover.

* _{Significant at the 0.01 level.} ** _{Significant at the 0.05 level.} ***_{Significant at the 0.1 level.}