機器學習在顧客關係管理之應用－以汽車服務個案為例 - 政大學術集成

全文

(1)國立政治大學資訊管理學系. 碩士學位論文指導教授：尚孝純博士. 立. 政治大. ‧ 國. 學. 機器學習在顧客關係管理之應用－以汽車服務個案. ‧ y. sit. Nat. 為例. n. er. io. A Case Study on Machine Learning for Customer Relationship aManagement in Service i v Industry l C n hengchi U. 研究生：何元君中華民國一 O 七年一月.

(2) Acknowledgement First and foremost, I would like to show my deepest gratitude to my thesis advisor Prof. Shari Shang, a respectable, knowledgeable, and open-minded scholar, who has always provided me with valuable guidance and encouragement. Without her continuous support and instruction, I would not be able to complete this research. I would also like to thank the IT personnel of the T-Company, who played a keen role throughout the project. Without their dedicated participation and input, the whole project could not have been successfully conducted. Additionally, I would also like to express thanks to Prof. Ya-Ling Wu and Dr. Yu-Ru Du as my thesis committee. I am gratefully indebted to them for their helpful and insightful comments on this thesis. Last but not the least, I would like to express my very profound gratitude to my parents and to all my friends for providing me with positive encouragement and support throughout my years of study and my life in general. This accomplishment would not have. 學. ‧ 國. 立. 政治大. been possible without them. Thank you.. ‧. n. er. io. sit. y. Nat. al. Ch. engchi. i. i n U. v.

(3) 摘要隨著科技進步，基於機器學習技術的資料分析工具在顧客關係管理領域已被廣為使用。過去的相關研究文獻大多著重於高交易頻次、與客戶互動頻繁的產業，諸如金融、電信、零售業等，但對於具有相反產業特性的服務業等則是缺乏著墨。本研究希望透過案例研究的方式，完整呈現企業如何實際將基於機器學習技術的資料分析工具應用於顧客關係管理業務的過程，以及這些新技術如何幫助企業提升顧客關係管理的成效。本案例使用行動研究方法來歸納、分析、整理整個專案的過程與結果，文末總結本案例於作業、管理以及策略層面的管理意涵與建議。本研究使用的資料來源為台灣一間大型汽車經銷商的資訊部門與其旗下的服務廠，總共包含了約 273 萬筆資料。利用於微軟 Azure 平台上的決策樹模型分析資料，產出高購買機率的顧客推薦名單，服務廠的技師可以針對名單上的顧客推銷，不僅能有效提高推銷的成功率，節省第一線技師的時間，還能夠提升技師以及顧客的滿意度。最後本研究的結果顯示，運用機器學習技術產出的推薦顧客名單，確實能夠幫助本案例公司於顧客區隔以及顧客發展，並達成更有效的顧客關係. 立. 管理。. 政治大. ‧ 國. 學. 關鍵字：行動研究、顧客關係管理、資料探勘、機器學習、服務業. ‧. n. er. io. sit. y. Nat. al. Ch. engchi. ii. i n U. v.

(4) Abstract Data-mining tools and machine-learning techniques have long been used in customer relationship management (CRM), including for customer retention in the financial, retail, and telecommunications industries. However, research on machine learning for CRM in service industries remains rare. Accordingly, this paper uses action research to arrive at a holistic understanding of the process of applying machine learning-based data mining in a specific service-sector business, and whether, how, and how much these novel techniques can help it improve its customer relationships. Key areas of interest include operational, managerial and strategic decision-making processes. Based on approximately 2.73 million rows of data collected from a large car dealership’s IT department and its vehicle-maintenance plants, Microsoft Azure’s boosted decision-tree model generated lists of recommended customers. Such lists could be used by the company to increase the success rates of its promotional activities and to decrease both the overall duration and frequency of technicians’ involvement with promotion. This in turn could lead to more efficient and effective frontline operations, and increased satisfaction not only among customers but also. 立. 政治大. ‧ 國. 學. among technicians. In short, machine learning-based recommended-customer lists helped the. ‧. company achieve more effective CRM through better customer segmentation and customer development.. Nat. n. al. er. io. sit. y. Keywords: Action research, Customer relationship management, Data mining, Machine learning, Service industry. Ch. engchi. iii. i n U. v.

(5) Contents Acknowledgement ...................................................................................................................... i 摘要............................................................................................................................................ii Abstract .................................................................................................................................... iii Contents .................................................................................................................................... iv Table ......................................................................................................................................... vi Figure .......................................................................................................................................vii Chapter 1 Introduction ............................................................................................................... 1 1.1. Background: Business use of machine learning and data mining ................................... 1. 政治大. 1.2. Research motivation ........................................................................................................ 2. 立. 1.3. Research objective........................................................................................................... 2. ‧ 國. 學. Chapter 2 Literature Review ...................................................................................................... 3 2.1. Defining machine-learning techniques............................................................................ 3. ‧. 2.2. Machine learning, data analytics, and customer relationship management .................... 3. sit. y. Nat. 2.3. Data analysis for CRM .................................................................................................... 5. io. er. 2.4. Critical success factors for CRM implementation .......................................................... 7. al. Chapter 3 Research Methodology.............................................................................................. 8. n. v i n Ch 3.1. Research approach........................................................................................................... 8 engchi U 3.2. Research process ............................................................................................................. 8 3.3. Research analysis .......................................................................................................... 10. Chapter 4 Case Study ............................................................................................................... 11 4.1. T-Company and its maintenance service: Background ................................................. 11 4.2. Data gathering ............................................................................................................... 11 4.3. Data feedback ................................................................................................................ 12 4.4. Data analysis: Regression method ................................................................................ 12 4.5. Data analysis: Classification method ............................................................................ 13 4.6. Action planning ............................................................................................................. 13 iv.

(6) 4.7. Implementation.............................................................................................................. 14 4.8. Evaluation...................................................................................................................... 15 Chapter 5 Managerial Implications .......................................................................................... 18 Chapter 6 Conclusion and Recommendations ......................................................................... 20 References ................................................................................................................................ 22 Appendix A. Information Attribute Inputted for Decision Tree Model .................................. 26 Appendix B. Important Variables from Decision Tree Regression Results ............................ 31. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. v. i n U. v.

(7) Table Table 1. Machine learning: Definitions ..................................................................... 3 Table 2. Action-research explanation ........................................................................ 9 Table 3. Experimental results, product A ................................................................ 16 Table 4. Experimental results, product B................................................................. 17. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. vi. i n U. v.

(8) Figure Figure 1. Action-research cycle ................................................................................. 9 Figure 2 Data-analysis process for T-Company CRM ............................................ 15 Figure 3 Timeline of the project .............................................................................. 15. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. vii. i n U. v.

(9) Chapter 1 Introduction 1.1. Background: Business use of machine learning and data mining Machine learning is defined as “a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty” (Murphy, 2012, p. 1). Software developer SAS (2017) further specifies that machine learning “automates analytical model building” and uses algorithms to learn from data iteratively, allowing computers “to find hidden insights without being explicitly programmed with where to look”. The best-known cases of applied machine learning include Google, Facebook and PayPal. Machine-learning algorithms help Google to improve its recommendation engine, and Facebook to recommend friends, ads, and search results to their users. PayPal uses machine-learning technology to identify patterns in users’ purchasing histories, and then uses such patterns to create new rules to stop repeated frauds.. 立. 政治大. ‧ 國. 學. ‧. The widespread use of computers and the Internet has led to the collection and storage of unprecedentedly enormous quantities of customer data, meaning that business analysts have had to move beyond their traditional reliance on analytic tools such as Excel spreadsheets, online analytical processing (OLAP), standard logit/probit models, and linear discriminant analysis (LDA) when seeking to facilitate decision-making processes. Particularly when it comes to customer relationship management (CRM), many have turned to data mining – a union of classical statistics, artificial intelligence (AI), and machine learning (Bose & Mahapatra, 2001; Malik, 2013) – as businesses increasingly devote themselves to uncovering customer behaviors and other patterns hidden in large-scale. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. business data. As well as being inadequate to the sheer quantity of data now available, conventional analytic tools are largely reliant on their users’ preexisting knowledge and experience, and their hunches about what features or dimensions should be analyzed, making it is very difficult to use such tools to predict data trends or patterns. Advanced data-mining tools based on machine learning are independent of this constraint, being able to help business users make future predictions in real time with little human intervention. Machine learning-based data-mining techniques can not only extract information from historical data but also find non-intuitive hidden relationships between variables that experts may miss because it lies outside their range of expectations (El-Zehery et al., 2013; Malik, 2013). In short, state-of-the-art machine learning techniques can improve the performance of data mining, and thus play a critical role in the overall CRM process.. 1.

(10) 1.2. Research motivation There have been many case studies of machine-learning applications in financial services, telecommunications, and other industries characterized by high transaction frequency and/or intensive interactions with customers (e.g., Ali et al., 2012; Bhattacharyya et al., 2011; Chiang, 2012; Coussement et al., 2017; Jones et al., 2015; Ö zden and Umut, 2014). However, there has been little research on the actual or potential effectiveness of machine learning in industries characterized by low transaction frequency or extensive interactions with customers, including but not limited to service industries. Nor do scholars have a clear picture of the downsides of machine learning for business applications. For these reasons, the current study provides a detailed examination of machine learning-based data mining as applied by one company specializing in vehicle sales, maintenance and repair, with special attention to the effects of data mining on its CRM effectiveness.. 政治大. 立. 1.3. Research objective. ‧ 國. 學. ‧. Using action research and approximately 2.73 million rows of data collected from a large Taiwanese car dealership’s IT department and vehicle-maintenance plants, this paper aims to develop a holistic understanding of how data mining is applied in this service-sector business, and whether, how, and how much such techniques help or might help it improve its CRM. Section 2 focus in greater detail on the above-mentioned differences between conventional and novel data-analytic methods, with special attention to CRM. Section 3 briefly introduces the empirical case study, along with the chosen data-collection and data-analysis methods. Section 4 presents the case in detail and the results. Managerial implications are presented in Section 5, and Section 6 comprises a brief conclusion and. n. er. io. sit. y. Nat. al. Ch. recommendations for future research.. engchi. 2. i n U. v.

(11) Chapter 2 Literature Review 2.1. Defining machine-learning techniques The term “machine learning” was coined in 1959 by Arthur Samuel (1959, p. 1). Over the subsequent decades, there has been little consensus on the exact definition of machine learning, as Table 1 indicates. Table 1. Machine learning: Definitions Definition. References. “[A] field ... concerned with the question of how to construct computer Mitchell (1997), programs that automatically improve with experience.” p. xv. 政治大. “[P]rogramming computers to optimize a performance criterion using Alpaydin, example data or past experience.” (2004), p. 3. 立. ‧ 國. 學. “A mature and well-recognized research area of computer science, Fürnkranz et al., mainly concerned with the discovery of models, patterns, and other (2012), p. 1 regularities in data.”. ‧. “A set of methods that can automatically detect patterns in data, and Murphy (2012), then use the uncovered patterns to predict future data, or to perform p. 1 other kinds of decision making under uncertainty.”. y. Nat. sit. er. io. “[C]omputational models using experience to improve performance or Mohri et al. to make accurate predictions.” (2012), p. 1 “[A] field of computer science involving creating and continuously Gualtieri (2015), improving algorithms that automatically analyze data to identify p. 2. n. al. patterns or predict outcomes.”. Ch. engchi. i n U. v. “[T]he science of getting computers to learn and act like humans do, and Faggella (2016) improve their learning over time in autonomous fashion, by feeding them data and information in the form of observations and real-world interactions.” The present research adopts a combination of these definitions’ three key commonalities: i.e., it holds that machine learning (1) automatically detects patterns in data, (2) automatically improves with experience, and (3) is used to uncover hidden patterns or predict future outcomes.. 2.2. Machine learning, data analytics, and customer relationship management 3.

(12) Machine learning has arisen from the maturing field of AI. As noted in the previous section, it has created a set of useful techniques for automating the tedious but crucial task of discovering patterns in databases. Business analysts’ traditional statistical modeling methods for finding patterns in business data have the disadvantage of requiring that their datasets conform to rigid distribution criteria. Traditional OLAP, meanwhile, requires the user to define the query objectives based on his or her personal knowledge and experience, i.e., to pre-judge the possible dimensions or features of the data that will yield the most valuable information, trends and patterns. As datasets grow ever larger, this already subjective process becomes ever more ineffectual as a means of establishing the causes of business problems (Bose & Mahapatra, 2001; Malik, 2013; Olafsson et al., 2008). In contrast, data-mining techniques base their observations on the data itself, impose fewer restrictions, and produce patterns that are easy to understand. Empirical evidence from various fields indicates that data-mining tools clearly outperform conventional tools such as logit and probit models and LDA (Bart & Dirk, 2005; Bhattacharyya et al., 2011; Jones et al., 2015). There are three main reasons for data mining’s superiority over traditional data-analytic tools when used in CRM. The first is that it reduces the effort involved in. 立. 政治大. ‧ 國. 學. feature selection, because it is able to handle many features simultaneously. In this respect, it contrasts sharply with conventional tools such as classical statistical analysis, which are. ‧. usually one-dimensional and therefore cannot handle huge feature spaces, leading to classification problems. Because data mining enables business users to distinguish sequences of data features according to their influences on the researched problem, it can help them to better allocate resources to the most profitable group of customers. As suggested by Liu and Yu (2005), CRM “is characterized by many input variables, often hundreds, with each one containing only a small amount of information”: a situation in which “human expert feature selection is known to be suboptimal” (Prinzie & Van, 2008, p. 2). Moreover, conventional. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. statistical tools’ lack of efficient feature-selection algorithms not only makes them time consuming to use, but compounds the seriousness of the problems they may cause when applied in CRM (Bhattacharyya et al., 2011). The second key advantage of data mining is its ability to handle noisy data, e.g., outliers and missing values. As noted by Prinzie and Van (2008, p. 3), “noise robustness is of the utmost importance to CRM applications typically tormented by noisy data”. Conventional tools like logistic regression have no inherent methods of dealing with missing values and are strongly affected by outliers (Coussement et al., 2017), and therefore are very sensitive to noisy data and the shape or structure of input variables. Data-mining techniques are robust in the face of noisy data, with classification trees (decision tree and random forest) being considered especially resistant to noise and overfitting because of their pruning strategies and ensemble of classifiers (e.g., bagging and boosting). As such, data-mining techniques are likely to require only minimal data intervention by the researcher, and provide better 4.

(13) predictive performance (Bhattacharyya et al., 2011; Jones et al., 2015, 2017; Kotsiantis, 2007; Prinzie & Van, 2008). Thirdly, data mining provides business users with the flexibility to model nonlinear relationships among variables. Most data-mining techniques, including neural networks, support vector machines (SVMs), generalized boosting, and random forest, are high-dimensional nonlinear models that represent significant improvements over traditional statistical methods. Simple linear classifiers such as standard logit/probit models and LDA have limited capacity to model nonlinearity and unobserved heterogeneity in the dataset (Jones et al., 2015). Since traditional statistical models deal with nonlinear relationships by assuming f; y = f(x), and by replacing the linear model with a high-degree polynomial function, the form of the nonlinearity must be known from the outset (Jones et al., 2017; West et al., 1997). The usefulness of classification trees as non-parametric classifiers when given non-linearly associated variables has been widely confirmed. A strength of such trees lies in their ability to deal with heterogeneity, “as separate models are automatically fit to subsets of data defined by early splits in the tree” (Prinzie & Van, 2008, p. 2). Therefore, as. 立. 政治大. suggested by Jones et al. (2015), the newer classifiers such as generalized boosting,. ‧ 國. 學. ‧. AdaBoost and random forest can strongly outperform all other classifiers, and have proved remarkably robust in the face of a wide range data structures and assumptions. Using classifiers that are flexible enough to model nonlinear input-output relationships can provide better overall predictive performance (Jones et al., 2015). Despite these three distinct advantages of machine-learning techniques, some studies have highlighted their drawbacks. One notable argument against machine learning is its lack of interpretability, or the so-called “black box” problem. Though decision trees are relatively easy to explain, other machine-learning techniques including SVMs and neural networks do not allow their users to decipher the role or behavioral influences of input variables (Jones et. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. al., 2017), rendering such techniques’ final classification logic non-understandable (Bose & Mahapatra, 2001). As a separate issue, applying complicated machine-learning techniques requires parameter tuning and calibration not only before, but also during the process (Coussement et al., 2017). And lastly, it has been argued that if data is collected properly and prepared well, pure statistical modeling can achieve comparable performance to advanced machine-learning techniques (Bhattacharyya et al., 2011; Coussement et al., 2017; Jones et al., 2015); and in many real-world cases, the comprehensibility and justifiability of the model are of much greater importance to its users than obtaining the highest possible level of predictive accuracy (Martens et al., 2011).. 2.3. Data analysis for CRM. 5.

(14) The ability to gain useful insights about customers based on business data is increasingly important to managers. As data-mining techniques mature and become more affordable, businesses have shown themselves more willing to adopt these tools, with the wider aim of increasing the effectiveness of their CRM. Swift (2001, p. 12) defined CRM as an “enterprise approach to understanding and influencing customer behavior through meaningful communications in order to improve customer acquisition, customer retention, customer loyalty, and customer profitability”. And Ngai et al. (2009, p. 3) defined CRM as “helping organizations to better discriminate and more effectively allocate resources to the most profitable group of customers”. Some theorists including Swift (2001, p. 12), Parvatiyar and Sheth (2001, p. 5), Kracklauer et al. (2004, p. 4) and Ngai et al. (2009, p. 2) have stated that CRM comprises four-part cycle – customer identification, customer attraction, customer retention, and customer development. The first phase of the CRM cycle, customer identification, involves analysis of customers’ characteristics, so that the most profitable customers and new/potential customers can be segmented and targeted (Ngai et al., 2009). In the second phase, customer attraction,. 立. 政治大. businesses focus and apply resources to directly targeting those customer segments identified. ‧ 國. 學. ‧. in the first phase, with the aim of transforming them into customers (Ngai et al., 2009). The core of CRM is its third phase, customer retention, which aims to maintain and increase customer satisfaction and prevent customer churn via individual marketing, loyalty programs, and complaints management (Ngai et al., 2009). Lastly, customer development involves customer lifetime value analysis, up/cross selling, and market basket analysis, as businesses strive to maximize individual customer profitability, transaction intensity and customer lifetime value (Ngai et al., 2009). Broken down into these dimensions, the benefits of data analysis to CRM are as follows:. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. (1) Customer identification: Enables targeting of the right customer segments and/or detection of the customers who are likely to churn. (2) Customer attraction: Establishes the best communication channels/processes to use with specific customers. (3) Customer retention: Helps improve customer satisfaction and the maintenance of long-term customer relationships. (4) Customer development: Increases customer profitability and customer lifetime value. Iriana and Buttle (2007) and Krishna and Vadlamani (2016) have divided CRM into three primary types: operational, analytical, and collaborative. Analytical CRM involves the use of data-mining techniques, as it aims to improve companies’ business decisions and market distinctiveness via analysis of customers’ behavioral patterns and other customer data. Therefore, only analytical CRM will be dealt with in the remainder of this paper, which uses the terms “CRM” and “analytical CRM” interchangeably except where otherwise noted. 6.

(15) 2.4. Critical success factors for CRM implementation Various studies have discussed the factors critical to the success of CRM implementation (Almotairi, 2008; Alt & Puschmann, 2007; Campbell, 2003; Chen & Chen, 2004; Croteau & Li, 2003; King & Burgess, 2007; Pan et al., 2007). The prior research divides such factors into people, processes, and technology. Regarding the first, all seven of the above-cited studies note that the full support of the top-level management of the organization is essential for effective CRM implementation. Additionally, King and Burgess (2007) and Almotairi (2008) note that explaining CRM strategy to staff can raise their awareness of its objectives and help them change their behaviors. And employees’ incentives to apply CRM in their day-to-day work depends principally on (1) the organization’s evaluation and reward structure, which should be aligned to CRM strategy, and (2) suitable training in how to use the CRM system (Almotairi, 2008; Campbell, 2003; Chen & Chen, 2004; Pan et al., 2007).. 立. 政治大. Within the category of success-critical CRM processes, scholars have identified two. ‧. ‧ 國. 學. broad categories: (1) knowledge management, i.e., the organization’s knowledge of its customers, markets, competitors, etc., and its capabilities of transferring such knowledge into useful information (Chen & Chen, 2004; Croteau & Li, 2003; King & Burgess, 2007); and (2) organizational redesign, whereby the organization is reorganized based on its CRM strategy, with various functions integrated or otherwise connected to support the flow of customer data (Almotairi, 2008; Alt & Puschmann, 2007; Pan et al., 2007). The third and final critical CRM success factor, technology or technological readiness, implies not only that the organization’s IT architecture should support its business strategies and business processes in a general sense, but also that – prior to CRM implementation – it. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. should be in a state of total readiness, extending to interface design, data storage capabilities, and system configurations (Almotairi, 2008; Chen & Chen, 2004; Croteau & Li, 2003; King & Burgess, 2007).. 7.

(16) Chapter 3 Research Methodology 3.1. Research approach This paper is an action research-based case study that aims to describe the whole process of one company’s application of machine learning (in the form of data mining) to CRM, while focusing on three additional key areas, i.e., the process’ operational, managerial and strategic dimensions. Its unifying question is how change can be introduced into the company’s operations such that machine learning can be merged into its business processes. Action research differs from traditional research in terms of how its subjects and objects are defined (Coghlan & Brannick, 2001). More specifically, action research is “an approach in which the action researcher and a client collaborate in the diagnosis of the problem and in the development of a solution based on the diagnosis” (Bryman & Bell, 2011). For Coughlan and Coghlan (2002), the four broad characteristics of action research are that it is (1) research in action, rather than research about action; (2) participative; (3) concurrent with action; and (4) a sequence of events as well as an approach to problem-solving. Action. 立. 政治大. ‧ 國. 學. research was selected for the present case because the members of the target organization participated actively in a cyclical process of interactive actions and collaborations.. ‧ sit. y. Nat. 3.2. Research process. er. io. As the originator of action research, Lewin (1946, p. 5) conceptualized its implementation as “a spiral of steps, each of which is composed of a circle of planning, action and fact-finding about the result of the action”. Coughlan and Coghlan (2002) also. al. n. v i n Ch proposed a framework for the action-research comprising a pre-step – aimed at i U e n g ccycle, h establishing the rationale for both action and research – and six main steps (Fig. 1; see also. Maestrini et al., 2016). The six main steps are (1) data gathering through interviews with organizational members, collection of internal documents, and observation; (2) data feedback, in which the researcher provides the information collected in step 1 to the studied organization’s managers for their analysis; (3) data analysis, which is performed collaboratively by the researcher and managers, thus crucially facilitating the effective implementation of an action or actions; (4) action planning, in which activities are scheduled, and roles and responsibilities assigned to organizational members; (5) implementation, in which the organization executes the planned action(s) with the researcher’s support; and (6) evaluation: reflection on the outcomes of the implemented action(s), so that the next cycle may benefit from the cycle that has just been completed; this is the key to continuous learning. Each action-research cycle leads to another cycle, and thus continuous planning, implementation and evaluation occur over time; and monitoring can be described as a 8.

(17) meta-step, insofar as it takes place throughout all cycles (Coughlan & Coghlan, 2002; Maestrini et al., 2016).. 政治大. 立. ‧ 國. 學. Nat.  Define business problems. Ch.  Collect managers’ view  List possible options Data feedback.  Report the data to managers  Plan the. er. al. How. n. Data gathering. What. io. Steps. sit. Table 2. Action-research explanation. y. ‧. Figure 1. Action-research cycle (Reference: Coughlan & Coghlan, 2002; Maestrini et al., 2016).  Collect first-hand and second-hand data. U enBusiness g c h idata. v ni. warehouse  Interviews and meetings  Observations Interviews and meetings. Results.  Definition of the problems  Clear view of the problems and the situation. Options of the actions. data-analysis scheme Data analysis. Feasibility analysis of  Feasibility analysis Feasible action plans the action plans  Discuss with members and managers 9.

(18) Steps. What. How. Results. Action planning. Schedule the follow-up actions. Meetings with managers. Action plan. Implementation. Action execution. Collaborate with relevant members. Execution. Evaluation. Effectiveness evaluation. Metrics or measures. Performance evaluation. 3.3. Research analysis As the above discussion implies, action research proceeds in an orderly, step-by-step fashion, and thus has the potential to provide a holistic yet detailed picture of a company’s CRM implementation process. The next section reports this case’s action-research process, and is organized according to the six main steps discussed above. After introducing the background of the case company and the business problems, the main section will focus on demonstrating the last three action-research cycles – action planning, implementation, and. 立. ‧. ‧ 國. 學. io. sit. y. Nat. n. al. er. evaluation.. 政治大. Ch. engchi. 10. i n U. v.

(19) Chapter 4 Case Study 4.1. T-Company and its maintenance service: Background T-Company is a leading car dealership in Taiwan, with a market share of more than 30%, 2.6 million customers, and an operating revenue of over 110 billion TWD (3.41 billion USD) in 2016. In the same year, in addition to its showrooms, T-Company operated 122 vehicle maintenance plants. CRM is critical to T-Company, given that it provides warranties on every car it sells. Usually, each warranty covers 120,000 kilometers or a period of four years, whichever is shorter. T-Company’s maintenance service generates considerable profits through routine maintenance fees and parts replacement, as well as a considerable amount and array of information about customers’ habits and needs. To retain existing customers and prevent churning, T-Company provides them with products not only within the warranty period, but also when they exceed it. These products were traditionally promoted in person by technicians to car owner who went to T-Company. 立. 政治大. ‧ 國. 學. plants for maintenance. Those within their warranty periods would be pitched product A,. ‧. which extended the warranty to cover a fifth year or 140,000 kilometers, while those who had already exceeded it were offered product B: three different types of cash cards with built-in discounts for car-maintenance services. T-Company had also set sales targets and sales-performance-related key performance indicators (KPIs) for its maintenance plants. As a result, every technician in those plants was required to promote the service program alongside his/her maintenance tasks, which was not only very time-consuming but raised questions about whether the sales role/mindset was incompatible with that of maintenance professionals. For customers seeking maintenance. er. io. sit. y. Nat. al. n. v i n service, safety is the main concern,C and hifethen technicians i Uappear to be using hard-sell tactics, h c g it could create a bad impression and decrease customer satisfaction. Therefore, T-Company’s management wanted to find out who was most likely to purchase the promoted services, as a means of increasing the success rate of promotion while decreasing the overall duration and frequency of technicians’ involvement with promotional activities. It was hoped that this, in turn, would boost the efficiency and effectiveness of frontline service in the company’s vehicle-maintenance plants, and render customers’ perceptions of such service more positive.. 4.2. Data gathering The current study’s dataset included information on T-Company’s maintenance customers, the customers’ cars, and whether they had bought product A or B (or neither) after having such products promoted to them by technicians between January 2009 and September 2016. In all, this consisted of around 2.73 million rows of data, which T-Company’s IT 11.

(20) personnel had previously cleaned. The data-cleaning process included abnormal-value deletion (e.g., odometer readings of over 500,000 kilometers, or unusual customer dates of birth), and data-format transformation (e.g., transforming dates of birth into ages, and regularizing the names of car models and of vehicle-maintenance plants), as well as filtering the qualification of the car. For example, if product A had already been given to a customer for free, then it was not qualified; and for product B, the number of months since the last regular maintenance should have been fewer than 24. After data cleaning was completed, the dataset was segmented into two parts, according to whether car’s date of manufacture was less than four years previously, or four years or more ago. Both these data segments were divided into training data (70%) and testing data (30%) in order to feed the machine-learning model. In addition to collecting this database data, the researcher interviewed the company’s after-sales team, including its IT technicians; the infrastructure team; and T-company dealers.. 4.3. Data feedback. 立. 政治大. After the data-collection procedures described above were completed, the researcher. ‧. ‧ 國. 學. discussed the data with the company’s middle-level IT managers, after-sales associate managers, and managers of the vehicle-maintenance plants, and collected suggestions from them. For example, how to use the collected data and how to design the machine-learning model.. sit. y. Nat. io. al. er. 4.4. Data analysis: Regression method. n. The objective of the data-analysis step was to reveal which customers had the highest purchasing power. Regression method were the most commonly used business analysis tools.. Ch. engchi. i n U. v. T-Company used the boosted decision tree regression model, which was created on the Microsoft cloud service Azure Machine Learning Studio, to generalize some critical characteristics of high-potential customers and to identify the critical factors affecting customer buying behavior. In a boosted decision tree, the second tree corrects the errors of the first, the third corrects the errors of the first two, and so on. Thus, predictions are derived from the entire ensemble of trees (Microsoft, 2017). After inputting the historical data to the boosted decision tree regression model, several variables were identified. For product A, the more annual compulsory insurance premium is paid for the car, the more likely will the customer buy product A (correlation coefficient: 5.2%). Each vehicle must be insured with the compulsory insurance when registered. The annual insurance premiums are related to the car owner’s age, gender, and the records of traffic violation and car accident. Therefore, this research assumes that if the. 12.

(21) annual compulsory insurance premium is higher, the car is more possible to get car accidents. For this reason, the car owner will be more willing to extend the warranty period. And for product B, if the customer uses the online reservation system to book maintenance services frequently, he or she is more likely to purchase cash cards with built-in discounts for car-maintenance services (correlation coefficient: 13%). It might because that these customers prefer or be used to go to the original manufacturer for maintenance services. As a result, even their warranty periods were exceeded, they still use the online reservation system to book maintenance services. And with such needs, cash cards with built-in discounts can be an attractive option for them. Despite these obvious findings, the remaining variables didn’t have such significant correlations (see appendix B), which means they were not decisive factors. Consequently, the results of the boosted decision tree regression model were not as useful as expected.. 治政 4.5. Data analysis: Classification method 大立 Since the results of the regression model could not be used, this project then used ‧ 國. 學. ‧. two-class boosted decision trees, which was also created on the Microsoft cloud service Azure Machine Learning Studio, to classify customers and generate lists of high-potential customers for T-Company’s vehicle-maintenance plants. The framework of the boosted decision tree algorithm of this case mainly followed the setting from the Microsoft Azure Machine Learning Studio. The height of the tree was about ten layers. After the historical data were input to the machine-learning model, the first decision-tree model was built and the first recommended-customer list was produced. This list included three types of customers: (1) those who had come to T-company’s vehicle-maintenance plants and bought a product already; (2) those who had come to the. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. plants but had not bought any product; and (3) those who had neither come to the plants nor bought a product. The third category was deemed to include customers most likely to buy a product in the future, because among those customers who had visited a plant would have been promoted already, thus those who had never visited a plant were more likely to buy than those who had visited one. This was intended to enable T-Company technicians to focus their selling efforts on customers with high buying potential, while confining their activities to the provision of maintenance services in the case of customers with low buying potential.. 4.6. Action planning To test the performance of this project experimentally, T-Company selected 27 vehicle maintenance plants, and designated 13 of them as the treatment group and the other 14 as the control group. The selection criteria included service capacity, average daily 13.

(22) customer flow, square footage, numbers of maintenance machines, and numbers of technicians; and the treatment group and control group were balanced in terms of these characteristics, to help eliminate interference. After deciding the suitable vehicle maintenance plants, the project leader had to convince the managers of those selected plants that by applying the recommended-customer lists, they could reduce the overall duration of promotion activities while achieving the same sales targets. And then the researcher discussed the implementation plan with the managers of those selected plants, including how to integrate the recommended-customer lists into the operation process, the project schedule, and the evaluation measures.. 4.7. Implementation The first recommended-customer list, as described in Section 4.4, was offered to the 13 vehicle-maintenance plants in the treatment group, and those plants’ technicians instructed to promote products A and B only to those customers on the list. The control group, in contrast, maintained business as usual, i.e., continued promoting the products to every. 立. 政治大. ‧ 國. 學. ‧. customer who came in. During each successive month-long iteration of the experiment, the technicians recorded whether each customer to whom a product was promoted bought one or not. After the first iteration ended, these records were transmitted to analysts in T-Company’s IT department, and these new data input into the decision-tree models to improve their performance. After some parameter tuning and calibration, the new decision-tree models produced the second recommended-customer list, which was then sent to the treatment group as a replacement for the first list. The same process of producing a new recommended-customer list, applying it to the treatment group, feeding new collected data. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. into the model, evaluating model performance, and tuning the model parameters, was repeated a total of five times between April and September 2016.. 14.

(23) 政治大 Figure 2. Data-analysis process for T-Company CRM 立. ‧ 國. 學. ‧. Over the course of all five monthly cycles, those customers who came to the vehicle maintenance plants but did not buy a product despite the technicians’ promotional efforts were removed from consideration, to help the decision-tree models improve themselves, and thus render future recommended-customer lists more accurate.. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. Figure 3. Timeline of the project. 4.8. Evaluation The research team evaluated the performance of this project in terms of (1) the precision and recall fractions of the recommended-customer lists, and (2) the savings achieved in terms of the number of promotion attempts that were needed to achieve the same sales targets (i.e., the number of promotion attempts needed without the lists [X], minus the 15.

(24) number of promotion times needed with the lists [Y], divided by [Y]). The formula for precision was the number of customers in the list who bought a product after having it promoted to them, divided by the total number of customers in the list who were promoted. And the formula for recall was the number of customers on the recommended-customer list who bought a product following promotion, divided by the total number of customers (on or off the list) who bought a product following promotion. 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 (𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑔𝑟𝑜𝑢𝑝) = (𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑙𝑖𝑠𝑡 𝑤ℎ𝑜 𝑏𝑜𝑢𝑔ℎ𝑡 𝑎 𝑝𝑟𝑜𝑑𝑢𝑐𝑡 𝑎𝑓𝑡𝑒𝑟 𝑝𝑟𝑜𝑚𝑜𝑡𝑒𝑑)/ (𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑙𝑖𝑠𝑡 𝑤ℎ𝑜 𝑤𝑒𝑟𝑒 𝑝𝑟𝑜𝑚𝑜𝑡𝑒𝑑). (1). 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 (𝐶𝑜𝑛𝑡𝑟𝑜𝑙 𝑔𝑟𝑜𝑢𝑝) = (𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑤ℎ𝑜 𝑏𝑜𝑢𝑔ℎ𝑡 𝑎 𝑝𝑟𝑜𝑑𝑢𝑐𝑡 𝑎𝑓𝑡𝑒𝑟 𝑝𝑟𝑜𝑚𝑜𝑡𝑒𝑑)/ (𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑤ℎ𝑜 𝑤𝑒𝑟𝑒 𝑝𝑟𝑜𝑚𝑜𝑡𝑒𝑑). (2). 政治大. 立. 𝑅𝑒𝑐𝑎𝑙𝑙 = (𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑙𝑖𝑠𝑡 𝑤ℎ𝑜 𝑏𝑜𝑢𝑔ℎ𝑡 𝑎 𝑝𝑟𝑜𝑑𝑢𝑐𝑡 𝑎𝑓𝑡𝑒𝑟 𝑝𝑟𝑜𝑚𝑜𝑡𝑒𝑑)/ (3). 𝑆𝑎𝑣𝑖𝑛𝑔 𝑝𝑟𝑜𝑝𝑜𝑟𝑡𝑖𝑜𝑛 (𝑝𝑟𝑜𝑚𝑜𝑡𝑖𝑜𝑛 𝑎𝑡𝑡𝑒𝑚𝑝𝑡𝑠) = (𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑟𝑜𝑚𝑜𝑡𝑖𝑜𝑛 𝑎𝑡𝑡𝑒𝑚𝑝𝑡𝑠 𝑛𝑒𝑒𝑑𝑒𝑑 𝑤𝑖𝑡ℎ𝑜𝑢𝑡 𝑡ℎ𝑒 𝑙𝑖𝑠𝑡𝑠 − 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑟𝑜𝑚𝑜𝑡𝑖𝑜𝑛 𝑎𝑡𝑡𝑒𝑚𝑝𝑡𝑠 𝑛𝑒𝑒𝑑𝑒𝑑 𝑤𝑖𝑡ℎ 𝑡ℎ𝑒 𝑙𝑖𝑠𝑡𝑠)/ (𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑟𝑜𝑚𝑜𝑡𝑖𝑜𝑛 𝑎𝑡𝑡𝑒𝑚𝑝𝑡𝑠 𝑛𝑒𝑒𝑑𝑒𝑑 𝑤𝑖𝑡ℎ𝑜𝑢𝑡 𝑡ℎ𝑒 𝑙𝑖𝑠𝑡𝑠). (4). al. er. io. sit. Nat. y. ‧. ‧ 國. 學. (𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑤ℎ𝑜 𝑏𝑜𝑢𝑔ℎ𝑡 𝑎 𝑝𝑟𝑜𝑑𝑢𝑐𝑡 𝑎𝑓𝑡𝑒𝑟 𝑝𝑟𝑜𝑚𝑜𝑡𝑒𝑑). n. Tables 3 and 4 present the results of the third, fourth, and fifth/final cycles of the experiment with regard to products A and B, respectively.. Ch. engchi. i n U. v. Table 3. Experimental results, product A Experiment period (2016). Precision (Treatment group). Precision (Control group). Recall. Saving proportion (promotion attempts). 8/29-9/19 8/08-8/28. 5.98% 5.86%. 4.29% 4.10%. 56.18% 51.79%. 15.91% 15.56%. 7/18-8/07. 6.00%. 3.67%. 40.15%. 15.58%. 16.

(25) Table 4. Experimental results, product B Experiment period (2016). Precision (Treatment group). 8/25-9/19 8/08-8/24 7/18-8/07. Precision (Control group). 6.72% 7.75% 5.83%. Recall. 5.51% 5.64% 6.44%. Saving proportion (promotion attempts). 37.31% 47.00% 17.13%. 6.73% 12.77% -1.78%. As the tables indicate, the treatment group outperformed the control group overall, implying that the recommended-customer lists did reduce the time spent selling products by the technicians in the vehicle-maintenance plants. Moreover, repetitions of the cycle and the associated improvements in the decision-tree model led to newer recommended-customer lists performing better than older ones. However, it is worth noting that the experimental results regarding product B were unstable. The reason for this, discovered by the team in August 2016, was that the vast. 立. 政治大. ‧ 國. 學. ‧. majority of customers’ buying of product B was associated with a single, decisive variable: the customer’s number of reward points, which could be used to pay for this product. Customers received reward points when they purchased vehicle-maintenance services at the. n. al. er. io. sit. y. Nat. plants, as well as from some marketing events; and the more reward points a customer had, the more likely he or she was to buy product B. As a result, almost all the customers on the recommended-customer lists had large numbers of reward points and had already bought product B, meaning that the technicians were unable to use the lists to find new potential customers for product B. As soon as the team discovered this phenomenon, the reward-points variable was removed from the input data, after which the treatment group’s experimental. Ch. engchi. i n U. v. results for product B finally surpassed those of the control group; and the recall fractions of the treatment group also improved.. 17.

(26) Chapter 5 Managerial Implications The present study’s empirical results confirm that the machine learning-generated recommended-customer lists helped T-Company distinguish more clearly between their customers, and thus achieve more effective CRM through customer segmentation and customer development. By utilizing machine-learning techniques, T-Company improved its CRM from some aspects: (1) User/Customer intelligence: machine-learning techniques can identify those high-potential customers more precisely; (2) Enhance understanding of customer profile: through machine-learning models, T-Company tried to generalize some critical characteristics of high-potential customers (see appendix B), although the results were unable to put into practice in this project, still this is a worth-a-try direction; (3) CRM integration: by providing the recommended-customer lists, the IT department of T-Company could support the frontline vehicle-maintenance plants and technicians more proactively, which could lead to better CRM integration among different departments. The project also revealed some key factors for successful future application of approach to similar problems. First, the aspect of the project most critical to its overall. 立. 政治大. ‧ 國. 學. ‧. success was its initial stage, comprising problem definition, data collection and model design. Which data is collected, and whether it is collected appropriately, depends on how well the business problem has been defined. Moreover, effective model selection and model development depend on what kind of results are expected. As well as using classification decision trees to generate the recommended-customer lists, this project also tried using decision-tree regression modeling to identify factors critical to customers’ buying behavior. However, the regression-model results could not be used, since every single factor accounted for small correlation percentages, and there was no decisive factor among all the inputted. er. io. sit. y. Nat. al. n. v i n variables. This result implies that,C when h eunsuitable i U are applied, the investment in n g c h models. research could be in vain. Second, this project highlighted the importance of the means used to verify model results. In this case, the research approach was carefully designed to confirm whether the machine learning-generated lists were correct or not, i.e., by actually using the lists in T-Company’s vehicle-maintenance plants over a period of five months, during which period updated lists were regularly retrieved from the technicians, and the model further improved. Other cases could benefit from the use of similar verification mechanism. Third, how KPIs are set can profoundly affect employees, not least in terms of. whether they will apply the model results in their day-to-day routines. A well-designed strategic plan must establish targets that can translate strategy into manageable operational actions for employees. In other words, targets need to be realistic so that both managers and employees feel comfortable about trying to achieve them (Baroudi, 2014). In this case, where 18.

(27) technicians take on maintenance and sales roles simultaneously, the question of how to set proper KPIs may be especially pertinent to successful project execution. And fourth, organizations should assign more resources to collecting critical data attributes. In this case, it was essential to the performance of the machine-learning model that it have records of whether the customers who bought products did so following product promotion or not. By continuously feeding new data into the model, it was able to revise its algorithm and improve by itself. Therefore, companies should blend data-collection processes into their daily operations in a more strategic fashion. Moreover, this case allows us to compare machine-learning techniques against conventional data-analytic tools. Such a comparison confirms, firstly, that machine-learning techniques can considerably reduce human effort in the sphere of feature selection; the decision-tree model used in this case could handle 135 features simultaneously. Secondly, machine-learning techniques are more robust in the face of outliers and missing values, whereas standard statistical software either only works effectively with complete data, or uses very generic methods for filling in missing values (Patidar & Tiwari, 2013). Decision-tree models, in contrast, do not need to handle missing values, due to its ability to replace them. 立. 政治大. ‧ 國. 學. with new values, or simply ignore them. And thirdly, as mentioned in Section 2, classification. ‧. trees can perform especially well when the relationships among variables are nonlinear, unlike some traditional statistical models.. n. er. io. sit. y. Nat. al. Ch. engchi. 19. i n U. v.

(28) Chapter 6 Conclusion and Recommendations The contributions of this study are both academic and practical. Academically, it has verified the advantages of machine-learning techniques over conventional analytic tools, and of the action-research approach to CRM problems. Practically, it has important implications for organizations who might wish to learn from T-Company’s experience of applying machine-learning techniques on CRM. Apart from finance and telecommunications, scant CRM research has focused on the service sector, despite data-mining techniques’ demonstrated capability to uncover hidden patterns in large-scale data that can help CRM processes become more effective, and the critical importance of shaping customers’ perceptions of services – i.e., that the overall image created by service providers can significantly influence service quality, and thus further influence customers’ choices of the service providers (Ince & Bowen, 2011; Wei et al., 2013; Xiao & Nicholson, 2011). The present research used machine learning-based data-mining techniques and an. 立. 政治大. ‧ 國. 學. action-research approach to improve the CRM process of a large car-sales and maintenance. ‧. company. The decision-tree model selected to generate recommended-customer lists succeeded in helping the company’s frontline technicians better distinguish between customers, and thus improve their operating efficiency and effectiveness. The results of this case support the advantages of applying machine-learning techniques even in service industries where the number of transactions per customer are relatively rare. Much as Croteau and Li (2003) suggested, the present results imply that before project implementation, all the required IT infrastructure and technical staff should be ready; and from the outset, the organization should clearly define its problem and list all the. er. io. sit. y. Nat. al. n. v i n necessary data sources precisely. C Additionally, h e n g chigh-level i U managers should explain their h CRM strategic plan to their employees, and link it to specific targets for them to follow.. Managers must address the benefits of using the new techniques to their business operations, and commit to providing the required support and resources. If a CRM project is cross-functional, different functions of the organization should be integrated and connected, ideally with the involvement of top-level managers. During implementation, monitoring and reporting is essential for model improvement. And the organization should strive to enhance model performance even as they implement it continuously. The features of service-sector CRM applications are closely interrelated with human factors. Unlike in other industries, people are the most important element of service providers. Therefore, when implementing new techniques, staff commitment and motivation are the key factors to success, and businesses should therefore allow ample time not only for training employees in the use of the new techniques, but also for convincing them that such techniques can bring advantages to themselves as well as to the company. 20.

(29) This research has certain limitations. First, because the data used were all from one company operating in one country, its outcomes may not be generalizable to other companies or places. Future research should therefore explore additional cases. Second, only 27 vehicle-maintenance plants were utilized in the experiment, which lasted only five months, which may not have been sufficient to verify the reliability and validity of the model. Third, among a number of machine learning-based data-analytic models including neural networks, SVMs, nearest neighbor, and so forth, only decision trees were used, and this could further limit the applicability of the analytical results. And fourth, the experimental method and the methods of evaluating model performance were both designed specifically for this case, meaning that applications of a similar approach in studies of other firms will require adaptations.. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 21. i n U. v.

(30) References Ali A., Morteza S., & Zahra J. (2012). An intelligent decision support system for forecasting and optimization of complex personnel attributes in a large bank. Expert Systems with Applications, 39, 12358–12370. Almotairi, M. (2008, May). CRM success factors taxonomy. European and Mediterranean Conference on Information Systems, Dubai, UAE. Alpaydin, E. (2004). Introduction to machine learning. London, England: MIT Press. Alt, R., & Puschmann, T. (2004, January). Successful practices in customer relationship management. Proceedings of the 37th Hawaii International Conference on System Sciences 2004, Big Island, Hawaii, USA. Baroudi, R. (2014). KPIs: Winning tips and common challenges. Performance, 6(2), 36-43. Bhattacharyya, S., Jha, S., Tharakunnel, K., & Westland, J. C. (2011). Data mining for credit card fraud: A comparative study. Decision Support Systems, 50, 602-613. Bose, I., & Mahapatra, R. K., (2001). Business data mining – a machine learning perspective.. 立. 政治大. ‧ 國. 學. Information & Management, 39, 211-225.. ‧. Bryman, A., & Bell, E. (2011). Business research methods (3rd ed.). Oxford, England: Oxford University Press. Campbell, A. J. (2003). Creating customer knowledge competence: Managing customer relationship management programs strategically. Industrial Marketing Management, 32, 375-383. Chen, Q., & Chen, H.-m. (2004). Exploring the success factors of eCRM strategies in practice. Database Marketing & Customer Strategy Management, 11(4), 333-343. Coghlan, D., & Brannick, T. (2001), Doing action research in your own organization. London:. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. Sage. Coughlan, P., & Coghlan, D. (2002). Action research for operations management. International Journal of Operations & Production Management, 22(2), 220-240. Coussement, K., Lessmann, S., & Verstraeten, G. (2017). A comparative analysis of data preparation algorithms for customer churn prediction: A case study in the telecommunication industry. Decision Support Systems, 95, 27-36. Croteau, A.-M., & Li, P. (2003). Critical success factors for CRM technological initiatives. Canadian Journal of Administrative Sciences, 20(1), 21-34. El-Zehery, A. M., El-Bakry, H. M., & El-Kasasy, M. S. (2013). Applying data mining techniques for customer relationship management: A survey. International Journal of Computer Science and Information Security, 11(11), 76-82. Faggella, D. (2016). What is machine learning? https://www.techemergence.com/what-is-machine-learning/, accessed 30 September 2017. 22.

(31) Fürnkranz, J., Gamberger, D., & Lavrač, N. (2012). Foundations of rule learning. Berlin Heidelberg: Springer. Gualtieri, M., & Curran, R. (2015). A machine learning primer for BT professionals. Cambridge, England: Forrester Research. Ince, T., & Bowen, D. (2011). Consumer satisfaction and services: Insights from dive tourism. Service Industry Journal, 31(11), 1769-1792. Iriana, R., & Buttle, F. (2007). Strategic, operational, and analytical customer relationship management: Attributes and measures. Journal of Relationship Marketing, 5(4), 23-42. Jones, S., Johnstone, D., & Wilson, R. (2015). An empirical evaluation of the performance of binary classifiers in the prediction of credit ratings changes. Journal of Banking & Finance, 56, 72-85. Jones, S., Johnstone, D., & Wilson, R. (2017). Predicting corporate bankruptcy: An evaluation of alternative statistical frameworks. Journal of Business Finance & Accounting, 44(1-2), 3-34.. 立. 政治大. King, S. F. &, Burgees, T. F. (2008). Understanding success and failure in customer. ‧ 國. 學. ‧. relationship management. Industrial Marketing Management, 37, 421-431. Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. Informatica, 31, 249-268. Kracklauer, A. H., Mills, D. Q., & Seifert, D. (2004). Customer management as the origin of collaborative customer relationship management. Collaborative Customer Relationship Management - taking CRM to the next level, 3–6. Krishna, G., & Vadlamani, R. (2016). Evolutionary computing applied to customer relationship management: A survey. Engineering Applications of Artificial Intelligence, 56, 30-59.. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. Larivie, B., & Van den Poel, D. (2005). Predicting customer retention and profitability by using random forests and regression forests techniques. Expert Systems with Applications, 29, 472-484. Lewin, K. (1946). Action Research and Minority Problems. Journal of Social Issues, 2(4), 34-46. Liu, H., & Yu, L. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4), 491-502. Maestrini, V., Luzzini, D., Shani, A. B., & Canterino, F. (2016). The action research cycle reloaded: Conducting action research across buyer-supplier relationships. Journal of Purchasing & Supply Management, 22, 289-298. Martens, D., Vanthienen, J., Verbeke, W., & Baesens, B. (2011). Performance of classification models from a user perspective. Decision Support Systems, 51(4), 782-793. 23.

(32) Malik, F. (2013). Application of data mining in changing times and its role in future. Indian Journal of Commerce & Management Studies, 4(1), 73-77. Microsoft Azure (2017). Two-class boosted decision tree. Retrieved October 14, 2017, from https://msdn.microsoft.com/en-us/library/azure/dn906025.aspx Mitchell, T. M. (1997). Machine learning. New York City, United States: McGraw-Hill. Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foundations of machine learning. Cambridge, MA and London: MIT Press. Murphy, K. P. (2012). Machine learning: A probabilistic perspective. London, England: MIT Press. Ngai, E.W.T., Xiu, L, & Chau, D.C.K. (2009). Application of data mining techniques in customer relationship management: A literature review and classification. Expert Systems with Applications, 36(2), 2592-2602. Olafsson, S., Li, X., & Wu, S. (2008). Operations research and data mining. European Journal of Operational Research, 187, 1429-1448. Özden Gür Ali, & Umut Arıtürk. (2014). Dynamic churn prediction framework with more effective use of rare event data: The case of private banking. Expert Systems with. 政治大. 學. ‧ 國. 立. Applications, 41, 7889–7903.. ‧. Pan, Z., Ryu, H., & Baik, J. (2007, August). A case study: CRM adoption success factor analysis and six sigma DMAIC application. Fifth International Conference on Software Engineering Research, Management and Applications, Busan, South Korea. Parvatiyar, A., & Sheth, J. N. (2001). Customer relationship management: Emerging practice, process, and discipline. Journal of Economic and Social Research, 3(2), 1-34. Patidar, P., & Tiwari, A. (2013). Handling missing value in decision tree algorithm. International Journal of Computer Applications, 70(13), 31-36. Prinzie, A., & Van den Poel, D. (2008). Random forests for multiclass classification Random. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. MultiNomial Logit. Expert Systems with Applications, 34, 1721-1732. Samuel, Arthur L. (1959). Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 3(3), 210-229. SAS (2017). Machine learning: What it is and why it matters. Retrieved September 14, 2017, from https://www.sas.com/it_it/insights/analytics/machine-learning.html Swift, R. S. (2001). Accelerating customer relationships: Using CRM and relationship technologies. Upper Saddle River, NJ: Prentice Hall PTR. Wei, J.-T., Lee, M.-C., Chen, H.-K., & Wu, H.-H. (2013). Customer relationship management in the hairdressing industry: An application of data mining techniques. Expert Systems with Applications, 40, 7513-7518. Chiang, W. Y. (2012). To establish online shoppers’ markets and rules for dynamic CRM systems: An empirical case study in Taiwan. Internet Research, 22(5), 613-625.. 24.

(33) West, P. M., Brockett, P. L., & Golden, L. L. (1997). A comparative analysis of neural networks and statistical methods for predicting consumer choice. Marketing Science, 14(4), 370-391. Xiao, S. H., & Nicholson, M. (2011). Mapping impulse buying: A behaviour analysis framework for services marketing and consumer research. Service Industry Journal, 31(15), 2515-2528.. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 25. i n U. v.

(34) Appendix A. Information Attribute Inputted for Decision Tree Model Customer Information Attribute. Notes. I.D Card No. of the new license plate received person I.D Card No. of the car user I.D Card No. of the invoice recipient I.D Card No. of the contact person I.D Card No. of the maintenance service contact person Number of cars owned by new license plate received person Number of cars owned by car user Number of cars owned by invoice received person Number of cars owned by contact person. 政治大. Number of cars owned by maintenance service contact person Postal code of new license plate received person. 立. Postal code of car user. ‧ 國. 學. Postal code of invoice received person Postal code of contact person Postal code of maintenance service contact person. ‧. Gender of new license plate received person Gender of car user. y. Nat. sit. Gender of invoice received person. io. Gender of maintenance service contact person. n. al. Age of new license plate received person Age of car user Age of invoice received person. Ch. engchi. er. Gender of contact person. i n U. v. Age of contact person Age of maintenance service contact person Number of complaints. From telephone survey. Car Information Attribute. Notes. License Plate Number Car Brand Car Name Car Type Car classification. 1.Private 2.Business 26.

(35) 3.Rental Car Dealer Name Car Delivery Date The number of days passed since the Car Delivery Date Driving rate per 10,000 kilometers. L: Low, H: High. Average kilometers driven per year Maintenance Information Attribute. Notes. Customer designated maintenance plants Latest maintenance date Latest maintenance plants Latest maintenance recorded odometer. 政治大. Latest panel-beating or car-paintng date. Latest panel-beating or car-paintng plants. 立. Latest panel-beating or car-paintng recorded odometer. ‧ 國. 學. Latest regular maintenance date Latest regular maintenance plants Latest regular maintenance recorded odometer. ‧. Regular maintenance reminder. sit. n. al. er. io. Contact person. y. Nat. E: Email S: SMS T: Telephone D: DM. Ch. engchi. Maintenance frequency. i n U. v. 1. New license plate received person 2. Car user 3. Invoice received person 4. Contact person 1. Normal 2. Every 5,000 kilometers 3. Every 10,000 kilometers 4. Twice a year. Oil type. 1. Bring own oil 2. Taxi oil 3. Mineral oil 4. Synthetic oil 27.

(36) Number of panel-beating and car-paintng Panel-beating and car-paintng revenue Regular maintenance every year or not. Y: Yes, N: No. Latest maintenance date over 1 year but within 2 years. Y: Yes, N: No. The number of days passed since the latest panel-beating or car-paintng date The number of days passed since the latest regular maintenance date Next regular maintenance date The number of days passed since the next regular maintenance date Number of online reservations for maintenance by app. 政治大. Number of online reservations for maintenance by web page Number of maintenance times due to inducing. 立. Number of reservations due to inducing. ‧ 國. 學. Number of times for failed inducing Number of times for successful inducing Number of times for postponed reminder. ‧. Number of times for failed reminder Number of times for successful reminder. Nat. sit. n. al. er. io. 1. Every 1,000 kilometers 2. Every 10,000 kilometers. y. Maintenance reservation type. Ch. engchi. Maintenance reservation source. i n U. v. 3. Normal maintenance 4. Panel-beating 5. Car-paintng 6. Others 1. Induced by maintenance plants 2. Induced by agents 3. Reserved by customer 4. Reserved by sales 5. Online reservation. 28.

(37) Product Information Attribute. Notes. Qualified to purchase product A. Y: Yes, N: No (The warranty period is within 4 years, or the odometer is under 120,000 kilometers). Average usage of customer's reward points per year Have purchased product B or not. Y: Yes, N: No. Product B purchased date Product B sold car dealer name Product B sold maintenance plants The number of days passed since product B sold. 政治大. Recorded odometer when product B sold. Age of new license plate received person when product B sold. 立. Age of car user when product B sold. ‧ 國. 學. Age of invoice received person when product B sold Age of contact person when product B sold. Age of maintenance service contact person when product B sold. ‧. The sum of used customer's reward points before purchased product B. Nat. er. n. al. Insurance Information Attribute. Ch. Car damage insurance expired date. sit. io. The number of purchased times for product B. y. The sum of total customer's reward points before purchased product B. engchi. i n U. v. The number of days passed since the car damage insurance expired date Duration of the car insurance Total premium for the car insurance Annual premium for the car insurance Duration of the car damage insurance Number of purchase times of the car damage insurance Total premium for the car damage insurance Annual premium for the car damage insurance Duration of the accident insurance Number of purchase times of the accident insurance 29. Notes.

(38) Total premium for the accident insurance Annual premium for the accident insurance Duration of the passenger insurance Number of purchase times of the passenger insurance Total premium for the passenger insurance Annual premium for the passenger insurance Duration of the compulsory insurance Number of purchase times of the compulsory insurance Total premium for the compulsory insurance Annual premium for the compulsory insurance Duration of the life insurance Number of purchase times of the life insurance Total premium for the life insurance Annual premium for the life insurance. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 30. i n U. v.

(39) Appendix B. Important Variables from Decision Tree Regression Results Important Variables for Product A. Correlation Coefficient. Annual premium for the Compulsory insurance. 5.20%. Total Premium for the Compulsory insurance. 1.50%. Average annual usage of customer's reward points. 0.71%. Number of purchase times of the compulsory insurance. -0.66%. Annual premium for the car Damage insurance. 0.50%. Average miles driven per year. -0.49%. Gender of New License Plate Received Person: Null. -0.37%. Total Premium for the car insurance. 0.32%. 治政kilometers Maintenance reservation type: Every 1,000 大 Number of Panel-beating and Car-paintng times 立 Keep Regular Maintenance every year or not: Yes. -0.24% 0.23% 0.22%. Number of purchase times of the accident insurance. ‧ 國. 學. Maintenance reservation type: Every 10,000 kilometers Car Name: S. -0.21% 0.19% 0.19%. ‧. Gender of New License Plate Received Person: Female. Nat. Correlation Coefficient. sit. y. Important Variables for Product B. n. er. io. Number of online reservation times for Maintenance by Web Page. al. Ch. 0.16%. Customer Designated Maintenance Plants: C. engchi. Gender of Maintenance Service Contact Person: Female Number of Panel-beating and Car-paintng times Gender of New License Plate Received Person: Null. i n U. v. 13% 0.14% -0.11% 0.084% -0.056%. Customer Designated Maintenance Plants: D. 0.028%. Driving frequency per 10,000 kilometers: Low. 0.028%. Car classification: Null. 0.028%. 31.

(40)