應用類神經網路於學生微型信貸 - 政大學術集成

全文

(1)國立政治大學資訊管理學研究所碩士學位論文. 應用類神經網路於學生微型信貸. Application of Artificial Neural 治 Networks to Student. 立. 政. Microfinance. 大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. 指導教授：蔡瑞煌博士研究生：陳韋翰撰. 中華民國 107 年 07 月 DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(2) 摘要. 普惠金融現在被視為金融業的重要領域，而小額信貸是普惠金融的基本形式。學生族群是處於金融領域的弱勢群體。人工神經網路是機器學習系統的其中之一。它具有學習能力，並且可以進一步推廣所預測的結果，它也適用於非線性問題的應用。這項研究調整了蔡瑞煌教授以及吳佳真研究生的研究，以推導有效的異常值檢測和機器學習機制。使用 GPU 設備和機器學習工具建立神經網絡系統藉由. 政治大. TensorFlow 實現。我們基於在線 P2P 借貸平台收集的真實數據集進行實驗。從. 立. 2018/3/30〜2018/4/7 中收集到 200 個學生的貸款數據，隨機選取 140 個數據做. ‧ 國. 學. 訓練，60 個數據作為測試集。結果表明，所提出的機制在異常值檢測和機器學習方面是有前途的且有效果的。. ‧ y. Nat. n. al. er. io. sit. 關鍵詞：微型信貸、普惠金融、人工神經網路、異常值檢測、機器學習. Ch. engchi. 1. i n U. v. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(3) Abstract. Inclusive Finance is regarded as an important area of financial industry now day, and microfinance is a basic form of Inclusive Finance. Student group is an underprivileged group in financial field. Artificial Neural Networks is one of machine learning systems. It has the ability to learn, and it can further generalize the results to be predicted, and it is also suitable for applications in nonlinear problems. This study adapts the work of Tsaih and Wu (2017) to derive a mechanism for. 政治大. effective outlier detection and machine learning. To establish a neural network system. 立. using GPU equipment and machine learning tools - TensorFlow implementation. We. ‧ 國. 學. sets up an experiment based on real dataset collected by online P2P Lending platform. We collect 200 students’ loan data from 2018/3/30~2018/4/7, then randomly choosing. ‧. 140 data to do training, 60 data to be the testing set. The results show that the proposed. y. Nat. n. al. er. io. sit. mechanism is promising in outlier detection and machine learning.. i n U. v. Index Terms — microfinance, Inclusive Finance, Artificial neural networks, outlier detection, machine learning. Ch. engchi. 2. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(4) Index 1. INTRODUCTION ..................................................................................................... 4 1.1 Background & Motivation ............................................................................... 6 1.2 Purpose............................................................................................................. 9 2. LITERATURE REVIEW ......................................................................................... 11 2.1 P2P Lending ................................................................................................... 11 2.2 Students P2P Lending variables ..................................................................... 13 2.2.1 Student characteristics and background .............................................. 13 2.2.2 Academic performance ....................................................................... 14 2.2.3 Socioeconomic contexts...................................................................... 14. 政治大 2.3.1 Single-Hidden 立Layer Feedforward Neural Networks (SLFN)............ 15. 2.3 Artificial Neural Networks ............................................................................. 15. ‧ 國. 學. 2.3.2 The Resistant Learning with Envelope Module (RLEM) ................... 16 2.4 TensorFlow & GPU ....................................................................................... 19 2.4.1 TensorFlow.......................................................................................... 19. ‧. 2.4.2 GPU..................................................................................................... 22. y. Nat. 3. EXPERIMENT ........................................................................................................ 24. sit. 3.1 Variables Description ..................................................................................... 25. er. io. 3.1.1 Variables selection .............................................................................. 25. al. n. v i n C h ...................................................................... 3.2 ANN for students credit scores 30 engchi U 3.1.2 Data preprocessing .............................................................................. 26. 4. EXPERIMENT RESULTS ...................................................................................... 33 4.1 Results and discussion ................................................................................... 33 5. Conclusions and Future Works ................................................................................ 44 5.1 Conclusions .................................................................................................... 44 5.2 Future works .................................................................................................. 45 REFERENCE ............................................................................................................... 46 English Reference ................................................................................................ 46 Chinese Reference ............................................................................................... 48. 3. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(5) Table Index Table 1: P2P platform mode (李坤霖, 2017) ............................................................... 11 Table 2: The resistant learning with envelope module (Huang et al., 2014) ............... 18 Table 3: The proposed mechanism implemented by Wu (2017) ................................. 19 Table 4: The Experiment Environment ........................................................................ 24 Table 5: The data used in this experiment ................................................................... 27. 政治大 Table 7: Dummy variables data description statistics .................................................. 29 立 Table 6: Non-dummy variables data description statistics .......................................... 28. Table 8: Variables using in this study .......................................................................... 30. ‧ 國. 學. Table 9: The proposed mechanism .............................................................................. 30. ‧. Table 10: The training data and number of hidden nodes in different experiment. ..... 34. y. Nat. Table 11: The training time spent in different experiment ........................................... 34. er. io. sit. Table 12: Deviation between desired and predicted credit score in different experiment ............................................................................................................................. 40. al. n. v i n C hprediction in different Table 13: Accuracy of credit score experiment ....................... 41 engchi U Table 14: Accuracy of credit score prediction in different experiment ....................... 42 Table 15: Accuracy of credit score prediction in different experiment ....................... 43. 4. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(6) Figure Index Figure 1: TensorFlow operation examples ................................................................... 21 Figure 2: Example TensorFlow code fragment via Python .......................................... 22 Figure 3: Corresponding computation graph for Figure 2 ........................................... 22 Figure 4: The ANN of the students’ microfinance ....................................................... 32 Figure 5: The distribution of credit score rating groups .............................................. 33 Figure 6: The envelope module of the 1st RL to identify outliers in 1st experiment 35 Figure 7: The envelope module of the 1st RL to identify outliers in 1st experiment .. 35. 政治大. Figure 8: Desired and predicted credit score in 2nd RL in 1st experiment. 立. ................ 36. Figure 9: Desired and predicted credit score in 2nd RL in 1st experiment .................... 36. ‧ 國. 學. Figure 10: The envelope module of the 1st RL to identify outliers in 2nd experiment ...................................................................................................................................... 37. ‧. Figure 11: The envelope module of the 1st RL to identify outliers in 2nd experiment 37. y. Nat. sit. Figure 12: Desired and predicted credit score in 2nd RL in 2nd experiment ................. 37. n. al. er. io. Figure 13: Desired and predicted credit score in 2nd RL in 2nd experiment ................. 38. i n U. v. Figure 14: The envelope module of the 1st RL to identify outliers in 3rd experiment 39. Ch. engchi. Figure 15: The envelope module of the 1st RL to identify outliers in 3rd experiment . 39 Figure 16: Desired and predicted credit score in 2nd RL in 3rd experiment ................. 40 Figure 17: Desired and predicted credit score in 2nd RL in 3rd experiment ................. 40. 5. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(7) 1. INTRODUCTION 1.1 Background & Motivation According to the "The World Bank" survey, there are still 2 billion people in the world who still have no access to banking services. Therefore, Inclusive Finance is also regarded as a potential area of financial industry.1 The United Nations defines Inclusive Finance as a financial system that serves all sectors of society in an effective and comprehensive manner in “Financial Inclusion. 政治大 of financial services at reasonable cost. Second, stable financial institutions require tight 立. Systems”. Mainly includes four aspects: First, families and businesses get a wider range. internal control, market supervision. Third, the financial industry achieves sustainable. ‧ 國. 學. development and ensures long-term financial services. Fourth, to enhance the. ‧. competitiveness of financial services to provide consumers with a variety of options.2. y. Nat. The initial basic forms of Inclusive Finance are microcredit and microfinance.. er. io. sit. Over the last two decades, microfinance has been receiving increasing global acceptance as a social inclusion mechanism and a poverty-alleviating development. al. n. v i n C h this paper, microfinance strategy. In the definition used throughout is related to access engchi U. to financial services by low-income people, mainly those who are often excluded from the traditional financial system (Lavoie et al., 2011). Among several definitions of microcredit found in the literature, we adopted that which refers to programs that extend small loans to poor people for self-employment projects that generate income (Waller and Woodworth, 2001). Therefore, microcredit is an important and integral part of microfinance. Peer-to-peer Lending (P2P Lending) is a one of the microfinance service that UFA2020 Overview: Universal Financial Access by 2020 : http://www.worldbank.org/en/topic/financialinclusion/overview 2 NPF Commentary May 28,2015 : https://www.npf.org.tw/printfriendly/15076 6 1. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(8) matches funding provider and demander through the electronic marketplace. Because of the cost eliminating of intermediaries by traditional financial, making the supply of funds and demand match more efficient. P2P Lending let fund supplier get higher income and demander need to pay less cost. Student P2P Lending has actually been done for years. For instance, Chinese financial technology company Qudian Inc. (趣店)3 which is a financial technology company for 500 million non-credit card groups in China. This company's predecessor was 趣分期. 趣分期 is a financial services platform that provides college students. 政治大 Provide services such as staging and cash consumption, support for installments. 立. with phased consumption, covering nearly 30 million college students across China. 4. However, on September 5, 2016 趣分期 was halted by Chinese regulatory. ‧ 國. 學. authorities because of the chaotic development in the past few years. Declared to the. ‧. outside that they provide installments or lending cash services to school students and. y. Nat. low-income groups. In fact, providing high interest rates on loans.. er. io. sit. Generally, students who seek loan service are usually cannot effectively control their desire, at the same time there is no stable source of income. Therefore, making a. al. n. v i n C h P2P Lending loans more accurate credit score for student is more important than any engchi U. other P2P Lending. On the other hand, although there are many literatures about P2P Lending in domestic and foreign, the literatures about P2P Lending which focus on student groups are still almost empty. It is still a field with much research value. Artificial Neural Networks (ANN) is one of machine learning systems. ANN are now used in many fields. ANN have been studied for many years. These models of ANN are primarily intended to mimic the human nervous system. ANN are composed of many non-linear computing units (Neuron) and many links between these computing. 3 4. Qudian Inc. : http://www.qudian.com.cn/ Chinatimes.com : http://www.chinatimes.com/realtimenews/20171030003488-260410 7. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(9) units. These computing units are usually operated in a parallel and decentralized way, so that a large amount of data can be processed at the same time. This design can be used to deal with a variety of applications that require a large amount of data operations. Because of the computational complexity of deep learning, before the 1990s, deep learning was limited by hardware and each training was costed normally several weeks or months to complete. Under the era of big data, in order to meet these data computing, processing needs. In 2015, Google released its open source machine learning system, TensorFlow. Of course, TensorFlow is not the only one, there are lots of similarities.. 政治大 Lee Sedol in a five-game match, the first time a computer Go program has beaten a 9立 But, the artificial intelligence program which google developed, AlphaGo has beaten. dan professional without handicaps. It makes TensorFlow be the most famous and. ‧ 國. 學. successful one.. ‧. TensorFlow is an interface that allows people to design machine learning. y. Nat. algorithms easier and really execute such algorithms. It can operates at large scale and. er. io. sit. in heterogeneous environments, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of. al. n. v i n C hcards (Abadi et al.,U2016). computational devices such as GPU engchi. TensorFlow describes computation, shared state, and the operations that mutate. that state by dataflow graphs. It can be used in model setting, training, and testing for ANN or deep learning. It can map the nodes of a dataflow graph across many different hardware in a cluster, and within a machine across multiple computational devices, including multicore CPUs, general purpose GPUs. (Abadi et al., 2016). For relatively complex processing using in ANN, the GPU version is better than CPU version because of the parallel operation. On the other hands, Lian (2017) using ANN model to detect malicious behaviors on the Internet, Wu (2017) derives an ANN mechanism in predicting the return of carry 8. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(10) trade. Trying to implement an ANN model in creating credit score for student groups is a brand new research field and a topic with research value. Furthermore, because of TensorFlow and GPU, the implementation and training of ANN model is more feasible and timeliness.. 1.2 Purpose Based on the motivations and background mentioned above, the objective of this study is to develop an ANN model that is used to create a credit score for students who. 政治大. try to borrow money but do not have qualifications to access to banking services on a. 立. P2P Lending platform.. ‧ 國. 學. To achieve the objective, we collect the real data of students P2P loans from our website5, which is a student P2P Lending platform on Internet. Moreover, the ANN. ‧. model is developed by the infrastructure of TensorFlow and GPU. Therefore, we can. sit. y. Nat. finish the training in few hours. After the training process, hoping that the effective. al. er. io. ANN model is able to give the correct credit score for students. In the future, the more. v. n. data used, the more accurate credit score will be created by the model.. Ch. engchi. i n U. This is an experimental research, which combines machine learning and P2P Lending. The ANN mechanism is refer to the mechanism developed by Wu (2017). This research is not the same as traditional credit method used in P2P Lending field. Hope this research could contribute in both ANN development and students P2P Lending. This experiment is also designed to verify the timeliness and effectiveness of the derived ANN mechanism:. . 5. Effectiveness: In order to effectively determine whether to borrow money, we hope. Website of platform : https://studenloan-197508.appspot.com/feedback_done 9. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(11) ANN model can give a credit score as accurate as possible. Timeliness: Demanders on P2P Lending are often very urgent to need money, so we want to establish an ANN model in hours. In the other words, the training time of ANN model have to be finished in hours.. 立. 政治大. 學 ‧. ‧ 國 io. sit. y. Nat. n. al. er. . Ch. engchi. i n U. v. 10. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(12) 2. LITERATURE REVIEW 2.1 P2P Lending P2P Lending is an electronic lending platform, belongs to a part of the electronic marketplaces. According to Investopedia, it is a form of financial lending that takes away the role of traditional financial institutions as intermediaries and uses platforms to tie in with borrowers and lenders of financial demand. With the appearance of the Internet, electronic marketplaces are becoming more important in matching supply and. 政治大 promote economic activity, greatly reduce information and transaction costs, and may 立 demand. Even in complex and insecure conditions, electronic marketplaces can. replace traditional intermediaries in this way. Authors like Sarkar, Butler, and Stein field. ‧ 國. 學. (1998) or Hagel and Singer (1999) argue that electronic marketplaces may lead to new. ‧. forms of intermediation (Berger & Gleisner, 2010).. y. Nat. Organizing from available data and 李坤霖 (2017) pointed out that according to. er. io. sit. the degree of P2P Lending risk exposure, we can divide the business model into the following three mode: Information intermediary mode, Platform guarantee mode and. n. al. Third party guarantee mode.. Ch. engchi. i n U. v. Table 1: P2P platform mode (李坤霖, 2017). P2P platform mode. Platform features. Example company. Information. Platform is only responsible for the. SoFi, Future. intermediary mode. examine and match the demand & supply.. Finance , 逗. After the success of the match, investors. 派. need to take their own risk. Platform guarantee. Platform draws management fees and. 人人貸. 11. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(13) mode. reserve fees from profit.. Third party. Third party as a guarantee, if the borrower. guarantee mode. fails to repay, you can apply to the third. 陸金所. party for compliance.. Compared to the other two modes, Information intermediary mode can save more money than the traditional financial service model by decentralization and then got more profit. Therefore, Information intermediary model is currently the most choice for. 政治大 Future Finance mainly provide tuition and living expenses loans to undergraduate 立. P2P Lending.. ‧ 國. 學. students. It conducts risk control mainly through big data technology. On the one hand,. it analyzes the student's credit situation. On the other hands, it combines the students'. sit. y. Nat. repayment ability.. ‧. schools and majors to forecast the students' income after graduation to assess their. io. er. Founded in September 2011, Social Finance (SoFi) offers low-interest student loans to top students. Because of this SoFi segmentation is very strong, only to make. n. al. i loans to elite students, to ensureC thehquality of assets. U n engchi. v. Stiligtz (1981) believes that information is the key factor affecting the operation. of financial markets. The level of information transparency will cause the differences between P2P Lending market and traditional lending market. In traditional lending market, financial institutions have the right to decide whether to lend money. Compared to the traditional lending market, personal lenders decide whether to lend money. Therefore, borrowers have more choices and information disclosure degree is more flexible. Using data from Prosper.com, Freedman et al. (2008) found that Prosper.com uses a more complete set of data exposures to reduce the risk of asymmetry information. 12. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(14) Hampshire, Robert （2008）, Funk et al（2011） are all mentioned in their essay that P2P Lending has soft information, which included Participating communities, platform activities and so on. And the soft information is relevant to the status of repayment. All the above studies have shown that how to solve information asymmetry and find an affective credit model will be an important research topic in P2P Lending.. 2.2 Students P2P Lending variables Although there is a very small number of P2P Lending studies on specific student ethnicity, there has been a considerable amount of researches on traditional student loan. 政治大 variables and P2P Lending立 variables. In this section, we will review the literature on. variables. However, there is a very large correlation between traditional student loans. ‧ 國. 學. the selection of traditional student loan variables to conduct the subsequent selection of our student P2P Lending variables.. ‧. From the literatures of traditional student loans, we can classify traditional student. sit. y. Nat. loan variables into three categories: student characteristics and background、academic. al. er. io. performance socioeconomic contexts. The three categories are supported by the. n. researches, we will introduce each below.. i n U. C. v. h e nand 2.2.1 Student characteristics hi g cbackground Age. Almost all studies that consider the age of students, concluded that the possibility of default on loans increases with age, even after controlling for other important factors such as income (Christman, 2000; Harrast, 2004; Herr & Burt, 2005). Herr and Burt (2005) argue that older students may have greater financial responsibility and may be in conflict with repayments of loans or refuse to repay loans, such as families to support. Race/ethnicity. Practically, race/ethnicity become one of the strongest predictors of default (Harrast, 2004). For example, a study conducted in a traditional four-year 13. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(15) public agency found that race/ethnicity explained about 20% of loan default differences, second only to degree completion, which is about 26% (Herr & Burt, 2005).. 2.2.2 Academic performance Program of study. According to the coherent study, students studying at school seem to influence the probability of default in at least two ways - the number of debt incurring and post-graduation income. There is a lot of evidence that the graduate income associated with a research area affects personal income, thus affecting the ability of individuals to repay their loans (Herr & Burt, 2005; Volkwein & Szelest,. 政治大 Academic preparation. It is not surprising that academic preparation such as high 立. 1995).. school rank, GPA, standardized test scores is strongly related to default. There is a. ‧ 國. 學. relationship between degree completion and likelihood of default, according to these. y. Nat. loans.. ‧. traditional measures, academically prepared students are less likely to default on their. er. io. sit. Educational attainment. Achieving intermediate and tertiary education levels may be the strongest predictor of loan defaults. Most of the studies we reviewed show that. al. n. v i n C h is the strongest single completing a postsecondary program predictor, regardless of the engchi U type of institution (Volkwein et al., 1998; Gross, Jacob PK, 2009). There is also a. founding that students who enroll continuously, enroll in more rather than fewer credit hours, complete their attempted, and graduate within eight semesters are less prone to default on average (Gross, Jacob PK, 2009).. 2.2.3 Socioeconomic contexts Parental Education. We can easily understand that, given the close positive relationship between education and socioeconomic status, those parents who receive higher levels of formal education are less likely to default than first-generation college students are (Volkwein et al., 1998). 14. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(16) Family structure. Family structure affects the likelihood of a student loan default in many ways. First, the more family members students claimed, the more likely the loan will default (Volkwein & Szelest, 1995). Volkwein and Szelest (1995) found that the probability of default for every dependent child in a household increased by 4.5%. Being a single parent was also associated with a greater risk of loan default (Volkwein et al., 1998). Being separated, divorced, or widowed was found to increase the probability of defaulting by more than 7 percent (Volkwein & Szelest, 1995). The last way that family can affect student loan default is by providing a pocket money. Students. 政治大 family support (Volkwein et al., 1998). 立. who can rely on family support are less likely to default than those who do not have. Income. As common sense, students from low-income families tend to incur more. ‧ 國. 學. debt than their wealthier peers during school (Herr & Burt, 2005). Therefore, the higher. ‧. the family income, the lower the probability of a student default. Families with more. y. Nat. money can provide a financial safety net for students than low-income families. This. er. io. sit. financial safety net also helps students meet their loan obligations through variances in personal income (Gross, Jacob PK, 2009).. n. al. Ch 2.3 Artificial Neural Networks en. gchi. i n U. v. 2.3.1 Single-Hidden Layer Feedforward Neural Networks (SLFN) To handle anomaly detection and resistance learning, Tsaih and Cheng (2009) implemented an adaptive SLFN to solve this problem. The fitting function of SLFN is defined as: 𝑚. 𝑎𝑖 (𝑥) =. 𝐻 tanh (𝑤𝑖0. + ∑ 𝑤𝑖𝑗𝐻 𝑥𝑗 ) 𝑗=1. 15. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(17) 𝑝. 𝑤0𝑜. 𝑓(x) ≡. 𝑒 𝑥 − 𝑒 −𝑥. where tanh(𝑥) ≡ 𝑒 𝑥 +. +. 𝑚. ∑ 𝑤𝑖𝑜 𝑖=1. 𝐻 tanh (𝑤𝑖0. + ∑ 𝑤𝑖𝑗𝐻 𝑥𝑗 ) 𝑗=1. , m is the number of explanatory variables (𝑥𝑗 ) ; 𝐱 ≡. 𝑒 −𝑥. (𝒙𝟏 , 𝒙𝟐 , … , 𝒙𝒎 )T; p is the adaptive number of adopted hidden nodes; 𝑤𝑖𝑜 is the bias value of the 𝑖 𝑡ℎ hidden node; the superscript H throughout the paper refers to quantities related to the hidden layer; 𝑤𝑖𝑗𝐻 is the weight between the 𝑗 𝑡ℎ explanatory variable 𝑥𝑗 and the 𝑖 𝑡ℎ hidden node; 𝑤0𝑜 is the bias value of the output node; the superscript o throughout the paper refers to quantities related to the output layer; and. 政治大. 𝑤𝑖𝑜 is the weight between the 𝑖 𝑡ℎ hidden node and the output node. In their study, a. 立. character in bold represents a column vector, a matrix, or a set, and the superscript T. ‧ 國. 學. indicates the transposition.. ‧. Through this SLFN, the input information x is first transformed into ≡ 𝑇. sit. y. Nat. (𝑎1 , 𝑎2 , … , 𝑎𝑝 ) , and the corresponding value of f is generated by a rather than x.. al. n. 𝐻 calculated with 𝑎𝑖 ≡ tanh(𝑤𝑖0. er. io. Scilicet, given the observation, all the corresponding values of hidden nodes are first. v. 𝐻 + ∑𝑚 𝑗=1 𝑤𝑖𝑗 𝑥𝑗 ) for all I, and the corresponding. Ch. engchi. i n U𝑜. value 𝑓(x) is then calculated as 𝑓(x) = 𝑔(𝑎) ≡ 𝑤0 + ∑𝑝𝑖=1 𝑤𝑖𝑜 𝑎𝑖 . (Tsaih and Lian, 2017).. 2.3.2 The Resistant Learning with Envelope Module (RLEM) Tsaih and Cheng (2009) proposed a resistant learning outlier detection mechanism with the SLFN and a tiny pre-specified value as 10-6 to deduce a function form. The mechanism dynamically adapts the number of the hidden nodes and the relative weights of SLFN during the training process. By the way, they also implemented both robustness analysis and deletion diagnostics to exclude potential outliers at the early stage, thus prevent the SLFN from learning them (Rousseeuw and Driessen, 2006). 16. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(18) Above all, the weight-tuning mechanism, the recruiting mechanism, and the reasoning mechanism are implemented to allow the SLFN to evolve dynamically during the learning process and to explore the non-linear relationship that is acceptable between explaining the variables and the response in the presence of outliers. Huang et al. (2014) propose an envelope bulk mechanism integrated with the SLFN to handle outlier detection problem. This outlier detection algorithm is performed with an envelope bulk whose half width is 2ε. The ε is changed from a tiny value (10−6) to a non-tiny value (1.96) due to the envelope module. The value changes to 1.96. 政治大 The standard to distinguish whether the instance is outlier or not is the instance’s 立 similarly according to that the 5% significance level in given the distribution is normal.. residual is greater than ε ∗ γ ∗ σ, where σ is the standard deviation of the residual of the. ‧ 國. 學. current reference observations and γ is a constant that is equal to or greater than 1.0,. ‧. depending on the user’s stringency in the outlier detection. The smaller the γ value is,. y. Nat. the more stringent the outlier detection is. Furthermore, if our requirements are stricter,. er. io. sit. we also can modify the ε value to an appropriate value (Tsaih and Wu, 2017). In short, this envelope module allows us to encapsulate the response elements seen. al. n. v i n C h the response as U as inliers in the envelope. Vice versa, outliers won’t be wrapped in the engchi envelope. The quantity of the inliers is decided by the ε and γ. The stricter parameter is, the less inliers inside the envelope. In other aspect of outliers, there will be more potential outliers determined by the envelope module. As stated in Huang et al. (2014), the resistant learning algorithm with the envelope module in Table 2. In step 2, 𝑘 can be referred to the percentage of potential outlier, which means at least (1 − 𝑘) data will be wrapped into the envelope. For instance, if there are approximately at least 95% nonoutliers and at most 5% outliers, the SLFN will take 95% data into consideration while building the SLFN. Huang et al. (2014) proposed a resistant learning algorithm with the envelope 17. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(19) module to cope with outlier detection problem. In Table 2, N is the total amount of training cases, m is the amount of input nodes, and the width of the envelope bulk is 2ε. In step 2, k refers to the percentage of potential outliers and the potential outliers won’t be wrapped into the envelope. Table 2: The resistant learning with envelope module (Huang et al., 2014). Step 1: Use the first m+1 reference observations in the training data set to set up an acceptable SLFN estimate with one hidden node. Set n = m+2. Step 2: If n > N*(1 – k), STOP. Step 3.1: Use the obtained SLFN to calculate the squared residuals regarding all N. 治政 Step 3.2: Present the n reference observations (x , y 大 ) that are the ones with the 立 smallest n squared residuals among the current squared residuals of all N training data.. c. c. ‧ 國. 學. training data.. Step 4: If all of the smallest n squared residuals are less than ε (the envelope width),. ‧. then go to Step 7; otherwise, there is one and only one squared residual that is larger than ε.. Nat. sit. y. Step 5: Set 𝐰 ̃ = 𝐰.. er. io. Step 6: Apply the gradient descent mechanism to adjust weights w of SLFN. Use the obtained SLFN to calculate the squared residuals regarding all training. n. al. Ch. i n U. v. data. Then, either one of the following two cases occurs:. engchi. (1) If the envelope of obtained SLFN does contain at least n observations, then go to Step 7. (2) If the envelope of obtained SLFN does not contain at least n observations, then set 𝐰 = 𝐰 ̃ and apply the augmenting mechanism to add extra hidden nodes to obtain an acceptable SLFN estimate. Step 7: Implement the pruning mechanism to delete all of the potentially irrelevant hidden nodes; n + 1  n; go to Step 2. Wu (2017) present a mechanism that implements the moving window and RLEM, which is adapted from the works of Tsaih and Cheng (2009) and Huang et al. (2014). In Table 3, M is the index of the current window, N is the sample size of the training 18. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(20) block, B is the sample size of the testing block,  is the standard deviation of training data in the training block, and  represents the maximal deviation between actual and predicted outputs. Table 3: The proposed mechanism implemented by Wu (2017). Step 0: Set M as 1. Step 1.1: Apply the RLEM stated in Table 2 (with envelope width = 2) to the N training examples {(x(M-1)B+1, y(M-1)B+1+h), (x(M-1)B+2, y(M-1)B+2+h), …, (x(M1)B+N (M-1)B+N+h ,y )} to filter out Nk potential outliers and obtain an acceptable SLFN. Step 1.2: Remove the outlier candidates, and then use the SLFN obtained in Step 1.1 and the RLEM stated in Table 2 (with envelope width = 2) again to learn the remained N(1-k) training examples. Step 2: Apply the SLFN obtained in Step 1.2 to the B testing examples {(x(M1)B+N+1 (M-1)B+N+1+h ,y ), (x(M-1)B+N+2, y(M-1)B+N+2+h), …, (xMB+N, yMB+N+h)}.. 政治大. 立. Step 3: For more data, M  M+1 and GOTO Step 1.1; otherwise, STOP.. ‧ 國. 學 ‧. 2.4 TensorFlow & GPU 2.4.1 TensorFlow. y. Nat. io. sit. The Google Brain project started in 2011 to explore the use of very-large-scale. n. al. er. deep neural networks, which be used in both Google’s products and research. Based on. Ch. i n U. v. the DistBelief, first-generation scalable distributed training and inference system, they. engchi. have built TensorFlow, second-generation system for the implementation and deployment of largescale machine learning models. They have open-sourced the TensorFlow API and the reference implementation under the Apache 2.0 license for November 2015, available at www.TensorFlow.org. This could be said to be the evolution of DistBelief (Abadi et al., 2016). TensorFlow uses dataflow-like models to compute and map them to a variety of hardware platforms, from running inference on mobile device platforms like Android and iOS to simulations using a single machine containing one or more GPU cards to large-scale training systems running on hundreds of specialized machines with 19. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(21) thousands of GPUs (Abadi et al., 2016). TensorFlow uses a single dataflow graph to represent all calculations and states in a machine learning algorithm, including the individual mathematical operations, parameters and their update rules, and input preprocessing. The graph is composed of a set of vertices. In a TensorFlow graph, each vertex has zero or more inputs and zero or more outputs, and represents the instantiation of an operation. And each edge represents the output from, or input to, a vertex. We refer to the computation at vertices as operations,. 政治大 basic programming model and basic concepts of TensorFlow. (Abadi, Martín, et al 2016) 立 and the values that flow along edges as tensors. In this subsection, I will describe some. Tensors. 學. ‧ 國. . In TensorFlow, they model all data as tensors (n-dimensional arrays) with the. ‧. elements having one of a small number of primitive types, such as int32, float32, or. y. Nat. string (where string can represent arbitrary binary data). Tensors naturally represent the. er. io. sit. inputs to and results of the common mathematical operations in many machine learning algorithms. (Abadi et al., 2016). al. n. v i n C h machine learningU algorithms, connected engchi. This design is for. such as logistic. regression, ANN and so on. Connected machine learning algorithm can be expressed as a graph algorithm, Tensors from front to back in the Graph to complete the forward operation; and the residual from the back to go forward, to complete the back propagation. . Operations. An operation takes m ≥ 0 tensors as input and produces n ≥ 0 tensors as output. An operation has a named “type” (e.g., add of two tensors of type float versus add of two tensors of type int32) and may have zero or more compile-time attributes that determine its behavior. Each operation will have attributes, all attributes are established when the 20. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(22) Graph is established. The figure 1 is some of the operations that TensorFlow implements.. Figure 1: TensorFlow operation examples. 政治大 In most computations a graph is executed multiple times. Machine learning 立 . Variables. algorithm will have parameters, and the state of the parameters need to be saved. The. ‧ 國. 學. parameters in the Graph has its fixed position, not as normal data flow as normal. Thus,. ‧. Variables is implemented as a special operator in TensorFlow, which returns the. Sessions. sit. io. . er. model.. y. Nat. variable Tensor it holds, and are updated as part of the Run of the training graph for the. al. n. v i n C hthe TensorFlow system Clients programs interact with by creating a Session. In engchi U. normal mode, a session is created and an empty Graph is created; nodes and edges are. added to the session to form a Graph and then run. Using the arguments to Run, the TensorFlow implementation can compute the transitive closure of all nodes that must be executed in order to compute the outputs that were requested.. 21. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(23) Figure 2: Example TensorFlow code fragment via Python. 立. 政治大. ‧. ‧ 國. 學 sit. y. Nat. n. al. er. io. Figure 3: Corresponding computation graph for Figure 2. 2.4.2 GPU. Ch. engchi. i n U. v. In recent years, the accelerated development of deep learning and artificial intelligence technology, one of the most important factors is that GPU provides a powerful parallel computing architecture that enables deep learning training ten times faster than the CPU, the model training time, from weeks shorten to a few days. Deep learning simulates neuron operation with a large number of matrix operations. The characteristic of the matrix operation is that a single operation is simple, but requires a large number of operations and is particularly suitable for parallel operations. Compare CPU and GPU processing tasks, you can clearly understand the difference between the two. The CPU contains several cores that are used to optimize 22. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(24) sequential processing. The GPU contains thousands of smaller, more efficient cores that are optimized for handling multiple tasks simultaneously. Since NVIDIA's launch in February 2007, the CUDA programming model has been used to develop many of the GPU applications. CUDA provides an easy to learn ANSI C language extension. Programmers specify parallel threads, each of which running scalar code. NVIDIA’s graphics card all support CUDA technology after GeForce 8 series (Topa, T., Karwowski, A., & Noga, A. 2011). With hardware and software support, GPUs have become an integral part of. 政治大 functionality of GPUs have significantly increased. Modern GPU is not only a powerful 立 today's mainstream computing systems. In recent years, the performance and. graphics engine, but also a highly parallel programmable processor that substantially. ‧ 國. 學. outpaces its CPU (Owens et al., 2008).. ‧. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. 23. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(25) 3. EXPERIMENT This study derives an ANN mechanism for giving credit scores to students who try to borrow money on P2P Lending platform. This experiment is also designed to verify the timeless and effectiveness of the ANN mechanism via derived TensorFlow and GPU. The study sets up an experiment with the real dataset collected by our online P2P Lending platform. We collect 200 data from 2018/3/30~2018/4/7. There are 2 data that are invalid, then choosing 140 data to do training, 58 data to be the testing set. Moreover, we will repeat the random selection of testing and training groups for three times, and. 政治大. do the experiment for three times. The experiment also wants to know that whether the. 立. derived ANN mechanism can applied in the credit score field.. ‧ 國. 學. In first section, we will discuss how to choose the variables and data preprocessing. Then in second section, we will discuss the ANN mechanism for giving students credit. ‧. scores and data testing. An ANN model will be implemented in this study, the ANN. y. Nat. sit. model will create a credit score, which will be an integer between 1~100.. er. io. In addition, the experimental environment is shown as Table 4.. n. aTable iv l C4: The Experiment Environment n hengchi U Machine. OS. Ubuntu 16.04 LTS. CPU. Intel® Core™ i7-6900K. GPU. NVIDIA GeForce GTX 1080 1.898GHz. RAM. DDR4-2133 64 G. Language. Python 3.6. API. TensorFlow-gpu r1.4. CUDA. CUDA 8.0.27 for Linux 24. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(26) 3.1 Variables Description 3.1.1 Variables selection From the literature review, the following study will organize the recommendations and analysis of the previous chapter; select the appropriate and more accessible variables for students’ microfinance ANN model training. Some of the test data do not include the y-values (output) in ANN training, which are borrowing or not and the value of risk. We use crowd sourcing, finding some professionals in student microfinance to help us define the y-values, which include. 政治大. credit score and risk. The followings show the data we use in this study, 26 variables. 立. were selected.. Loan size: How much money borrowers want to borrow?. 2.. Desired times of repayment: When the borrower predict to pay the loan. 3.. Loan usage: The usage of this loan. (Classification data). 4.. Sex: Male or Female.. 5.. Age: Borrower’s age.. 6.. Marital status: Married / unmarried / divorced. 7.. Children: How many children the borrower have?. 8.. Number of Facebook friends: The borrower FB friends.. 9.. High school name: Which is the high school for borrower? In this. ‧. ‧ 國. 學. 1.. n. Ch. engchi. er. io. sit. y. Nat. al. i n U. v. experiment we divided high school in Taiwan into two levels. 10. High school graduate score: What is the high school graduation score of the borrower? 11. College name: Which is the college for borrower? In this experiment we divided college into two groups (National, Else). 12. Highest degree attained: The highest degree of the borrower. 25. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(27) 13. Current grade: Current years of the borrower. 14. Major or department: The majoring subject of the borrower. In this experiment we refer to the classification of department on 教育部網站6 and divided into two groups according to the salary in the future. 15. Cross-school credits: Whether there are cross-school learning credits. 16. Credit failed to date: Currently failed the academic credit. 17. Credit passed to date: Currently passed the academic credit. 18. Last semester GPA/score: Last semester average score of the borrower.. 政治大 County into two groups (Municipalities, Other). 立. 19. Hometown: Household registration. In this experiment we divided Taiwan. 20. Residency: Current place of residence.. ‧ 國. 學. 21. Properties: Is there property under the name? (Movable property,、Real. ‧. estate, Both, None). y. Nat. 22. Average monthly allowance: The monthly allowance from family.. er. io. sit. 23. Average monthly income: The monthly income by borrowers themselves. 24. Living expenditure per month: Monthly expenses.. al. n. v i n C h The highest degree Highest degree of parents: of parents. engchi U. 25.. 26. Family’s aggregated income: The range of family’s aggregated income. 3.1.2 Data preprocessing We divide the selected variables into three categories: ordered category data, unordered category data and digital data. Different data preprocessing will be performed for each kind of data above. For the ordered category data, we use dummy variables to present. In the. 6. 106 學年度大專院校一覽表: https://ulist.moe.gov.tw/Home/Index 26. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(28) unordered category data part, we use onehot-encoding techniques to represent various classes to the same value in ANN. Last, we standardize digital data to facilitate the training of ANN. However, because we create the output (Credit score & Default risk) by crowd sourcing. Different experts will have different scoring standards in some data. Therefore, we adjust the range of both Credit score & Default risk to 95~5. Table 5 shows that the data we finally used in this experiment. Table 6 and Table 7 are statistical table of the use of the data.. 政治大. Table 5: The data used in this experiment. 立. Measuring unit. Y1. Credit score. Integer: 0~100. Y2. Default risk. Desired times of repayment 投資理財. X22. Nat. Loan usage. X23. 購買汽/機車生活費購買 3C. io. X24 X32. Input Variables. X33 X41 X42 X43. al. Male. n. X31. sit. X21. i n Female U. C hgender e n g c h i Other Marital status. y. ‧. X1. Integer: 0~100 Integer onehotencoding. er. ‧ 國. Output variables. 學. Variable name. Variable categories. v. Married Unmarried Divorced. onehotencoding onehotencoding. X5. Age. Integer. X6. Children. Integer. X7. Number of Facebook friends. Integer. X8. High school name. dummy variables. X9. High school graduate score. Integer. X10. College name. dummy variables. X11. Highest degree attained. dummy. 27. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(29) variables X12. Current grade. Integer. X13. Major or department. dummy variables. X14. Cross-school credits. dummy variables. X15. Credit failed to date. Integer. X16. Credit passed to date. Integer. X17. Last semester score. Integer. X18. Hometown. X19. Residencies. X20. dummy variables dummy variables. 政治大 Properties. 立. dummy variables. Average monthly allowance. dummy variables. X22. Living expenditure per month. dummy variables. X23. Highest degree of parents. X24. Family’s aggregated income. y. sit. io. n. i n U. dummy variables dummy variables. er. Nat. al. ‧. ‧ 國. 學. X21. v. Table 6: Non-dummy variables data description statistics. Ch. Variable name. Mean. Credit score. engchi SD. Max. min. 60.46. 21.6. 95. 5. Default risk. 45.14. 27.8. 95. 5. Age. 22.38. 2.87. 38. 18. 676.55. 427.04. 2340. 0. 9.85. 21.28. 115. 0. Number of Facebook friends Credit failed to date 28. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(30) Credit passed to 48.74. 44.21. 185. 0. 79.80. 18.02. 98. 0. date Last semester score. Table 7: Dummy variables data description statistics. Variable categories 投資理財. Numbers. Percentage. 45. 22.7%. 購買汽/機車. 17. 8.6%. 生活費. 27. 13.6%. 教育費. 37. 18.7%. 娛樂費. 19. 9.6%. Loan usage. 政治 1大購買 3C 52 立 Male 105. 26.3%. Female. 92. 46.5%. Other. 1. Married. 3. Unmarried. 194. Divorced. 1. PR score > 80. 129. Others. 69. Municipalities. al. 185. Ch. 13. Highest degree attained. Properties. Others. e n g c h i181U National university. y. sit er. n. College name. io. Living place. Nat. High school name. 53.0%. ‧. Marital status. 0.5%. 學. Gender. ‧ 國. 債務整合. v ni. 0.5% 1.5% 98.0% 0.5% 65.2% 34.8% 93.4% 6.6% 91.4%. Others. 17. 8.6%. Bachelor degree. 120. 60.6%. Master degree. 74. 37.4%. PhD degree. 4. 2.0%. None. 127. 64.1%. Movable property. 52. 26.3%. Real estate. 6. 3.0%. Both. 13. 6.6%. 29. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(31) 3.2 ANN for students credit scores For the ANN model using in students P2P Lending, this study is based on the mechanism of Wu (2017), which is adapts the works of Tsaih and Cheng and Huang et al. However, the implementation of moving window is not necessary in this field, this mechanism only implement RLEM. Table 9 show the proposed mechanism. Table 8: Variables using in this study. Variable categories. Variable names. Variable meanings. m. Number of input variables. 35 input variables. x. Input variables. Loan size, Loan usage ....... 立. y1. Sample size of the training data Sample size of the testing block. 140 data 58 data. Standard deviation of training data in the training block. y. Percentage of potential outliers. Nat.  / 10. 5%. A float number A float number. io. sit. ε. Integer (1~100). Table 9: The proposed mechanism. al. er. . Default Risk. ‧. k. Integer (1~100). 學. B. ‧ 國. y2 N. 政治大 Credit Score. n. v i n C hin Table 2 (with envelope Step 1: Apply the RLEM stated width = 2) to the N U i e h n training examples {(x , y ), (x , g y ),c…, (x , y )} to filter out Nk potential 1. 1. 2. 2. N. N. outliers and obtain an acceptable SLFN by linear programming. Step 2: Remove the outlier candidates, and then use the SLFN obtained in Step 1 and the RLEM stated in Table 2 (with envelope width = 2) again to learn the remained N(1-k) training examples from the first data (n=1) instead of n=m+2. Step 3: Apply the SLFN obtained in Step 2 to the B testing examples {(x1, y1), (x2, y2), …, (x B, y B)}. In Step 1, we apply the RLEM stated in Table 2 (with envelope width = 2) to the N training examples to filter out Nk potential outliers and get an acceptable SLFN, 30. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(32) which has an envelope contains at least N(1-k) examples. However, because of the onehot-encoding data preprocessing, we can’t set up an acceptable SLFN estimate with one hidden node with matrix operations. Instead, we use linear programming to get the same goal. In Step 2, remove the outlier candidates, and then use the SLFN obtained in Step 1 and the RLEM stated in Table 2 (with envelope width = 2 ) again to learn the remained N(1-k) training examples from the first data. . represents the 1/10 of  in. Step 1.. 政治大 The main ANN structure of the system is shown in Figure 4, in which the number 立 In Step 3, we apply the obtained SLFN to the testing examples.. of nodes in the input layer is the selected variable influencing the students microfinance,. ‧ 國. 學. while the training paradigm is for each sample loan application data, and x is the impact. ‧. variable for each loan application data. p is the number of hidden layer nodes, which is. y. Nat. preset to one, and with the increase of training examples, the number of hidden nodes. er. io. sit. is appropriately increased or decreased to meet the set learning objectives. The number of output layer node is one.. n. al. Ch. engchi. i n U. v. 31. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(33) Figure 4: The ANN of the students’ microfinance. 學 ‧. ‧ 國. 立. 政治大. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. 32. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(34) 4. EXPERIMENT RESULTS 4.1 Results and discussion During the experiment, we will automatically increase or decrease the number of hidden nodes in the hidden layer of the network. Therefore, the number of hidden nodes may be different for different numbers of samples. Stage 1 is Step 1 in our proposed mechanism, which can also be represented by 1st RL with envelope width = 2. Stage 2 is Step 2, and can be represented by 2nd RL with envelope width = 2.. 政治大 three rating groups: Excellent (100~70) is a group of high credit score which we suggest 立 In addition, for the testing progress of the experiment, we divide credit score into. you can lend them money, Moderate (69~31) is a group we should consider about. ‧ 國. 學. lending them money, Poor (30~0) is a group we should not lend them.. Nat. 15% 40%. n. al. Ch. engchi. 45%. Excellent group(78). er. io. sit. y. ‧. Credit Score Rating Groups. i n U. Moderate group(87). v. Poor group(30). Figure 5: The distribution of credit score rating groups. Due to the high complexity of the algorithm, the training time is often a big issue in the past. After using the GPU and TensorFlow in this study, the training time of ANN can be completed within a few days in both stage 1 and stage 2. Table 10 and Table 11 shows the total number of adopted hidden nodes and training time spent in proposed 33. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(35) ANN mechanism of each stage. Table 10: The training data and number of hidden nodes in different experiment.. Experiment 1. Experiment 2. Experiment 3. Average. Stage 1. Stage 2. Stage 1. Stage 2. Stage 1. Stage 2. Stage 1. Stage 2. Training data number. 140. 133. 140. 133. 140. 133. 140. 133. Inlier. 133. 133. 133. 133. 133. 133. 133. 133. Initial hidden nodes number. 1. 147. 1. 159. 1. 163. 1. 156. Final hidden nodes number. 147. 159. 203. 163. 209. 156. 208. training data number. 立211. 政治大. ‧ 國. 學. al. 15:32:17. n. Stage 1 1st Stage 2. Stage 1 2nd Stage 2 Stage 1 3rd Stage 2. Average. Stage 1 Stage 2. Ch. 84.1% 29:48:45. engchi U. sit. io. (hh:mm:ss). Total training (hh:mm:ss). er. Stage. y. Pruning mechanism. Nat. Experiment. ‧. Table 11: The training time spent in different experiment. v ni. 18:28:38 100% 35:11:09. 84.7%. 100%. 18:45:31. 22:05:51. 84.9%. 100%. 35:10:10. 40:58:43. 85.8%. 100%. 20:12:18. 23:52:26. 84.6%. 100%. 37:20:18. 43:21:46. 86.1%. 100%. 18:10:02. 21:28:58. 84.6%. 100%. 34:06:24. 39:50:32. 34. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(36) 85.6%. 100%. In Figure 6~17, the gray dots represent the envelope module wrapped the response instances seen as inlier instances in the envelope, the blue dots represent the actual credit score and the orange dots represent the predicted credit score we got from ANN. Figure 6 and Figure 7 shows the envelope module of the 1st RL (with envelope width = 2) to identify potential outliers. After the Stage 1, No. 40, 42, 48, 67, 84, 108 and 125 instances in the training data is identified as outlier candidates. In the proposed ANN mechanism Stage2, the 2nd RL (with envelope width = 2), we remove the outlier candidates. Figure 8 and Figure 9 shows the actual return of credit. 政治大. score in  =  /10 (4.5). After Stage2, all inlier instances are wrapped in the envelope. 立. outlier candidates : No. 40, 42, 48, 67, 84, 108, 125. 0. 10. y 20. io. -100. 30. sit. 0. Nat. 50. 40. 50. 60. 70. er. 100. ‧. 150. -50. Stage1 , epsilon = 2 (45.01). 學. ‧ 國. of the 2nd RL.. al. n. v i n Predicted Credit Credit Score Envelope CScore U h e nActual i h gc. -150. Figure 6: The envelope module of the 1st RL to identify outliers in 1st experiment. Stage1 , epsilon = 2 (45.01) outlier candidates : No. 40, 42, 48, 67, 84, 108, 125. 150 100 50 0 -50. 71. 81. 91. 101. 111. 121. 131. -100 -150 Predicted Credit Score. Actual Credit Score. Envelope. Figure 7: The envelope module of the 1st RL to identify outliers in 1st experiment 35. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(37) Stage2 , envelope width = 2 (4.5) remove outlier candidates : No. 40, 42, 48, 67, 84, 108, 125. 100 80 60 40 20 0 0. 10. 20. 30. Predicted Credit Score. 40. 50. Actual Credit Score. 60. 70. Envelope. Figure 8: Desired and predicted credit score in 2nd RL in 1st experiment. 政治大. Stage2 , envelope width = 2 (4.5). 立. remove outlier candidates : No. 40, 42, 48, 67, 84, 108, 125. 100. 40 20 71. 81. Nat. 0. 91. 101. io. al. 121. 131. Actual Credit Score. Envelope. er. Predicted Credit Score. 111. sit. 60. ‧. ‧ 國. 學. 80. y. 120. v. n. Figure 9: Desired and predicted credit score in 2nd RL in 1st experiment. Ch. engchi. i n U. Figure 10 and Figure 11 shows the envelope module of the 1st RL (with envelope width = 2) to identify potential outliers. After the Stage 1, No.72, 88, 89, 97, 102, 125 and 132 instances in the training data is identified as outlier candidates. In the proposed ANN mechanism Stage2, the 2nd RL (with envelope width = 2), we remove the outlier candidates. Figure 12 and Figure 13 shows the actual return of credit score in  =  /10 (4.46). After Stage2, all inlier instances are wrapped in the envelope of the 2nd RL.. 36. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(38) Stage1 , epsilon = 2 (44.62) outlier candidates : No.72, 88, 89, 97, 102, 125, 132. 120 100 80 60 40 20 0 -20. 0. 10. 20. 30. Predicted Credit Score. 40. 50. Actual Credit Score. 60. 70. Envelope. Figure 10: The envelope module of the 1st RL to identify outliers in 2nd experiment. 政治大. Stage1 , epsilon = 2 (44.62). 立. outlier candidates : No.72, 88, 89, 97, 102, 125, 132. 150. 100. -100. 81. 91. 101. 121. 131. y. Nat. -150. 111. n. al. Actual Credit Score. Envelope. er. io. Predicted Credit Score. sit. 71. ‧. -50. ‧ 國. 0. 學. 50. i n U. v. Figure 11: The envelope module of the 1st RL to identify outliers in 2nd experiment. Ch. engchi. Stage2 , envelope width = 2 (4.46) remove outlier candidates : No.72, 88, 89, 97, 102, 125, 132. 100 80 60 40 20 0 0. 10. 20 Predicted Credit Score. 30. 40 Actual Credit Score. 50. 60. 70. Envelope. Figure 12: Desired and predicted credit score in 2nd RL in 2nd experiment. 37. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(39) Stage2 , envelope width = 2 (4.46) remove outlier candidates : No.72, 88, 89, 97, 102, 125, 132. 100 90. 80 70 60 50. 40 30 20 10 0 71. 81. 91. 101. 121. 政治大. Predicted Credit Score. 立. 111 Actual Credit Score. 131. Envelope. Figure 13: Desired and predicted credit score in 2nd RL in 2nd experiment. ‧ 國. 學. Figure 14 and Figure 15 shows the envelope module of the 1st RL (with envelope width = 2) to identify potential outliers. After the Stage 1, No. 23, 61, 70, 88, 89, 105. ‧. and 127 instances in the training data is identified as outlier candidates.. y. Nat. sit. In the proposed ANN mechanism Stage2, the 2nd RL (with envelope width = 2),. n. al. er. io. we remove the outlier candidates. Figure 16 and Figure 17 shows the actual return of. i n U. v. credit score in  =  /10 (4.34). After Stage2, all inlier instances are wrapped in the envelope of the 2nd RL.. Ch. engchi. 38. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(40) Stage1 , epsilon = 2 (43.99) outlier candidates : No. 23, 61, 70, 88, 89, 105, 127. 140 120 100 80 60 40 20 0. -20 0. 10. 20. 30. 40. 50. 60. 70. -40 -60 -80 Predicted Credit Score. Actual Credit Score. Envelope. 政治大. Figure 14: The envelope module of the 1st RL to identify outliers in 3rd experiment. 立. Stage1 , epsilon = 2 (43.99). ‧ 國. 101. 111. 121. y. 91. al. 131. v i n C h Actual Credit Score Predicted Credit Score e n g c h i U Envelope n. -150. io. -100. 81. sit. 71. Nat. 0. ‧. 50. er. 100. -50. 學. outlier candidates : No. 23, 61, 70, 88, 89, 105, 127. 150. Figure 15: The envelope module of the 1st RL to identify outliers in 3rd experiment. 39. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(41) Stage2 , envelope width = 2 (4.34) remove outlier candidates : No. 23, 61, 70, 88, 89, 105, 127. 100 90 80 70 60 50 40 30 20 10 0 0. 10. 20. 30. Predicted Credit Score. 40. 50. Actual Credit Score. 60. Envelope. Figure 16: Desired and predicted credit score in 2nd RL in 3rd experiment. 政治大 Stage2 , envelope width = 2 (4.34) 立. remove outlier candidates : No. 23, 61, 70, 88, 89, 105, 127. 學. 100 80 60. ‧. ‧ 國. 120. sit. n. al. er. io. 20. y. Nat. 40. 0 71. 81. 91. Ch. 101. 111. v i 121 n U. e nActual h i Score g cCredit. Predicted Credit Score. 131. Envelope. Figure 17: Desired and predicted credit score in 2nd RL in 3rd experiment. This study will test the training instances, inlier training instances, and test instances separately based on the ANN after training. In the Table 12, we calculate mean and standard deviation of deviation between desired and predicted credit score in different experiment. After we remove the outlier candidates, there are significant declines on both mean of deviation and SD of deviation. For the 2nd RL with smaller envelope width, the mean of deviation and SD of deviation are also less than 1st RL. Table 12: Deviation between desired and predicted credit score in different experiment 40. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(42) 1st RL (envelope width = 2*σ) inlier (133) all (140) training training instances instances. Experiment Mean of Deviation 1st SD of Deviation Mean of Deviation nd 2 SD of Deviation Mean of Deviation 3rd SD of Deviation Mean of Ave Deviation rage SD of Deviation. 2nd RL (envelope width = 2*ε) inlier (133) testing(58) training instances instances. 1.3434. 7.8594. 0.2622. 22.31. 1.8501. 28.6476. 0.5347. 20.7167. 0.9882. 8.3558. 0.2842. 25.1536. 1.61. 32.2024. 0.6017. 21.4272. 0.9438. 7.3614. 0.2184. 26.903. 1.6601. 28.4262. 0.4829. 28.9855. 1.0918. 7.8589. 0.2549. 24.7889. 1.7067. 立. 治 0.5398 政 29.7587 大. 23.7098. Table 13 presents the deviation between desired and predicted credit score. In. ‧ 國. 學. training instances, our ANN can make sure all deviation between desired and predicted credit score is within one standard deviation, expect for the outliers instances. In testing. ‧. instances, the average percentage of deviation between desired and predicted credit. Nat. sit. y. score which is within one standard deviation is about 63%. And only about 10%. n. al. er. io. instances will beyond two standard deviations.. i n U. v. Table 13: Accuracy of credit score prediction in different experiment. deviation. 1st. 2nd. Ch. e n g cInlier h i (133). All (140) training instances. training instances after 2nd RL. Testing instances(58). Number. Percenta -ge. Number. Percenta -ge. Number. Percenta -ge. ≤σ. 133. 95%. 133. 100%. 42. 72.4%. >σ&< 2σ. 0. 0%. 0. 0%. 11. 19%. ≥ 2σ. 7. 5%. 0. 0%. 5. 8.6%. ≤σ. 133. 95%. 133. 100%. 35. 60.3%. >σ&< 2σ. 0. 0%. 0. 0%. 17. 29.3%. ≥ 2σ. 7. 5%. 0. 0%. 6. 10.4%. 41. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(43) 3rd. Ave rage. ≤σ. 133. 95%. 133. 100%. 33. 56.9%. >σ&< 2σ. 0. 0%. 0. 0%. 16. 27.6%. ≥ 2σ. 7. 5%. 0. 0%. 9. 15.5%. ≤σ. 133. 95%. 133. 100%. 110. 63.2%. >σ&< 2σ. 0. 0%. 0. 0%. 44. 25.3%. ≥ 2σ. 7. 5%. 0. 0%. 20. 11.5%. Table 14 shows the accuracy of credit score prediction in different experiment. A represents a group with a credit score of Excellent we have mentioned in the beginning of Chapter 4. B represents a group with a credit score of Moderate and C represents a. 政治大. group with a credit score of Poor. After the Step2, except for some prediction errors at. 立. the edge of the group, it gave each P2P loan information into the right rating group. For. ‧ 國. 學. the testing instance, it can also predict the rating group within a certain accuracy rate.. 2nd. 3rd. B. C. sit. A. 0/55v i n C5/58 0/54 0/54 54/54 hengchi U 23/24 0/24 1/24 23/24 2/58. B. C. 11/16. 2/16. 3/16. 3/27. 18/27. 6/27. 2/15. 5/15. 8/15. A. 55/58. B. 0/58. 53/58. C. 0/24. 1/24. A. 51/53. 0/53. 2/53. 51/51. 0/51. 0/51. 15/24. 6/24. 3/24. B. 0/62. 57/62. 5/62. 0/57. 57/57. 0/57. 3/25. 16/25. 6/25. C. 0/25. 0/25. 25/25. 0/25. 0/25. 25/25. 4/9. 1/9. 4/9. A. 55/56. 0/56. 1/56. 51/51. 0/51. 0/51. 15/22. 4/22. 2/22. B. 0/61. 56/61. 5/61. 0/59. 59/59. 0/59. 2/25. 20/25. 3/25. C. 0/23. 1/23. 22/23. 0/23. 1/23. 22/23. 2/11. 2/11. 7/11. 55/55. 0/55. Testing instances(58). A. er. al. 1/58. C. instances after 2nd RL. n. 1st. B. io. A. Inlier (133) training. ‧. Nat. Group. All (140) training instances. y. Table 14: Accuracy of credit score prediction in different experiment. 42. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(44) Table 15 presents the predict accuracy in training instances and testing instances. After Stage2, the ANN model can predict the inlier training instances well with an accuracy rate of 95% or more. The results shows that it is better to predict under the assumption that there is a possibility of outliers in the process of the ANN training. For the training instances, it can predict both Excellent and Moderate group with an accuracy rate of 65% or more. However, it can only predict the low credit score rating group with an accuracy rate of 54%. Table 15: Accuracy of credit score prediction in different experiment. Predict/ Actual. Percentage. 55/58. 94.8%. 55/55. 100%. 11/16. 68.9%. 53/58. 91.4%. 54/54. 100%. 23/24. 95.8%. 23/24. 95.8%. 51/53. 96.2%. 51/51. 100%. B. 57/62. 91.9%. 57/57. 100%. C. 25/25. A. 55/56. B. 56/61. C. 22/23. A. 66.7%. 15/24. 62.5%. 16/25. 64%. 4/9. 44.4%. 15/22. 68.2%. 20/25. 80%. er. A. 100% 25/25 100% a l98.2% 51/51 100%i v n C h 59/59 U 91.8% 100% engchi. n. Ave rage. Percentage. io. 3rd. Predict/ Actual. Nat. 2nd. Percentage. y. C. Predict/ Actual. 18/27 8/15. 53.3%. sit. B. RL. ‧. 1st. Testing instances(58). nd. 學. A. 立. ‧ 國. Gro up. (133) training 治政Inlier instances after 大2. All (140) training instances. 95.7%. 22/23. 95.7%. 7/11. 63.6%. 161/167. 96.4%. 157/157. 100%. 41/62. 66.1%. B. 166/181. 91.7%. 170/170. 100%. 53/77. 68.8%. C. 70/72. 97.2%. 70/72. 97.2%. 19/35. 54.3%. 43. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(45) 5. Conclusions and Future Works 5.1 Conclusions This research derives a hybrid ANN mechanism for timely and effectively predicting the credit score about students who try to borrow money on P2P Lending platform. Implemented the experiment with recently released machine learning toolTensorFlow with GPU device. Using the real data collected from the P2P Lending platform to train the ANN and found some valuable issues based on the experimental. 政治大 Issue on ANN algorithm field. Although we obtain the ANN algorithm which has 立. results: 1.. already designed and it can do the perfect prediction in training data, it still needs. ‧ 國. 學. a lot of great effort to apply to the real-world situation. Especially, the giving of. ‧. credit score are very sensitive in P2P students lending field, so it is important that. io. y. sit. In the application field, the most important task in risk control is to be able to. er. 2.. Nat. we still need to improve the ANN model.. identify the Poor group. The experiment results show that it can predict both. al. n. v i n C hwith an accuracy rate Excellent and Moderate group of 65% or more. However, it engchi U can only predict Poor group with an accuracy rate of 54%. 3.. Continuing the second point, since the experiment was in the initial stage, the training data was not enough for us to train the ideal ANN model. Adding more training data in the future is believed to improve forecast accuracy. Due to the ANN can self-adjust the parameters and the proposed algorithm will automatically increase or decrease the number of hidden nodes. It is indeed an auxiliary method for the credit scoring system without certain rules.. 44. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(46) 5.2 Future works In the research process, we face different issues, difficulties and experiences in all aspects. Here are the future outlook and recommendations for this study: 1.. The experiment results show that using a machine learning tool TensorFlow with graphics processing GPU operations under a large amount of input data, input variables and complex algorithms are indeed significant for ANN. The training time of ANN can be in hours. If we can add the comparisons about performance and experiment time with other method, it will be more representative of whether. 政治大 As the training time is立 extended, the established TensorFlow graph will increase, ANN is suitable for predicting credit score in students P2P Lending.. 2.. ‧ 國. 學. and the system will not release the memory, resulting in slow operation. From the empirical results, it is found that GPUs do have good computing performance. If. ‧. we upgrade hardware devices in the future and use multiple GPUs to perform. y. sit. al. er. Continuing the second point, if it is possible to reduce training time due to. io. 3.. Nat. neural network operations, how much time can be reduced will be a research issue.. v. n. hardware upgrades, consider using more data as a training paradigm. The training. Ch. engchi. i n U. paradigm for this study is 70% of the data and it is recommended to use 80% of the data, even all the data are trained as training examples in the future.. 45. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(47) REFERENCE English Reference 1.. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., ... &. Ghemawat, S. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467. 2.. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., ... & Kudlur, M.. (2016, November). TensorFlow: A System for Large-Scale Machine Learning. In OSDI (Vol. 16, pp. 265-283). 3.. 立. 政治大. Berger, S. C., & Gleisner, F. (2010). Emergence of financial intermediaries in. ‧ 國. 4.. 學. electronic markets: The case of online P2P lending.. Funk, B., Buerckner, D., Hilker, M., Kock, F., Lehmann, M., & Tiburtius, P. (1970).. ‧. Online Peer-to-Peer Lending Ã¢ Â Â A Literature Review. The Journal of Internet. Nat. sit. n. al. er. Christman, D. E. (2000). Multiple realities: Characteristics of loan defaulters at a. io. 5.. y. Banking and Commerce, 16(2), 1-18.. i n U. v. two-year public institution. Community College Review, 27(4), 16-32. 6.. Ch. engchi. Freedman S., Jin, G.（2008）, "Do Social Networks solve Information Problems. for Peer-to-Peer Lending? Evidence from Prosper.com" 7.. Gross, J. P., Cekic, O., Hossler, D., & Hillman, N. (2009). What Matters in Student. Loan Default: A Review of the Research Literature. Journal of Student Financial Aid, 39(1), 19-29. 8.. Hampshire, R. (2008). Group Reputation Effects in Peer-to-Peer Lending Markets:. An Empirical Analysis from a Principle-Agent Perspective. mimeo. 9.. Harrast, S. A. (2004). Undergraduate borrowing: A study of debtor students and. their ability to retire undergraduate loans. Journal of Student Financial Aid, 34(1), 2146. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(48) 37. 10. Herr, E., & Burt, L. (2005). Predicting student loan default for the University of Texas at Austin. Journal of Student Financial Aid, 35(2), 27-49. 11. Lavoie, F., Pozzebon, M., & Gonzalez, L. (2011). Challenges for inclusive finance expansion:. The. case. of. CrediAmigo,. a. Brazilian. MFI.. Management. international/International Management/Gestión Internacional, 15(3), 57-69. 12. Owens, J. D., Houston, M., Luebke, D., Green, S., Stone, J. E., & Phillips, J. C. (2008). GPU computing. Proceedings of the IEEE, 96(5), 879-899.. 政治大. 13. Tsaih, R. H. and J. Z. Wu. (2017). Application of machine learning to predicting the returns of carry trade. 立. 14. Tsaih, R. H. and M. C. Lian. (2017). Exploring the timeliness requirement of. ‧ 國. 學. artificial neural networks in network traffic anomaly detection. ‧. 15. Tsaih, R. H. and T. C. Cheng. (2009). A resistant learning procedure for coping. y. sit. io. er. 180.. Nat. with outliers, Annals of Mathematics and Artificial Intelligence, vol. 57, no. 2, pp. 161-. 16. Rousseeuw, P. J., & Van Driessen, K. (2006). Computing LTS regression for large. al. n. v i n C h discovery, 12(1),U29-45. data sets. Data mining and knowledge engchi. 17. Stiglitz, J. E., & Weiss, A. (1981). Credit rationing in markets with imperfect information. The American economic review, 71(3), 393-410. 18. Topa, T., Karwowski, A., & Noga, A. (2011). Using GPU with CUDA to accelerate MoM-based electromagnetic simulation of wire-grid models. IEEE Antennas and Wireless Propagation Letters, 10, 342-345. 19. Tsaih, R. R. (1993). The softening learning procedure. Mathematical and computer modelling, 18(8), 61-64. 20. Volkwein, J. F., & Cabrera, A. F. (1998). Who defaults on student loans? The effects of race, class, and gender on borrower behavior. In R. Fossey & M. Bateman 47. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(49) (Eds.), Condemning students to debt: College loans and public policy (pp. 105-126). New York: Teachers College Press. 21. Volkwein, J. F., & Szelest, B. P. (1995). Individual and campus characteristics associated with student loan default. Research in Higher Education, 36(1), 41-72. 22. Waller, G. M., & Woodworth, W. (2001). Microcredit as a Grass‐Roots Policy for International Development. Policy Studies Journal, 29(2), 267-282.. Chinese Reference. 政治大 ——以 Lending Club立為例，資訊管理學系碩士論文. 李坤霖，(2017)，應用倒傳遞類神經網路於 P2P 借貸投資報酬率預測之研究. 學 ‧. ‧ 國 io. sit. y. Nat. n. al. er. 1.. Ch. engchi. i n U. v. 48. DOI:10.6814/THE.NCCU.MIS.016.2018.A05.

(50)