A Study of Taiwan's Issuer Credit Rating Systems Using Support Vector Machines

(1)

A study of Taiwan’s issuer credit rating systems using support

vector machines

Wun-Hwa Chen

a,1

, Jen-Ying Shih

b,1,

*

a_{Graduate Institute of Business Administration, National Taiwan University, Taiwan, ROC} b_{Securities and Futures Institute, 9F, 3, Nan-Hai Rd., Taipei 100, Taiwan, ROC}

Abstract

By providing credit risk information, credit rating systems benefit most participants in financial markets, including issuers, investors, market regulators and intermediaries. In this paper, we propose an automatic classification model for issuer credit ratings, a type of fundamental credit rating information, by applying the support vector machine (SVM) method. This is a novel classification algorithm that is famous for dealing with high dimension classifications. We also use three new variables: stock market information, financial support by the government, and financial support by major shareholders to enhance the effectiveness of the classification. Previous research has seldom considered these variables. The data period of the input variables used in this study covers three years, while most previous research has only considered one year. We compare our SVM model with the back propagation neural network (BP), a well-known credit rating classification method. Our experiment results show that the SVM classification model performs better than the BP model. The accuracy rate (84.62%) is also higher than previous research.

Keywords: Credit ratings; Support vector machines; Taiwan’s banking industry

1. Introduction

An issuer credit rating is a current assessment of an issuer’s overall capacity to meet its financial obligations based on its ability and willingness to meet its financial commitments on a timely basis. Such information is a useful and important reference for raising corporate capital, the granting of credit by banks, and providing credit risk information for investment decisions. It therefore benefits most participants in financial markets, including issuers, investors, market regulators and intermediaries.

Although rating agencies claim that both financial and non-financial information is considered in the rating decision process, their rating criteria are not explicit. Thus, many researchers have tried to construct automatic classification systems by using data mining methods, such as statistical and artificial intelligence (AI) techniques. The former include

linear regression (Horrigan, 1966), linear multivariate

discriminant analysis (MDA) (Belkaoi, 1980), quadratic

MDA (Pinches & Mingo, 1975), probit regression (Ederington,

1985), logit analysis (Ederington, 1985) and multidimensional

scaling (Mar, Apellaniz & Cinca, 1996). The latter consist of

back propagation neural networks (BP) (Dutta & Shekhar,

1988) and case base reasoning (CBR) (Shin & Han, 2001).

A novel classification technique, the support vector machine

(SVM) proposed by Vapnik, (1999), has been successfully

applied and evaluated in many areas, including financial time

series forecasting (Tay & Cao, 2001), credit scoring (Baesens

et al., 2003), the prediction of protein structural classes (Cai & Lin, 2002), drug design (Burbidge et al., 2001) and the

identification of organisms (Morris & Autret, 2001). However,

very few researchers have studied the suitability of SVMs for credit rating systems. In this paper, we apply the SVM method to the automatic classification of issuer credit ratings. Our goal is to test the SVM’s capacity to perform the classification of credit ratings, not to examine the speed of a new algorithm.

The classification categories of previous credit rating research were based on issue credit ratings, including bond

ratings (Ederington, 1985) and commercial paper ratings (Shin

& Han, 2001), instead of issuer credit ratings. The main reason is that the classification accuracy rates of issue credit ratings can be improved by some issuing variables, such as

subordination, the scale, and the period of issue (Horrigan,

1966; Pinches & Mingo, 1975; West, 1970). However, www.elsevier.com/locate/eswa

* Corresponding author. Address: Graduate Institute of Business Adminis-tration, National Taiwan University, 9F, 3, Nan-Hai Rd, 100 Taipei, Taiwan. Tel.: C886 2 23574357; fax: C886 2 23224302.

E-mail address: [email protected] (J.-Y. Shih).

1

(2)

we believe that issuer credit ratings provide a more fundamental type of credit risk information because issue credit ratings are generally determined on the basis of issuer credit ratings. Generally, junior debt may be rated below the issuer credit rating, while well-secured debt can be rated above. We therefore propose an issuer credit rating model to explore this research area.

Most previous research has focused on using financial

variables as input variables. According to (Atiya, 2001),

trading information, such as the volatility of stock prices, can be taken into consideration when measuring the possibility of bankruptcy. Thus, we included a market information variable in the classification model, namely: each share’s average daily closing price in the previous year. Also, according to the rating criteria published by Standard and Poor’s and the Taiwan ratings corporation (TRC), financial support is very important in determining ratings. Therefore, two financial support variables regarding government and major shareholders are included in the classification model. In addition, it should be noted that the data period of the input variables used in our model covers the three years prior to the rating year, whereas previous researchers have only used data for the year immediately before the rating year. In our opinion, it is more appropriate to use multi-year data because rating agencies use it as the benchmark for the rating decisions.

In this research, we choose the banking industry to test the applicability of SVM models for the issuer’s credit rating. Banks play a pivotal role in an economy, and, as loans are one of their major sources of revenue, it’s important that the level of non-performing loans (NPL’s) is kept to a reasonable minimum. In Taiwan, however, the ratio of NPL’s has increased rapidly in the last decade because of a steep decline in the local stock market and over-competition between banks. The Economist (Nov. 11, 2000) reported that bad loans in Taiwan’s domestic banks had reached new highs, while Business Week (Dec. 11, 2000) pointed out that the NPL ratio among listed banks in Taiwan amounted to more than six percent, However, because of the narrow definition of official NPL statistics, the ratio could, in reality, be as high as 10–15 percent. As rising NPLs may increase the banks’ risks, it’s important to be able to measure those risks. In addition, in most case, the banking industry is highly regulated, information about its operations is usually open to the public and relatively easy to obtain.

The remainder of this paper is organized as follows. In Section 2, we briefly introduce the SVM algorithm, which is the research method we applied in this paper. In Section 3, we describe the test bed, i.e. Taiwan’s banking industry. The research design is described in Section 4, while our experiment results are presented and discussed in Section 5. Finally, in Section 6, we present our conclusions and suggestions for further research.

2. SVM methodology

The SVM model is a type of learning machine that is based on statistical learning theory. The basic procedure for applying SVMs to a classification model can be stated briefly as follows.

First, map the input vectors into a feature space, which is possible with a higher dimension. The mapping is either linear or non-linear, depending on the kernel function selected. Then, within the feature space, seek an optimized division, i.e. construct a hyper-plane that separates two or more classes. Using the structural risk minimization rule, the training of SVMs always seeks a globally optimized solution and avoids over-fitting. It, therefore, has the ability to deal with a large number of features. The decision function (or hyper-plane) determined by a SVM is composed of a set of support vectors, which are selected from the training samples. A complete

description of the theory of SVMs can be found inVapnik,

(1999).

2.1. Support vector classification (SVC)

In the following, we briefly describe how the support vector classification (SVC), which is a type of SVM algorithm for classification purposes, can determine an optimal separating hyper-plane from the division composed of some of the existing samples. The basic principle is to find a maximum margin, i.e. maximize the distance between the hyper-plane and the nearest data point of each class.

2.1.1. Linear separable case

Consider the problem of separating a set of training vectors belonging to two separate classes, ðx1; y1Þ; :::; ðxl; ylÞ; x2

Rn; y2fC1;K1g, where x are input vectors, and y are output

vectors. The data set can be separated by a hyper-plane as two classes, C1 and –1. The set of vectors is said to be optimally separated by the hyper-plane if it is separated without error and the distance between the closest vector and the hyper-plane is maximal. In such a case, the classifier is called the largest margin classifier. The hyper-plane in canonical form must satisfy the following constraints,

yi½hw; xi CbR1; i Z 1; :::; l (1)

w is the weight vector and b is the bias. The optimal hyper-plane is obtained by maximizing the margin r, subject to the constraints of Eq. (1). The margin is given by:

rðw; bÞ Z min xi;yiZK1 dðw; b; xiÞ C min xi;yiZ1 dðw; b; xiÞ Z min xi:yiZK1 jhw; xii C bj jjwjj Cxmini:yiZ1 jhw; xii C bj jjwjj Z 2 jjwjj (2) where d( ) is the distance function. Hence, the hyper-plane that optimally separates the data is the one that minimizes

FðwÞZ 1=2jjwjj2, subject to constraint (1). By applying the

Lagrange relaxation method, the problem can be formulated as follows: min w_;b Fðw; b; aÞ Z 1 2jjwjj 2 K Xl iZ1 aiðyi½hw; xii C bK1Þ; (3)

In essence, the SVM training procedure is the same as solving a convex quadratic programming problem. The solution is

(3)

a unique globally optimized result, which can be stated as follows: wZX l iZ1 aiyixi b ZK1 2hw _{; x} rCxsi (4)

xr, xsare called support vectors if the corresponding ar,asO0,

yrZK1, ysZ1. After training the model, the decision function

(hard classifier) can be written as:

f ðxÞ Z sgnðhw; xi CbÞ (5)

where sgn( ) in Eq. (5) is the given sign function. The soft classifier can be written as:

f ðxÞ Z hðhw; xi CbÞ where hðzÞ Z K1 : z!K1 z : K1% z% 1 C1 : zO 1 8 > < > : (6)

2.1.2. Linear non-separable case-soft margin technique

To allow for training errors, Cortes and Vapnik, (1995)

introduced non-negative slack variables, xiR0, and a cost

function, FsðxÞZPixsi sO 0, where the xi is a measure of

the misclassification errors. The optimization problem is now posed so as to minimize the classification error, as well as minimizing the bound on the VC dimension of the classifier. The constraints of (1) are modified for the non-separable case to yi hw; xiiC b

R1Kxi; iZ1; :::; l and the generalized

optimal separating hyper-plane is determined by the vector w that minimizes the function:

min Fðw; xÞ Z1 2jjwjj 2 CCX l iZ1 xi (7) s:t: yi½hw; xii C bR 1Kxi; i Z 1; :::; l; where xiR0

where C is a parameter used to decide a trade-off between the

training error and the margin.Minoux, (1986)apply Lagrange

relaxation to obtain the following equation: Fðw; b; a; x; bÞ Z1 2jjwjj 2_CCX l iZ1 xiK Xl iZ1 aiðyi½wTxiCb K1 C xiÞK Xl iZ1 bixi ð8Þ

where a and b are Lagrange multipliers.

2.1.3. Generalization in high dimensional feature space—Kernel substitution

If a linear boundary is inappropriate, the SVM can map the input vector, x, into a higher dimensional feature space, z. The mapping is determined by the kernel function. Three typical kernel functions applied in mapping are the polynomial

function, Kðx; x0_{ÞZ ðhx; x}0iC 1Þd_{; the Gaussian radial basis}

function (RBF), Kðx; x0_{ÞZexpðKgjjxKx}0jj2_{Þ; and exponential}

RBF, Kðx; x0_{ÞZexpðKgjjxKx}0jjÞ. Therefore, the decision

function is: aZ arg min a 1 2 Xl iZ1 Xl jZ1 aiajyiyjKðxi; xjÞK Xl kZ1 ak (9)

where K(xi, xj) is the kernel function.

3. Data set

Almost all the samples used in previous credit rating research were industrial bond ratings, which form a type of cross-industry

data set for credit ratings (Belkaoi, 1980; Pinches & Mingo,

1973; Pinches & Mingo, 1975; Shin & Han, 2001). The banking industry specializes in credit ratings, which are different from industrial ratings to some degree. In Taiwan, in order to control the financial markets, the banking industry used to be a highly regulated and traditional industry until the deregulation of new banks and the promulgation of the financial holding company law in the 1990s. In addition, the entry of many new financial commodities into financial markets has promoted competition in the industry. In this volatile financial environment, the measurement of credit worthiness of banks is thus important for depositors, investors and the governments.

We collected data from the TRC and the securities and futures institute (SFI). Information about issuer credit ratings was obtained from the TRC, which is one of the largest credit rating organizations in Taiwan, as well as being the first credit rating service established in this country. The data about financial, operational, equity structure and market information was obtained from the SFI, which plays a key role as a data center for Taiwan’s securities and futures markets. So far, 37 of Taiwan’s banks have been rated by the TRC, including 22 that

are traded on either the TSEC market or the GTSM2; another

15 unlisted banks have also been rated. Only three listed banks haven’t acquired ratings from the TRC. We did not include unlisted banks because of the lack of complete data, leaving 22

banks in the data set, as shown in Table 1. The data covers

nearly 65% of all bank ratings published by the TRC and 88% of all the banks listed by the TSEC and GTSM. Between 1998 and 2003, there were 99 ratings for the 22 banks. Some banks have not been rated since 2001 because they were merged with financial holding companies and not listed on either market. As the information about these banks is incomplete they are not considered in this paper.

We randomly partitioned the data set into two parts in a proportion of 3:1. The first part was used for training and validation to select optimal parameters for the SVMs and prevent the over-fitting problem commonly found in the BP neural network. The second part was used for testing. There were 73 samples in the training and validation set, and 26

samples in the test set, as shown inTable 2.

2_{The full name of TSEC is Taiwan Securities Exchange Corporation, and}

GTSM is GreTai Securities Exchange Market. Both of them are two major stock markets in Taiwan.

(4)

4. Research design 4.1. Input variables

In this paper, the input variables are determined from the following types of information: major shareholders’ support, the market price of each share, company scale, financial structure, solvency, business performance, profitability, finan-cial coverage and cash flow. Except for major shareholders support and the market price of each share, information for the input variables covered a three-year period, making a total of 72 input variables.

The definition of each variable is listed inTable 3. The X1

and X2variables are the measurement of major shareholders’

support, including the government and major shareholders who

hold more than 10% of the outstanding shares. The X3variable

represents the average price of the common stocks issued by the bank in the previous year, based on the daily closing prices. The variables, X4, X5, X6, X7and X8, are measurements of the

scale of the bank from financial statements. X9, X10 and X11

variables represent the financial structure. X12 measures the

solvency. X13, X14, X15, X16, X17, X18and X19are indicators that

measure business performance. X20, X21, X22, X23, X24, and X25

are measurements of profitability. Finally, X26 represents the

cash flow. Most of these variables can be determined from publicly disclosed information, which banks are required to file with authorities like the securities and futures commission. Thus, these input variables are important in financial analysis.

4.2. Output variables

A TRC rating indicates an issuers’ capacity to meet its financial commitments over a one-year period, or more. The ratings are: twAAA, twAA, twA, twBBB, twBB, twB, twCCC, twCC and twD. The prefix ‘tw’ denotes Taiwan and the rating scale focuses on Taiwan’s financial markets. The twAAA rating indicates that an organization has an extremely strong capacity to meet its commitments, whereas the twD rating denotes an organization that may be in high risk. However, the TRC rating scale does not address sovereign risk, so it’s not directly comparable to standard and poor’s global scale. So far, in Taiwan, the range of ratings information for banks determined by the TRC is from twBB to twAAA. As the number of samples in extreme-rating classes is too small, we combine both twAAA and twAA as gtwAA, while twBB and

Table 2

Description of the samples

Ratings Number Percentage

(%)

Number of samples used in Training and

validation set

Test set twAA and the

higher ratings 22 22.22 16 6 TwA 34 34.34 25 9 TwBBB 35 35.35 26 9 twBB and the lower ratings 8 8.08 6 2 Sum 99 100.0 73 26 Table 1

The selected banks

Tick No Name of Bank 1998 1999 2000 2001 2002 2003

2802 First Commercial Bank twAAK twAAK twAAK twAAK twAAK twAAK

2807 Hsinchu International Bank – – twBBBK twBBBK twBBBK twBBBK

2808 International Bank of Taipei – – TwA twA twA TwA

2809 Tainan Business Bank – – – – – TwBBC

2815 Chinatrust Commercial Bank – twAAK twAAK twAAK – –

2822 The Farmers Bank Of China – twAC twAC/twA twA twAK twAK

2828 Grand Commercial Bank – twAK twAK twBBBC twBBB –

2829 Dah an commercial bank twAK twAK twAK twBBBC – –

2830 Taipeibank Co. twAAK twAAK twAAK twAAK twAAK –

2831 The Chinese bank TwAAA TwAAA TwAAA twAAA TwAAA/

twAAC

TwBBC

2834 Taiwan business bank – – twAC twAC twA –

2836 Bank of Kaohsiung twBBBK twBBBK – twBBBK TwBBB

2837 Cosmos bank, Taiwan – – twBBBK twBBBK/

twBBC

twBBB-2838 Union bank of Taiwan – – TwBBB TwBBB – twBBB–

2839 Bank Sinopac Co. – – TwA TwA – –

2840 E.Sun Bank TwA TwA TwA twA – –

2842 Fubon Commercial Bank twAK twAK twAK – – –

2843 Asia Pacific Commercial Bank – twBBBC twBBBC twBBBC/

twBBB

twBBB –

2846 Chung Shing Bank – – TwBBBK/

twBBK

– – –

2847 Ta Chong Bank – – TwBBBK twBBBtwBBK twBBK twBBK

2849 Entie Commercial Bank twA TwA TwA TwA twA –

5818 Bank of Overseas Chinese – – – – – TwBBK

(5)

lower ratings are merged as &twBB. In this classification model, we have designed an output variable to represent the four classes, namely: twAA and above (gtwAA); twA; twBBB; and twBB and below (&twBB).

4.3. Experimental design

To determine the effects of the length of the data period and the three new variables on credit ratings, we designed the ten

models shown inTable 4. The effects of the length of the data

period are analyzed by comparison of the SVM_1 and SVM_2

models and comparison of the SVM_3 and SVM7 models. The SVM_1 model is comprised of all input variables, while the SVM_2 model only considers 1-year financial variables and the other kinds of variables. Meanwhile, the SVM_3 model includes 3-year financial variables, while the SVM_7 model only includes 1-year financial variables. To understand the contribution of each newly designed variable, we compare: SVM_3 with SVMs_4, 5 and 6; and SVM_1 with SVMs_8, 9 and 10.

4.4. SVM models

We applied the LIBSVM program, downloaded fromhttp://

www.csie.ntu.edu.tw/wcjlin/libsvm/, to construct the classifi-cation model and chose Gaussian RBF as the kernel function. The parameters C and g, which control the error-margin trade-off, are generated by a grid search algorithm and the ten-fold

cross validation rule (Hsu, Chang & Lin, 2003; Van Gestel

et al., 2004). Basically pairs of (C, g) are tried and the one with the best cross-validation accuracy is picked. The parameters of

the ten models are shown inTable 4.

We use the ‘one-against-one’ approach (Knerr, Personnaz &

Dreyfus, 1990) in which k(kK1)/2 classifiers are constructed

and each one trains data from two different classes.Friedman,

(1996) was the first research using this strategy on SVM. In classification we use a voting strategy: each binary classifi-cation is considered to be a voting where votes can be cast for all data points x—in the end point is designated to be in a class with maximum number of votes. In summary, under the same (C, g), the one-against-one is used for training multi-class data. Hence, in the final model, all decision functions k(kK1)/2 share the same (C, g).

5. Results and discussion

Like previous researchers, we used the confusion matrix (Pinches & Mingo, 1973; Pinches & Mingo, 1975), the

accuracy rate of each class (Shin & Han, 1999; Shin & Han,

2001), the overall accuracy rate of the model (Pinches &

Mingo, 1973; Pinches & Mingo, 1975) and the average

accuracy rate of the model (Shin & Han, 2001) to measure the

performance of the classification model.

Table 4

Description of the proposed SVM models

Models Input variables Parameters of SVM models Overall accuracy rates

X1 X2 X3 # of years of

financial data

g C Testing set (%) Training set (%)

SVM_1 O O O 3 0.03125 32768 76.92 100.00 SVM_2 O O O 1 0.5 2048 80.77 100.00 SVM_3 3 0.03125 8 76.92 89.04 SVM_4 O 3 0.03125 32 80.77 97.26 SVM_5 O 3 0.125 32768 84.62 100.00 SVM_6 O 3 0.125 8192 84.62 98.63 SVM_7 1 0.5 32 80.77 97.26 SVM_8 O O 3 0.125 8192 84.62 98.63 SVM_9 O O 3 0.125 32768 84.62 100.00 SVM_10 O O 3 0.03125 32768 73.08 100.00 Table 3

Definition of input variables Variables Definition

X1 Proportion of shares in the company held by the

government

X2 Proportion of shares in the company held by major

shareholders who hold more than 10% of the outstanding shares

X3 Average stock price of each share in the previous year

X4i, iZ1,2,3 Total assets

X5i, iZ1,2,3 Liabilities

X6i, iZ1,2,3 Shareholders’ equity

X7i, iZ1,2,3 Net Sales

X8i, iZ1,2,3 Operating Income (Loss)

X9i, iZ1,2,3 Liabilities to assets ratio

X10i, iZ1,2,3 Deposits to net worth ratio

X11i, iZ1,2,3 Fixed assets to net worth ratio

X12i, iZ1,2,3 Current reserve ratio

X13i, IZ1,2,3 Total assets turnover (times)

X14i, iZ1,2,3 Deposit to loan ratio

X15i, iZ1,2,3 Non-performing loan ratio

X16i, IZ1,2,3 Interest payments to annual average balance of deposits

ratio

X17i, iZ1,2,3 Interest income to annual average balance of credit

extension ratio

X18i, iZ1,2,3 Average amount of business income per employee

X19i, IZ1,2,3 Average amount of profit per employee

X20i, iZ1,2,3 Return on total assets

X21i, iZ1,2,3 Return on shareholders’ equity

X22i, iZ1,2,3 Operating income to paid-in capital

X23i, iZ1,2,3 Profit before tax to paid-in capital

X24i, iZ1,2,3 Net profit to sales

X25i, iZ1,2,3 Earnings per share (NTD)

(6)

5.1. The overall accuracy rates of the SVM models

FromTable 4, we can see that the overall accuracy rate of the ten models is between 89 and 100% in the training set, which contains both training and validation samples, and between 73.08 and 84.62% in the test set. The SVM_5, SVM_6, SVM_8 and SVM_9 models have the best accuracy rates of 84.62%. Among them, SVM_5 is the most cost-efficient model because it has the least number of variables and the best accuracy rate. Clearly, the accuracy rate of the SVM_1 model, which contains all of the variables, is lower than models that have fewer variables. In other words, more information does not necessarily improve accuracy rates.

5.2. The impact of the input variables on accuracy rates In conventional bond rating models, financial variables are the major consideration in designing classification models, especially if only a one-year data set is used. In addition to the financial variables, we included three non-financial variables

and financial data for the previous three years. FromTable 4we

can observe the following:

& From the comparison of SVM_1 and SVM_2 as well

as SVM_3 and SVM_7, it is clear that the one-year data set performs better than the three-year data set. This implies that the most recent year’s data plays an important role in the rating decision, and may explain why most early research only applied one-year data to rating decision problems.

& From the comparison of SVM_3, SVM_4, SVM_5

and SVM_6, the latter three models perform better than the first model by increasing the overall accuracy rate from 4 to 8%. Clearly, the three non-financial

variables increase the accuracy of the rating models, especially in the proportion of the major shareholders’ holding and the average stock price. In this case, the government’s holding proportion is not as good as the major shareholders’ holding proportion in the rating decision.

& Comparison of SVM_1 and SVM_3, shows that

including the three non-financial variables does not improve the performance of the model that only uses financial variables. However, by selecting only one-additional-variable model (SVM_4, SVM_5 or SVM_6) or two-additional-variables model (SVM_8 or SVM_9) the non-financial variables may increase the overall accuracy rate.

In the first observation, although one-year financial variables may be the most important prediction factors, the integration of the major shareholders’ holding proportion and average stock price with three-year financial variables can improve the performance of the rating model.

5.3. Analysis of confusion matrices and misclassified samples in test models

To test the classification ability of SVM models in detail, we

chose two confusion matrices for further analysis. Table 5

shows the confusion matrix of a full-variable model, SVM_1,

while Table 6 shows the confusion matrix of the best

performance model SVM_5. The latter consists of the major shareholders’ holding proportion and 3-year financial vari-ables. In the training set, the overall accuracy rate and the average accuracy rate of each of the four classes was 100%. However, in our test set, the accuracy rates declined to 76.92% for SVM_1 and 84.62% for SVM_5. In the test set, the models

Table 5

Confusion matrix and accuracy rate of the SVM_1 model

Predicted Training set Test set

Target StwAA TwA TwBBB &twBB Accuracy

rate (%)

StwAA twA twBBB &twBB Accuracy rate (%)

StwAA 16 0 0 0 100.00 6 0 0 0 100.00

twA 0 25 0 0 100.00 1 7 1 0 77.78

twBBB 0 0 26 0 100.00 0 3 6 0 66.67

&twBB 0 0 0 6 100.00 0 0 1 1 50.00

Overall accuracy rate 100.00 76.92

Average accuracy rate 100.00 73.61

Table 6

Confusion matrix and accuracy rate of the SVM_5 model

Predicted Training set Test set

Target StwAA twA twBBB &twBB Accuracy

rate (%)

StwAA twA twBBB &twBB Accuracy

rate (%)

StwAA 16 0 0 0 100.00 6 0 0 0 100.00

twA 0 25 0 0 100.00 0 8 1 0 88.89

twBBB 0 0 26 0 100.00 0 2 7 0 77.78

&twBB 0 0 0 6 100.00 0 0 1 1 50.00

Overall accuracy rate 100.00 84.62

(7)

performed very well in classifying StwAA, which did not have any misclassified samples. However, for the other lower ratings, the accuracy rates declined according to the rating level. For example, in the lowest class, & twBB, the accuracy rate declined to 50%. We found that these misclassified samples were either downgraded or upgraded one class; most of them were upgraded. This may be because TRC’s analysts use some input variables that are not included in our automatic

rating model. Tables 7 and 8 show the samples that were

misclassified by the models. We further analyzed the difference between the TRC ratings and our rating model regarding these misclassified samples. The model downgraded the rating of the farmers bank of China in 2003, as some financial ratios related to profitability slumped in 2002. The bank sustained large losses because it had to write off bad loans to meet the new financial regulation that the non-performing loan ratio of all banks should be reduced to less than 2.5%. Although the TRC maintained the bank’s twA rating, we downgraded it to twBBB. With regard to the upgrade example, such as the rating of Cosmos Bank in 2001, the rating was ‘twBBB’ on 18 June 2001, but then downgraded to twBB after two months because of the poor outlook for the domestic economy at that time. Our rating model finds it difficult to detect such changes.

5.4. Comparison of the SVM and back propagation models We used a benchmark model, the back propagation algorithm (BP), which is famous for solving classification problems in the AI domain, to make a comparison with the SVM_1 model. Due to the possibility of a local optimum in the

BP model, it is adequate to execute the model many times (35 times in this study) and then compare the accuracy rates with those generated by the SVM model.

To see if the SVM model outperforms the BP model, the Wilcoxon signed-rank test is used for testing. The hypothesis is as follows.

H0 Accuracy rate of BPZAccuracy rate of SVM

H1 Accuracy rate of BP!Accuracy rate of SVM

As a result, the statistical test ZZK4.7663!Z(aZ 0.01)ZK2.326, which means that the SVM model is significantly superior to the BP model. We chose the BP model whose overall accuracy rate is closest to the mean of the overall accuracy rates of all BP models. The training and test

results of both models are shown inTable 9. It is clear that the

BP model is inferior to the SVM model in both the training and test sets according to all the performance criteria. However, in classifying the twBBB class, the BP model has a powerful identification capability.

In addition, we useTable 10to make a comparison for three

methods, SVM, BP and linear regression (LR), by using ten

model designs. The detailed design can be referred toTable 4.

We execute a matched-pair Wilcoxon signed-rank test on the matched-pair sample composed of accuracy rates of SVM and BP models, and the result shows that SVM is significantly superior to BP models (ZZK2.814). Besides, the test is also executed on the other matched-pair samples composed of accuracy rates of SVM and LR models. The result shows that SVM is significantly superior to LR models (ZZK2.823) as well.

Table 7

Misclassified samples by SVM_1

Sample Year Target Predict Error

happened in International Bank of

Taipei

2003 twA StwAA Upgrade

The Farmers Bank of China

2003 twA TwBBB Downgrade

Asia Pacific Commercial Bank

1999 twBBB TwA Upgrade

Cosmos Bank, Taiwan 2001 &twBB TwBBB Upgrade Hsinchu International Bank 2001 twBBB TwA Upgrade Hsinchu International Bank 2002 twBBB TwA Upgrade Table 8 Misclassified samples by SVM_5

Sample Year Target Predict Error

happened in The Farmers Bank of

China

2003 twA TwBBB Downgrade

Asia Pacific Commercial 1999 twBBB TwA Upgrade Grand Commercial Bank 2001 twBBB TwA Upgrade Cosmos Bank, Taiwan 2001 &twBB TwBBB Upgrade

Table 9

Accuracy Rate of the SVM model and the BP model

Models Training set Test set

Ratings SVM (%) BP (%) SVM (%) BP (%)

StwAA 100.00 81.25 100.00 33.33

TwA 100.00 84.00 77.78 33.33

TwBBB 100.00 96.15 66.67 100.00

&twBB 100.00 16.67 50.00 0.00

Overall accuracy rate 100.00 82.19 76.92 53.85 Average accuracy rate 100.00 69.52 73.61 41.67

Table 10

Accuracy rates of the SVM, BP and LR models (test set)

Models SVM (%) BP (%) LR (%) Model_1 76.92 53.85 26.92 Model_2 80.77 61.53 50 Model_3 76.92 53.85 38.46 Model_4 80.77 61.54 30.77 Model_5 84.62 61.54 38.46 Model_6 84.62 61.54 23.08 Model_7 80.77 53.85 50 Model_8 84.62 50 34.62 Model_9 84.62 65.38 34.62 Model_10 73.08 50 30.77 Mean 80.77 57.31 35.77

(8)

5.5. Concluding remarks

Our test results showed a high overall accuracy rate of 84.62% for the models proposed. The results are much better than a random probability of 25%. For the multiple-class classification problem, the performance of some of the SVM models is better than that of previous research models that applied linear regression MDA, BP and CBR methods (see

Table 11). In addition, each SVM model outperforms the benchmarking BP model.

To provide a full picture of the credit rating status of all commercial banks in Taiwan, we also collected the data of the only 3 banks not yet rated by TRC: Chang Hwa Commercial Bank, Taitung business bank and far eastern international bank. We then applied the SVM_1 model to provide their credit

rating in Table 12. This information hopefully may provide

investors valuable advices.

6. Conclusions and suggestions

An issuer credit rating is an important type of credit risk information, yet few researchers have focused on this area. We applied a novel classification technique, the SVM method that is famous for high dimension classifications, to construct an automatic classification model for rating decisions. The applicability of SVMs to the credit rating problem was examined using a data set composed of various types of

information about Taiwan’s banking companies. The results show that our classification is more accurate than the benchmark BP model, and superior to previous research on multiple-class credit rating problems.

The new input variables, i.e. stock market information and financial support by major shareholders and by the government are intended to enhance the classification effectiveness of credit ratings. The results show that the average stock price of each share in the previous year and the proportion of shares in a company held by the major shareholders who hold more than 10% of the outstanding shares can increase the overall accuracy rate to 84.62%, while the proportion of shares held by the government only increases the rate to 80.77%. Future research may take these factors into consideration. The combination of all the input variables reduced the accuracy rate, which means that including more information does not guarantee higher accuracy rates.

Extending the data period of financial variables from one to three years reduces the accuracy rate of our model. This implies that the most recent year’s financial data plays the major role in rating decisions. However, the combination of three-year financial data and non-financial variables performs better than the combination of one-year financial data and non-financial variables. The mixed effect that multi-year data has on rating models requires further study.

References

Atiya, A. F. (2001). Bankruptcy prediction for credit risk using neural networks: A survey and new results. IEEE Transactions on Neural Networks, 12(4), 929–935.

Baesens, B., Van Gestel, T., Viaene, S., Stepanove, M., Suykens, J., & Vanthienen, J. (2003). Benchmarking state of the art classification algorithms for credit scoring. Journal of Operations Research Society, 54(6), 627–635.

Belkaoi, A. (1980). Industrial bond ratings: A new look. Financial Manage-ment, Autumn, pp. 44–51.

Burbidge, R., Trotter, M., Buxton, B., & Holden, S. (2001). Drug design by machine learning: Support vector machines for pharmaceutical data analysis. Computers and Chemistry, 26, 5–14.

Cai, Y.-D., & Lin, X.-J. (2002). Prediction of protein structural classes by support vector machines. Computers and Chemistry, 26, 293–296. Cortes, C., & Vapnik, V. (1995). Support vector network. Machine Learning,

20(3), 273–293. Table 11

Summary of research approaches and their test results

Research Rating targets Methodology Output class Overall accuracy rate

Horrigan, 1966 Moody’s and S&P’s bond ratings Linear regression Six classes 58%(Moody’s)

52%(S&P’s)

Pinches and Mingo, 1975 Moody’s bond ratings Quadratic MDS Five classes 75.4%

Belkaoi, 1980 S&P’s bond ratings MDS Six classes 62.8%

Dutta and Shekhar, 1988 S&P’s bond ratings BP Two classes 83.3%

Surkan and Singleton, 1990 Moody’s bond ratings for AT&T subcompanies BP Two classes 88%

Maher and Sen, 1997 Moody’s bond ratings BP Six classes 70%

Shin and Han, 2001 Commercial papers’ ratings of Korean rating agencies

CBR Five classes 70.0%a

This paper Issuers’ credit ratings of banking industry SVM Four classes 84.62%

a

The accuracy rate is measured by the average accuracy rate of the model.

Table 12

Credit ratings information regarding TRC’s unrated banks

Bank Year Ratings give by

the SVM_1 model

Chang Hwa Commercial Bank 2001 twBBB

Chang Hwa Commercial Bank 2002 StwAA

Chang Hwa Commercial Bank 2003 StwAA

Taitung Business Bank 2001 twBBB

Taitung Business Bank 2002 &twBB

Taitung Business Bank 2003 twBBB

Far Eastern International Bank 2001 StwAA Far Eastern International Bank 2002 twA Far Eastern International Bank 2003 twA

(9)

Dutta, S., & Shekhar, S. (1988). Bond rating: A non-conservative application of neural networks. Proceedings of the IEEE International Conference on Neural Networks, II, 443–445.

Ederington, L. H. (1985). Classification models and bond ratings. The Financial Review, 20(4), 237–262.

Friedman, J. (1996). Another approach to polychotomous classification. Technical report, Department of Statistics, Stanford University. Available athttp://www-stat.stanford.edu/reports/friedman/poly.ps.Z.

Horrigan, J. O. (1966). The determination of long term credit standing with financial ratios. Journal of Accounting Research, Suppl., 44–62. Hsu, C.-W., Chang, C.-C., & Lin, C.-J. (2003). A practical guide to support

vector classification. Department of Computer Science and Information Engineering, National Taiwan University.

Knerr, S., Personnaz, L., & Dreyfus, G. (1990). Single-layer learning revisited: A stepwise procedure for building and training a neural network. In J. Fogelman (Ed.), Neurocomputing: Algorithms, architectures & appli-cations. Berlin: Springer.

Maher, J. J., & Sen, T. K. (1997). Predicting bond ratings using neural networks: A comparison with logistic regression. Intelligent Systems in Accounting, Finance and Management, 6, 59–72.

Mar, M. C., Apellaniz, C. G., & Cinca, C. S. (1996). A multivariate study of spanish bond ratings. Omega, 24(4), 451–462.

Minoux, M. (1986). Mathematical programming: Theory and algorithms. London: Wiley.

Morris, C. W., & Autret, A. (2001). Support vector machines for identifying organisms—A comparison with strongly partitioned radial basis function networks. Ecological Modelling, 146, 57–67.

Pinches, E., & Mingo, K. A. (1973). A multivariate analysis of industrial bond ratings. Journal of Finance, March, 1–18.

Pinches, E., & Mingo, K. A. (1975). The role of subordination and industrial bond ratings. Journal of Finance, March, 201–206.

Shin, K.-S., & Han, I. (1999). Case-based reasoning supported by genetic algorithms for corporate bond rating. Expert Systems with Application, 16, 85–95.

Shin, K.-S., & Han, I. (2001). A case-based approach using inductive indexing for corporate bond rating. Decision Support Systems, 32, 41–52. Surkan, A. J., & Singleton, J. C. (1990). Neural networks for bond rating

improved by multiple hidden layers. Proceedings of the IEEE International Conference on Neural Networks, II, 163–168.

Tay, F. E. H., & Cao, L. (2001). Application of support vector machines in financial time series forecasting. Omega, 29, 309–317.

Van Gestel, T., Suykens, J., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., et al. (2004). Benchmarking least squares support vector machine classifiers. Machine Learning, 54(1), 5–32.

Vapnik, V. N. (1999). The nature of statistical learning theory (2nd ed.). New York: Springer.

West, R. R. (1970). An alternative approach to predicting corporate bond ratings. Journal of Accounting Research, Spring, 118–127.