出席國際學術會議心得報告
4. Empirical Tests and Results
5.3 Empirical Results
5.3.1 Testing Results of Default Definition I
We first present in Table 7 the performance of default prediction by decile-based analysis and provide the percentages of performance delisting in each decile. Defaulting firms are sorted into deciles by corresponding physical default probability estimates of each model, where the physical default probabilities of firms for the in-sample and out-of-sample tests are computed
one year (252 trading days) before the delisting date, respectively. One can clearly find that the Merton and the Black and Cox models outperform the Brockman and Turtle model, especially in the out-of-sample predictions.
We next present in Figure 3, Figure 4, and Figure 5, respectively, the in-sample, out-of-sample (six-month) and out-of-out-of-sample (one-year) ROC curves of the tested models. Formal statistical tests are carried out by the Accuracy Ratios (ARs) and the z statistics. Z statistics, compared with the Merton model, for the tested models are reported in the parentheses in Table 8. We find that in accordance with the results in the decile-based analysis, the Brockman and Turtle model is clearly inferior to the Merton and the Black and Cox models.
The Leland model of in-sample test in both tax rate settings also underperforms the Merton model.
Figure 3 ROC Curves – One Week In-Sample Test (All Sample)
ROC - One Week
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%
Default Probability of Survival Group ordered by Percentiles
Default Probability of Default Group ordered by Percentiles
Merton BT BC Leland
Figure 4 ROC Curves – Six-Month Out-of-Sample Test (All Sample)
ROC - 6 Months
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%
Default Probability of Survival Group ordered by Percentiles
Default Probability of Default Group ordered by Percentiles
Merton BT BC Leland c
Figure 5 ROC Curves – One-Year Out-of-Sample Test (All Sample)
ROC - 1 Year
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%
Default Probability of Survival Group ordered by Percentiles
Default Probability of Default Group ordered by Percentiles
Merton BT BC Leland
Our empirical result shows that the simple Merton model surprisingly outperforms the flat barrier model in default prediction. Furthermore, the performance of the Merton model is also similar to that of the Black and Cox model in all tests. The Black and Cox model has slightly higher ARs than those of Merton’s model, however, the differences are not statistically significant based on the z test. Moreover, Merton’s model also performs significantly better than the Leland model of the in-sample test.
The results of z test indicate that the difference of prediction capability between the Merton and the flat barrier models is statistically significant and the results hold for both in-sample and out-of-sample tests. Although theoretically the down-and-out option framework should nest the standard call option model, practically it may not perform better in the default prediction. Several possible reasons may explain our empirical results.
One of the possible explanations is that the continuous monitoring assumption of the flat barrier model makes it possible to default before debt maturity, and thus increases the estimated default probabilities of the survival firms. One may argue that the implied default probabilities of the default firms increase as well. However, the magnitude of the increments may not be the same, and we do observe this in our empirical results.
For example, the case of Alfacell Corporation, a survival firm, (CRSP permanent company number 35) clearly reflects this issue as shown in Figure 6. Alfacell experienced a drastic downfall of share prices in year 2005. However, it still survived through the end of 2006. In Figure 6, we present the one-year market equity, the estimated firm value of the Merton model, the estimated firm value of the Brockman and Turtle model, the implied barrier, and the debt level of the KMV formula, respectively. Both models generate reasonable firm value estimates based on the corresponding model assumptions. Estimated firm values of the flat barrier model are higher than those of the Merton model due to the existence of the claims of the bondholders modeled as the down-and-in option. The implied default probability of Alfacell Corporation is merely 0.04% by the Merton model, while the default probability of the flat barrier model is as high as 61.21%. The gigantic difference comes from the implied default barrier. The debt level by the KMV formula is $1.75 million, but the implied barrier from the Brockman and Turtle model is $31.37 million! Such a high implied barrier leads to a high default probability by the flat barrier model. In contrast, default in Merton’s model is only related to the debt level at debt maturity and thus the default probability is very low.
Note that to prevent from the local optimum problem of the barrier estimate, we also use another optimization routine, the fmincon function in Matlab, to re-estimate the Alfacell case but still obtain the same implied default barrier.
One may argue that imposing constraints on the default barrier can solve this issue. However, the high implied default barrier is a result of the return distribution of the equity value process. Imposing constraints clearly violates the fundamental of the maximum likelihood estimation method and hinders the MLE method from searching the global optimum. In the case of Alfacell Corporation, the likelihood function of the Brockman and Turtle model and the Merton model are 566.397 and 562.288, respectively. This indicates that the introduction of the barrier does improve the fitting of the return distribution of the equity value process.
Furthermore, the equity pricing function of the flat barrier model in Equation (A.3) does not pre-specify the location of the barrier. The flat default barrier can be higher than the debt level, as assumed in the Brockman and Turtle model. Accordingly, the fundamental issue is that the flat barrier assumption itself might be unreasonable and unrealistic. Finally, we
should note that the extraordinarily high implied default barrier cannot happen in the Black-Cox model since it assumes that the default barrier is lower than the debt level. As a result, the implied default probability of Alfacell Corporation is only 0.06% by the Black and Cox model.
Another possible explanation is from our measure of the default prediction capability. The AR only preserves the ranking information of the default probabilities in our empirical test. The flat barrier model may generate the default probability distribution closer to the true default probability distribution, compared with that of the Merton model. It is the tails of the default probability distributions of survival and default groups that truly determine the ARs.
Nonetheless, one can clearly observe from the decile-based results in Table 7 that the Brockman and Turtle model does not have the same differentiating power for default and survival groups as that of the Merton model.
Finally, we cannot completely rule out the local optimum possibility, since it is well known that high dimensional optimization may not uncover the global optimum. The superior default prediction capability of the Merton model may come from the better estimates of model parameters due to its simpler likelihood function and lower dimension in the optimization procedure.
We next turn to the sub-sample analysis by financial (Table 9) versus non-financial (Table 10) firms. Financial companies have industry-specific high leverage ratios and thus cannot be modeled well in finance literature. Consistent with the findings by Chen, Hu, and Pan (2006), we find that the Brockman and Turtle model perform much better in finance sector than in the industrial sector, while the Merton and the Black and Cox models perform better in the industrial sector. Accordingly, the difference of default prediction power of the flat barrier and the Merton model in finance sector is no longer significant.
Another important finding is that the Leland model outperforms Merton’s model in the Non-financial sector, and the differences are significant in the six-month and one-year out-of-sample tests. The Leland model shows large differences of default predictability between financial and non-financial sectors. This difference leads to its superior power of prediction in non-financial sector.
Finally, we turn to the discussion of default barriers. Unlike the Wong and Choi (2009), we do not present the barrier-to-debt ratio and have our inference based on it. This is because of the nature of likelihood function of down-and-out option framework, one cannot expect to pin down the barrier when the barrier is low relative to the asset value, i.e., the default probability is low. Therefore, to get rid of this bias, we present the differences of default probabilities between barrier models and Merton’s model. Our results in Table 11, Table 12, and Table 13 show that the introduction of default barriers does influence default probabilities, especially on the default group. However, for most of the survival firms and around 30% of the firms in the default group, the impact is small. This in turn indicates that exogenous flat or exponential barriers do not have significant impact in equity pricing for these companies. Thus, our empirical finding is consistent with the results by Wong and Choi (2009) and does not support the finding by Brockman and Turtle (2003) that default barriers are significantly positive.
Table 7 Percentages of Performance Delisting firms in Each Decile (Default Definition I) In Sample Test One Week
15,598 firms – 10,727 survival and 4,871 performance delisting firms
Decile (P ) Def Merton Brockman and Turtle Black and Cox Leland (TC=20%) Leland (TC=35%)
1 (Large) 30.86% 30.82% 31.02% 30.65% 30.63%
2 28.27% 28.04% 28.33% 28.68% 28.80%
3 22.34% 20.80% 22.28% 21.64% 21.58%
4 9.46% 8.91% 9.16% 8.97% 8.89%
5 3.37% 4.52% 3.49% 3.63% 3.55%
6-10 (Small) 5.71% 6.92% 5.73% 6.43% 6.55%
Out of Sample Test Six Months
14,765 firms - 10,232 survival and 4,533 performance delisting firms
Decile (P ) Def Merton Brockman and Turtle Black and Cox Leland (TC=20%) Leland (TC=35%)
1 (Large) 28.04% 27.05% 28.08% 27.91% 27.95%
2 24.69% 22.81% 24.88% 24.73% 24.75%
3 18.47% 17.36% 18.60% 17.67% 17.63%
4 11.03% 12.11% 11.01% 11.21% 11.23%
5 7.32% 7.54% 7.10% 6.86% 6.93%
6-10 (Small) 10.46% 13.13% 10.32% 11.63% 11.52%
Out of Sample Test 1 Year
13,744 firms - 9,637 survival and 4,107 performance delisting firms
Decile (P ) Def Merton Brockman and Turtle Black and Cox Leland (TC=20%) Leland (TC=35%)
1 (Large) 26.83% 25.66% 26.91% 26.86% 26.86%
2 22.23% 20.50% 22.40% 21.50% 21.48%
3 17.09% 16.78% 17.39% 17.34% 17.36%
4 12.25% 12.15% 11.91% 12.66% 12.86%
5 8.01% 8.13% 7.74% 8.04% 7.94%
6-10 (Small) 13.59% 16.78% 13.66% 13.61% 13.51%
Table 8 Accuracy Ratios and z Statistics of Physical Probabilities (Default Definition I; All Sample) Accuracy Ratio Merton Brockman and Turtle Black and Cox Leland (TC=0.2) Leland (TC=0.35) One Week
(In Sample) 0.9357 0.9253 (-5.8513) 0.9365 (0.7667) 0.9314 (-2.3810) 0.9315 (-2.1933) Six Months
(Out of Sample) 0.8749 0.8531 (-8.5565) 0.8768 (1.5632) 0.8705 (-1.6938) 0.8711 (-1.3984) One Year (Out
of Sample) 0.8422 0.8156 (-8.8537) 0.8433 (0.8055) 0.8442 (0.6621) 0.8449 (0.8316) In-Sample One-Week (15,598 firms – 10,727 survival and 4,871 performance delisting firms)
Out-of-Sample 6-Month (14,765 firms - 10,232 survival and 4,533 performance delisting firms) Out-of-Sample 1-Year (13,744 firms - 9,637 survival and 4,107 performance delisting firms)
Table 9 Accuracy Ratios and z Statistics of Physical Probabilities (Default Definition I; Financial Firms) Accuracy Ratio Merton Brockman and Turtle Black and Cox Leland (TC=0.2) Leland (TC=0.35) One Week
(In Sample) 0.8939 0.8900 (-0.5698) 0.8926 (-0.3598) 0.8750 (-3.1532) 0.8744 (-3.0896) Six Months
(Out of Sample) 0.8496 0.8539 (0.5305) 0.8520 (0.5858) 0.8209 (-3.9062) 0.8199 (-3.7674) One Year (Out
of Sample) 0.8319 0.8240 (-0.8894) 0.8333 (0.3162) 0.8083 (-2.7714) 0.8097 (-2.6578) In-Sample One-Week (2,809 firms – 2,409 survival and 400 performance delisting firms)
Out-of-Sample 6-Month (2,694 firms – 2,313 survival and 381 performance delisting firms) Out-of-Sample 1-Year (2,556 firms – 2,195 survival and 361 performance delisting firms)
Table 10 Accuracy Ratios and z Statistics of Physical Probabilities (Default Definition I; Non-Financial Firms) Accuracy Ratio Merton Brockman and Turtle Black and Cox Leland (TC=0.2) Leland (TC=0.35) One Week
(In Sample) 0.9371 0.9255 (-6.2231) 0.9380 (0.8838) 0.9373 (0.0707) 0.9376 (0.2090) Six Months
(Out of Sample) 0.8714 0.8437 (-10.0585) 0.8729 (1.1717) 0.8777 (2.2951) 0.8786 (2.5352) One Year (Out
of Sample) 0.8379 0.8054 (-9.8963) 0.8385 (0.3975) 0.8543 (5.1790) 0.8555 (5.2588) In-Sample One-Week (12,789 firms – 8,318 survival and 4,471 performance delisting firms)
Out-of-Sample 6-Month (12,071 firms – 7,919 survival and 4,152 performance delisting firms) Out-of-Sample 1-Year (11,188 firms – 7,442 survival and 3,746 performance delisting firms)
Figure 6 An Illustration of the Problem of the Brockman and Turtle Model by Alpacell Corporation
ALFACELL CORP
0 20 40 60 80 100 120 140 160 180
0 50 100 150 200 250
Trading Days
Value (Million Dollars)
Market Equity
Estimated Firm Value (Merton) Estimated Firm Value (BT) Debt Level
Default Barrier
Table 11 The Effect of Default Barriers in Terms of the Default Probabilities (In-Sample Test)
Difference of Default Probabilities BT: PDef(BT)-PDef(Merton) BC: PDef(BC)-PDef(Merton)
All Survival Default
Percentile BT BC Percentile BT BC Percentile BT BC 5% -0.010% 0.000% 5% -0.010% 0.000% 5% -0.009% -0.001%
10% 0.000% 0.000% 10% 0.000% 0.000% 10% -0.001% 0.000%
20% 0.000% 0.000% 20% 0.000% 0.000% 20% 0.000% 0.001%
30% 0.000% 0.000% 30% 0.000% 0.000% 30% 0.001% 0.045%
40% 0.000% 0.000% 40% 0.000% 0.000% 40% 0.226% 0.277%
50% 0.003% 0.000% 50% 0.000% 0.000% 50% 1.562% 0.879%
60% 0.415% 0.005% 60% 0.015% 0.000% 60% 5.293% 2.090%
70% 3.321% 0.141% 70% 0.808% 0.001% 70% 12.572% 4.295%
80% 12.113% 1.294% 80% 6.233% 0.024% 80% 23.580% 7.785%
90% 30.705% 5.947% 90% 23.732% 0.760% 90% 39.896% 11.195%
95% 47.584% 10.034% 95% 42.673% 3.838% 95% 53.779% 13.118%
Mean 7.540% 1.421% Mean 5.685% 0.504% Mean 11.729% 3.491%
Standard
Deviation 17.314% 3.789% Standard
Deviation 16.407% 2.099% Standard
Deviation 18.537% 5.538%
Absolute
Difference
<0.1% 51.673% 67.057% <0.1% 59.548% 83.278% <0.1% 33.885% 30.419%
<0.5% 57.465% 74.028% <0.5% 65.012% 88.155% <0.5% 40.419% 42.119%
<1% 60.920% 77.788% <1% 67.875% 90.334% <1% 45.210% 49.448%
<5% 71.156% 88.348% <5% 76.681% 95.700% <5% 58.675% 71.744%
Table 12 The Effect of Default Barriers in Terms of the Default Probabilities (Six–Month Out-of-Sample Test) Difference of Default Probabilities BT: PDef(BT)-PDef(Merton) BC: PDef(BC)-PDef(Merton)
All Survival Default
Percentile BT BC Percentile BT BC Percentile BT BC 5% -0.002% 0.000% 5% 0.000% 0.000% 5% -0.252% -0.160%
10% 0.000% 0.000% 10% 0.000% 0.000% 10% -0.004% 0.000%
20% 0.000% 0.000% 20% 0.000% 0.000% 20% 0.000% 0.001%
30% 0.000% 0.000% 30% 0.000% 0.000% 30% 0.002% 0.054%
40% 0.000% 0.000% 40% 0.000% 0.000% 40% 0.164% 0.282%
50% 0.001% 0.000% 50% 0.000% 0.000% 50% 0.936% 0.852%
60% 0.197% 0.002% 60% 0.002% 0.000% 60% 2.806% 1.928%
70% 1.681% 0.081% 70% 0.310% 0.000% 70% 7.644% 4.020%
80% 6.862% 0.925% 80% 3.002% 0.004% 80% 17.188% 7.569%
90% 21.244% 5.191% 90% 13.707% 0.273% 90% 34.410% 11.534%
95% 36.824% 9.767% 95% 27.255% 1.829% 95% 48.125% 13.906%
Mean 5.779% 1.287% Mean 4.109% 0.330% Mean 9.455% 3.393%
Standard
Deviation 13.405% 3.747% Standard
Deviation 11.173% 1.801% Standard
Deviation 16.757% 5.601%
Absolute
Difference
<0.1% 55.166% 68.382% <0.1% 65.772% 87.081% <0.1% 31.822% 27.226%
<0.5% 61.851% 75.471% <0.5% 70.936% 91.023% <0.5% 41.855% 41.239%
<1% 65.754% 79.336% <1% 74.012% 93.018% <1% 47.579% 49.220%
<5% 77.157% 89.431% <5% 82.802% 97.204% <5% 64.731% 72.323%
Table 13 The Effect of Default Barriers in Terms of the Default Probabilities (One-Year Out-of-Sample Test) Difference of Default Probabilities BT: PDef(BT)-PDef(Merton) BC: PDef(BC)-PDef(Merton)
All Survival Default
Percentile BT BC Percentile BT BC Percentile BT BC 5% -0.001% 0.000% 5% 0.000% 0.000% 5% -0.026% -0.002%
10% 0.000% 0.000% 10% 0.000% 0.000% 10% -0.001% 0.000%
20% 0.000% 0.000% 20% 0.000% 0.000% 20% 0.000% 0.000%
30% 0.000% 0.000% 30% 0.000% 0.000% 30% 0.000% 0.006%
40% 0.000% 0.000% 40% 0.000% 0.000% 40% 0.055% 0.082%
50% 0.002% 0.000% 50% 0.000% 0.000% 50% 1.160% 0.389%
60% 0.210% 0.003% 60% 0.014% 0.000% 60% 4.384% 1.239%
70% 1.957% 0.064% 70% 0.435% 0.001% 70% 12.333% 2.900%
80% 8.712% 0.774% 80% 3.309% 0.022% 80% 24.328% 5.991%
90% 28.719% 4.472% 90% 17.572% 0.668% 90% 41.073% 10.129%
95% 45.118% 8.937% 95% 37.494% 3.165% 95% 54.392% 12.344%
Mean 2.137% 0.140% Mean 5.095% 0.488% Mean 11.752% 2.797%
Standard
Deviation 8.053% 1.180% Standard
Deviation 13.467% 2.298% Standard
Deviation 18.977% 5.387%
Absolute
Difference
<0.1% 55.530% 69.856% <0.1% 63.474% 83.543% <0.1% 36.888% 37.740%
<0.5% 61.940% 76.979% <0.5% 69.783% 88.627% <0.5% 43.535% 49.647%
<1% 65.221% 80.348% <1% 72.990% 90.661% <1% 46.993% 56.148%
<5% 75.662% 90.403% <5% 82.090% 96.223% <5% 60.579% 76.747%