decay factor for all series: 0.94 for daily data and 0.97 for monthly data (month defined as 25 trading days)
2.4 Back-testing
Backtesting (or back-testing) is the process of evaluating a strategy, theory, or model by applying it to historical data. A key element of backtesting that differentiates it from other forms of historical testing is that backtesting calculates how a strategy would have performed if it had actually been applied in the past.
Backtesting is a common and methodologically accepted approach to research, however a high or successful correlation between a backtested strategy and historical results can never prove a theory correct, since past results do not necessarily indicate future results. Varieties of tests have been proposed to gauge the accuracy of a VaR model. Some of the earliest proposed VaR backtests, e.g. Kupiec (1995), focused exclusively on the property of unconditional coverage. In short, these tests are concerned with whether or not the reported VaR is violated more than α
% of the time. He assumes failure rate (n/T) is binomial distribution.
( , , ) (1 )
In this section, this thesis uses two Taiwan stock index as for electronic and financial industries in Taiwan to calculate value at risk by RiskMetrics, GARCH (1, 1) and LAVE. When calculate value at risk of the portfolio, the proportion of each index
‧
is identical during time. Because the proportion is time-invariant, the distribution of the portfolio can be estimated by Monte Carlo simulation. The time period of data is from 2000/05/19 to 2010/05/18, each index has 2496 observations. First 1496 observations are used to estimate RiskMetrics and GARCH model parameter, the other are used to forecast volatility (LAVE does not specify the parameters).
Apply the FastICA algorithm to extract two independent components from two indexes. We want to compare VaR with different models, so each independent component use three different volatility estimators and two residual distributions to estimate the process. The demixing matrix W is:
09766 1.61416
to get two independent componentsS2 2496× . Table 1 shows statistical property of original index and estimated independent components. All the statistical property of data is slight right-skewed and leptokurtic which are common in financial data.
Before we fit the each IC of GARCH (1,1) and RiskMetrics with different distribution, we calculate volatility of LAVE first. This is because LAVE has different interval of homogeneity periods at different points, we must use the same period in GARCH (1,1) and RiskMetrics to compare. We use the equation (26) to test IC1 and IC2 from 600 to 2496. LAVE need enough past observations to apply testing. Our goal is to forecast from 1497 to 2496, so it is not important to choose the start point when used LAVE.
We apply LAVE, where λ =1.06 and γ=0.5 which are suggested by Mercurio and Spokoiny (2004), to IC1 and IC2. Once we know the interval of homogeneity, we can
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
use equation (14) to calculate corresponding volatility. Figure 1 shows the corresponding homogeneous interval of IC1 and IC2. The average homogeneous interval of IC1 (IC2) is 125.66(154.93), these values are used by GARCH and RiskMetrics when forecasting. Now we use first 1496 observations to estimate the parameter of GARCH and RiskMetrics. Table 2 presents the corresponding coefficient of independent component by GARCH (coefficient of RiskMetrics is λ =0.94). Figure 2 and 3 shows the corresponding volatility process of each IC. The volatility process is very similar by using the same method; although different assume distributions of two ICs. The GARCH has larger volatility than Risk-Metric at the same point. LAVE has the same trend like GARCH and Risk-Metric, but the value are different. The volatility pattern of LAVE is oscillatory, because LAVE is sensitive to a new point.
According to equation (14), the interval of homogeneity change, the volatility change immediately.
Table 1. Statistical summary of index1, index2, IC1 and IC2
Mean Std. skewness kurtosis
Index 1 0 1.18118 0.0249 4.4862
Index 2 0 1.9174 0.0832 4.7682
IC1 0.0082 1.0002 0.2107 4.8767
IC2 0.0054 1.0002 0.0194 4.2442
Table 1. Statistical summary of index1, index2, IC1 and IC2. All the statistical property of data is slight right-skewed and leptokurtic which are common in financial
‧
Table 2. Corresponding coefficient of independent component by GARCH GARCH with normal
Table 2. Corresponding coefficient of independent component by GARCH. According to equation (27), the variance equation of IC1 with normal distribution is
2 2 2
1 t-1
0.004503 0.927154 0.071108
t t
σ = + σ − + ε
Figure 1. Homogeneous interval of IC1 and IC2
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 1. Homogeneous interval of IC1 and IC2.These interval are estimated by
~ ~
~ ~
2 2
\ \
|θI J−θJ |>λ ν( I J + νJ ), and used to calculate
^ 2
1 2
| |t I Rt
τ I σ
∈
= ∑ . The pattern of
homogeneous interval is oscillatory, because LAVE estimate volatility at every point, and sensitive to a new point.
Figure 2 : Part 1 Volatility process of IC1 by using GARCH and Risk-Metrics with normal and student T distributions.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 2 : Part 1 Volatility process of IC1 by using GARCH and Risk-Metrics with normal and student T distributions. The volatility processes are estimate by GARCH and RiskMetric according to the parameter which presents in table 2. We calculate one-day-ahead forecast of volatility from 1497 to 2496. IC1 uses past 125 observations and IC2 uses past 155 past observations.
Figure 2 : Part 2 Volatility process of IC2 by using GARCH and Risk-Metrics with
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 2 : Part 2 Volatility process of IC2 by using GARCH and Risk-Metrics with normal and student T distributions. The volatility processes are estimate by GARCH and RiskMetric according to the parameter which presents in table 2. We calculate one-day-ahead forecast of volatility from 1496 to 2495. IC1 uses past 125 observations and IC2 uses past 155 past observations.
Figure 3: Part 1 Volatility process of IC1 by using LAVE
‧
Figure 3: Part 1 Volatility process of IC1 by using LAVE. According to homogeneous
interval at every point, we calculate volatility by
^ 2 vertical line which is the beginning point.
Figure 3: Part 2 Volatility process of IC2 by using LAVE
Figure 3: Part 2 Volatility process of IC2 by using LAVE. According to homogeneous
interval at every point, we calculate volatility by
^ 2 vertical line which is the beginning point.
The keys to evaluate VaR are volatility and quantile function, so the distribution of
‧
portfolio is important. The independent distributions are assumed ‘T’ and ‘Normal’, and it’s possible to find the combination of distribution. The Monte Carlo is a feasible method to estimate the quantile function. We know that
R=bX (32) Where R is return of portfolio, b is proportion of each index, X=return of index.
According to equation (3) and (13), we can represent (32) as
i i i
R =bAσ ε (33) Where R is return of portfolio, b is proportion of each index (equal in this thesis), A is mixing matrix, σi is decided by GARCH, RiskMetrics, or LAVE. εi is a normal or T distribution. This thesis generate d=2 samples, M=10000 observations, repeat 100 times and forecast 1000 days to calculate value at risk of portfolio. We illustrate this Monte Carlo simulation more explicit. For day 1497, we generate 10000 observations from normal or T distribution, and calculate equation (33) which b, A, and σi are
already known, then we can get the simulative distribution of R to calculate the VaR.
We repeat 100 times, and average the VaR. This is VaR at 1497, and we redo the steps for the other 999 days. Figure 4, 5, 6 shows the simulation of value at risk at different alpha. The failure number means the numbers of original return lower than forecasted VaR. The failure number of GARCH with T is 9, GARCH with normal is 32, RiskMetrics with T is 15, RiskMetrics with normal is 31, LAVE with T is 6, and LAVE with normal is 21 at alpha=0.01. The failure number of GARCH with T is 6, GARCH with normal is 18, RiskMetrics with T is 7, RiskMetrics with normal is 22, LAVE with T is 4, and LAVE with normal is 14 at alpha=0.005. The failure number of GARCH with T is 2, GARCH with normal is 12, RiskMetrics with T is 6, RiskMetrics with normal is 13, LAVE with T is 1, and LAVE with normal is 8 at alpha=0.0025. All the failure number is used to backtesting. Table 3 presents the result of simulation for alpha=0.01 with
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
different model and distributions.
Figure 4 One-day-ahead forecast VaR for 1000 days. Alpha=0.01.
Figure 4 VaR time plots of the portfolio with trading strategy b=[0.5 0.5] and risk level alpha=0.01.Three different volatility process (GARCH, RiskMetrics and LAVE), and two distributions (Normal and T). Blue line is original portfolio return. Green line is VaR of GARCH with normal distribution. Red line is VaR of GARCH with T distribution. Indigo line is VaR of LAVE with normal distribution. Pink line is VaR of LAVE with T distribution. Khaki line is VaR of RiskMetrics with normal distribution.
Gray line is VaR of RiskMetrics with T distribution. The failure number of GARCH with T is 9, GARCH with normal is 32, RiskMetrics with T is 15, RiskMetrics with normal is 31, LAVE with T is 6, and LAVE with normal is 21
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 5 One-day-ahead forecast VaR for 1000 days. Alpha=0.005
Figure 5 One-day-ahead forecast VaR for 1000 days. Alpha=0.005. Three different volatility process (GARCH, RiskMetrics and LAVE), and two distributions (Normal and T). Blue line is original portfolio return. Green line is VaR of GARCH with normal distribution. Red line is VaR of GARCH with T distribution. Indigo line is VaR of LAVE with normal distribution. Pink line is VaR of LAVE with T distribution. Khaki line is VaR of RiskMetrics with normal distribution. Gray line is VaR of RiskMetrics with T distribution. The failure number of GARCH with T is 6, GARCH with normal is 18, RiskMetrics with T is 7, RiskMetrics with normal is 22, LAVE with T is 4, and LAVE with normal is 14 at alpha=0.005
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 6 One-day-ahead forecast VaR for 1000 days. Alpha=0.0025.
Figure 6 One-day-ahead forecast VaR for 1000 days. Alpha=0.0025. Three different volatility process (GARCH, RiskMetrics and LAVE), and two distributions (Normal and T). Blue line is original portfolio return. Green line is VaR of GARCH with normal distribution. Red line is VaR of GARCH with T distribution. Indigo line is VaR of LAVE with normal distribution. Pink line is VaR of LAVE with T distribution. Khaki line is VaR of RiskMetrics with normal distribution. Gray line is VaR of RiskMetrics with T distribution. The failure number of GARCH with T is 2, GARCH with normal is 12, RiskMetrics with T is 6, RiskMetrics with normal is 13, LAVE with T is 1, and LAVE with normal is 8 at alpha=0.0025.
‧
Table 3: The value of simulation of each model at alpha=0.01.
Risk-Metrics(N) Risk-Metrics(T) GARCH(N) 1000 -2.76765 0.042671 1000 -3.33102 0.071665 1000 -2.87676 0.046473
GARCH(T) LAVE(N) LAVE(T)
Table 3: The value of simulation of each model at alpha=0.01. For example, mean of VaR of Risk-Metrics with normal distribution for day 1 is -3.248.
Back-testing result of each type model is presents in table 4. The failure rate (n/T) means failure numbers divided by total days of forecast. According to equation (33), we can calculate the value of LR test. The critical value of LR test for alpha=0.01 is
‧
6.63, alpha=0.005 is 7.88, and alpha=0.0025 is 9.14. The star sign is LR test value over critical value. Apparently, normal distribution is not a suitable distribution, no matter what volatility estimator use, the failure rate of given alpha are higher than student t distribution. Locally adaptive volatility estimate may a good estimator which has fewer stars than Risk-Metrics and GARCH by given distribution.
Table 4. Back-testing result of each type model
Risk-Metrics(N) Risk-Metrics(T) GARCH(N) Table 4. Back-testing result of each type model. The critical value of LR test for alpha=0.01 is 6.63, alpha=0.005 is 7.88, and alpha=0.0025 is 9.14. The star sign is LR test value over critical value.
‧
國立 政 治 大 學