Time Varying Biases and the State of the
Economy
Zixiong Xie
aShih-Hsun Hsu
b∗aDepartment of International Economics and Trade,
Research Institute of Natural Resources, Environment & Sustainable Development, College of Economics, Jinan University,
Guangzhou, China
bDepartment of Economics, National Chengchi University,
Taipei, Taiwan
Abstract
This paper is aimed at investigating whether the forecast is optimal given the available information when the forecast is made. Going beyond the pa-pers that study forecast errors based on the model of Nordhaus (1987), we apply a time varying procedure to forecast revisions and further account for the possibility that the duration of the state may also affect the bias. Three testable hypotheses are presented to help researchers test the optimality of forecasts, with the ultimate goal of determining whether these biases depend on the underlying economic state and whether they are persistent with duration of the state. The corresponding bias-corrected forecasts can then be made based on these results. The empirical study finds that, the one-quarter-ahead official forecast of GDP growth in Taiwan indeed suffers from state-dependent biases-persistent under-estimation bias at the rela-tively good state and under-reaction bias which decays with duration at the relatively bad one. Eliminating these biases in the forecast can reduce over 44.0% variation of forecast errors, and pseudo out-of-sample experi-ments further support the fact that the resulting bias-corrected forecast is
∗Correspondance author. Email: shhsu@nccu.edu.tw. Address: Department of Economics,
National Chengchi University, 64, Section 2, ZhiNan Road, Taipei 116, Taiwan. Tel: +886-2-29393091 ext. 51667. Fax: +886-2-9390344.
markedly better than that made by Taiwan’s government and those made using other competing models.
Key words: optimality, economic growth rate, forecast error, over-reaction, under-reaction
JEL Classification: C53, C41, C22
1
Introduction
Given the available information, conventional theory suggests that the optimal forecast in the sense of minimized expected mean squared forecast errors (MSFE), must be the “rational expectation” of the target variable. The optimal forecast is unbiased because the induced forecast error has a zero mean, and it is efficient since the error is un-predictable and orthogonal to the components of the given information set when the forecast is made.
To investigate whether the forecast is optimal, there are two conventional types of regression models proposed in the literature. One directly focuses on the relationship between realized and forecasted values (e.g., Mincer and Zarnowitz, 1969); the other turns to the properties of forecast error (e.g., Nordhaus, 1987, and Holden and Peel, 1990). Note that, the parameters of these two types of models are (implicitly) assumed constant over time, and thus the implied bias (non-zero mean of the forecast error) of a forecast is time-invariant; see e.g., Mincer and Zarnowitz (1969), Nordhaus (1987), Holden and Peel (1990), Loun-gani (2001), Artis and Marcellino (2001), and Gavin and Mandal (2003).
However, many studies of analyzing survey data have shown that forecast per-formance is sometimes strongly correlated with underlying economic conditions;
see, for example, Grunberg and Modigliani (1954), D¨opke (2001), Fildes and
Stekler (2002), Chauvet and Guo (2003), Swanson and van Dijk (2006), Sinclair et al. (2010), Sinclair et al. (2015), and Messina et al. (2015). These empiri-cal evidences reveal that, the bias of a forecast could be time-varying, possibly depending on the underlying economic states and their durations.
To explain the possible time-varying bias of a forecast, Sinclair et al. (2010), Sinclair et al. (2015), and Messina et al. (2015) modify the models of Holden and Peel (1990), Mincer and Zarnowitz (1969), and Nordhaus (1987) by introducing a state variable whose values are ex post identified for the corresponding target
periods. In essence, they aim at investigating whether the forecasts incorporate the knowledge of “future state” of the economy at the time when the forecast is made. Therefore, their models can not be used to infer whether the forecast is optimal indeed. Moreover, they neglect the possibility that the duration of the state may also help to explain the forecast error.
Going beyond the studies mentioned above, this paper proposes a new frame-work to investigate whether the forecast is optimal by extending the regression model of Nordhaus (1987). Since the proposed model accounts for the possibility that the underlying economic state and its duration may also affect the bias, we demonstrate how to detect those time varying biases of a forecast given the available information set at the time when the forecast is made. The innovative features of the proposed model, which cannot be found in previous works when applying to a series of published forecasts are that it can help researchers to test the conventional rationality of a forecast; to tell whether the existing biases depend on the underlying state of the economy when the forecast is made; to determine whether these biases are persistent even though the duration of that economic state is getting longer; and further to construct a feasible bias-corrected forecast with lower MSFE once the forecast is published (which is useful for the real-time analysis in particular).
Based on the framework proposed herein, the real GDP growth (one-quarter-ahead) forecast of Taiwan government is analyzed. The analysis based on 105 real-time data vintages in Taiwan shows that this forecast suffers from time-varying biases; they depend on the underlying economic state and its duration. Specifically, the Taiwan government tends to under-estimate GDP growth during the relatively good states and this bias is persistent. On the contrary, when at the relatively bad economic states, the government under-reacts to the received news but this bias disappears with the duration of the state. Moreover, neither official forecast nor forecasts made using competing models substantially outperform the proposed bias-corrected forecast in pseudo out-of-sample experiments on Taiwan’s GDP growth forecasts.
The rest of this paper is organized as follows. Section 2 introduces the method-ology. Section 3 presents a relevant empirical study of forecast in Taiwan, and Section 4 draws conclusions.
2
Methodology
2.1
Optimal Forecast and Testable Models
Let yt be the realized GDP growth rate at time t, and yt|t−hf be its h-step-ahead
forecast published at time t − h for h > 0. Accordingly, define the (one-step-ahead) forecast error as et≡ yt|t−1f − yt, where et> 0 indicates an over-prediction
while et< 0 indicates an under-prediction.
Let Ωt denote the available information set at time t, then the conventional
forecasting theory claims that the optimal one-step-ahead forecast should be
IE[yt|Ωt−1], the conditional expectation of yt given the information set at time
t − 1, while minimizing the expected MSFE (Patton and Timmermann, 2012; for
example). It immediately follows that, if yft|t−1 is optimal, the resulting forecast
error et should satisfy the moment condition as
IE[et|Ωt−1] = 0, (1)
which also implies that et should have zero mean (yt|t−1f is thus unbiased, say)
and be orthogonal to the variables in Ωt−1 (yt|t−1f is thus efficient, say).
To test for the unbiasedness of a forecast, Holden and Peel (1990) propose the following simple regression:
et = µ + ut, (2)
where ut is the zero-mean un-predictable disturbance at time t − 1. Given this
model, µ = 0 implies the forecast is unbiased while non-zero µ implies a biased
forecast.1 On the other hand, trying to introduce some available variables in
Ωt−1 into the test for the unbiasedness and efficiency of y
f
t|t−1, Nordhaus (1987)
considers the model for the forecast error as
et = α + β · FRt−1+ ut, (3)
where the variable FRt−1 ≡ yt|t−1f − yt|t−2f is known as the forecast revision, which
measures the forecaster’s adjustment of the forecast in response to the new
in-formation received between time t − 2 and t − 1.2 The intercept α indicates the
1Note that, however, the unbiased forecast does not necessarily imply the forecast is efficient
if the residual of regression (2) is serially correlated, see Joutz and Stekler (2000), and Gavin and Mandal (2003).
2For example, a positive revision occurs when the forecaster receives some good news about
the economy. If the forecaster expects the news to have a larger impact on the economy, then the public would see a larger revision in the forecast.
possibly systematic estimation error of the forecasts, and the slope β measures how all the new information is incorporated into the forecast made at time t − 1; β > 0 (β < 0) thus reflects the forecaster’s over-reaction (under-reaction) to the
received information.3 Throughout this paper, we thus interpret the positive α
as the over-estimation bias and the negative one as the under-estimation bias while the positive (negative) β indicates the over-reaction (under-reaction) bias. If the forecast is optimal, then the null that α = β = 0 should not be rejected significantly by the observed data. Besides, from the perspective of model selec-tion, if the estimate of β is significantly non-zero, the model (2) of Holden and
Peel (1990) may not be appropriate for modeling et since the variable FRt−1 can
help to explain et statistically.
2.2
Time Varying Biases and State of the Economy
Since many studies of analyzing survey data have shown that forecast
perfor-mance is sometimes strongly correlated with underlying economic conditions,4
we may further consider the time-varying biases in different state of the economy by modifying above Nordhaus’s model (3) as
et = [α0+ β0· FRt−1] × (1 − ˆst−1) + [α1 + β1· FRt−1] × ˆst−1+ ut, (4)
where ˆst−1is the available estimate (or proxy) of the underlying state of the
econ-omy at time t−1 given the information set Ωt−1. Throughout this paper, ˆst−1= 0
stands for relatively bad state and ˆst−1 = 1 for the relatively good one. Based on
this setting, we can capture the possible time-vary biases—state-dependent over-estimation/under-estimation biases and over-reaction/under-reaction biases. In practice, we would prefer using this model with time-varying biases to Nordhaus’s
model (3) if the data statistically rejects the null that α0 = α1 or/and the null
that β0 = β1.
3In the literature, the intercept α > 0 (α < 0) can be interpreted as the behavioral bias
of optimism (pessimism) while β > 0 (β < 0) captures the behavioral bias of over-reaction (under-reaction) to new information; see e.g., Ehrbeck and Waldmann (1996), Amir and Gan-zach (1998), and Ashiya (2003). In particular, Ehrbeck and Waldmann (1996) establish a structural model and Amir and Ganzach (1998) propose a theory to support these existing behavioral biases.
4See, for example, Grunberg and Modigliani (1954), Zarnowitz (1992), Granger (1996),
D¨opke (2001), Loungani (2001), Fildes and Stekler (2002), Gavin and Mandal (2003), Chauvet and Guo (2003), Swanson and van Dijk (2006), and Ashiya (2007).
2.2.1 Proxy for state variable
When the relatively bad and relatively good states of the underlying economic
conditions are of interest, given the information set Ωt−1, a feasible proxy
vari-able for ˆst−1 at time t − 1 can be constructed as in Ang et al. (2007). This
implementable proxy is modeled using deviations from D-quarter moving aver-age (MA): D-quarter MA: ˆst−1= 0, if yt−1− 1 D D X i=1 yt−1−i ≤ 0, 1, otherwise. (5)
The estimated underlying state of the economy at time t − 1 is identified as a
relatively bad state if yt−1 does not exceed its previous D-quarter moving
aver-age (yt−1 is decelerating); otherwise, the economy would be regarded as being a
relatively good state at time t − 1 (yt−1 is accelerating).
There are some remarks on this proxy. Firstly, the values of this proxy for the state of the underlying economy are not exactly equal to the ones identified ex post by the institute of the government such as National Bureau of Economic Research (NBER) in United States. The exact business cycle dates are not of our interest because they can not be used to test the optimality of the forecast since they are not observed at the time the forecast is made. This proxy in (5) tries to capture the relative momentum of the underlying economy based on the feasible information set instead. Secondly, in most cases, the finalized value of
yt−1also are unobserved at time t − 1 when the forecast for ytis made. However,
some initial, preliminary or estimated values for yt−1 (ˆyt−1say) based on Ωt−1can
be obtained, this MA proxy ˆst−1 thus still can be computed when the value of
yt−1 is replaced with ˆyt−1 in (5). Finally, alternative proxy variable for the state
of the economy may also be obtained by using approaches of band-pass filter proposed by Christiano and Fitzgerald (2003) and business cycle dating algorithm developed by Bry and Boschan (1971) and Harding and Pagan (2001), or derived from the well-known Markov-switching models or state-space framework in the literature. The induced estimation problems, however, are sometimes tedious.
2.3
Proposed Model
Besides the relative state of the economy at time t − 1, the length of the duration of this state until time t − 1 may also affect the released forecast such that the
potential biases are time-varying. Let ˆd0t−1 and ˆd1t−1 denote the duration of the relatively bad and of the relatively good states, respectively, until time t − 1 given ˆ
st−1,5 we thus propose a general model to test the moment condition (1) as
et= h α0exp −γ0 α· ˆd0t−1 + β0exp −γ0 β· ˆd0t−1 FRt−1 i × (1 − ˆst−1) +hα1exp −γ1 α· ˆd1t−1 + β1exp −γ1 β· ˆd1t−1 FRt−1 i × ˆst−1+ ut, (6) where γ0
α, γβ0, γα1 and γβ1 are non-negative state-dependent parameters with
su-perscript “0” standing for the relatively bad state and “1” for the relatively good one.
Some remarks must be made about this model. Firstly, for i = 0, 1 and
j = α, β, if the parameter γi
j is greater than zero, the function exp(−γji · ˆdit−1)
in (6) is nothing but the well-known exponential survivor function; that γi
j thus
can be interpreted as the probability that the forecaster leaves the situation of suffering from the corresponding bias, and the expected duration is known to be (γji)−1.6 This implication is of interest because it helps to elucidate how long these
biases from the forecast would take to die out on average. Secondly, if γi
j = 0,
then it can be interpreted as the situation that the forecaster is not aware of the ongoing economy very well even though the duration of the underlying state is longer, this induced bias is thus persistent. Moreover, this proposed model (6)
degenerates to the model (4) if all of γji, i = 0, 1, j = α, β, are equal to zero;
empirically, picking up either of these two models can thus be data-driven by
testing whether the estimates of γi
j are significantly non-zero.
5Consider for example the case ˆd0
t−1 = 3, which means that ˆst−1 = ˆst−2 = ˆst−3 = 0, and
ˆ
st−4= 1, the economy has stayed in the relative bad state for three periods from t − 3 to t − 1.
Likewise, ˆd0
t−1 = 1 means that the state at time t − 1 (relatively bad state) differs from what
at time t − 2 (relatively good state).
6It is possible to consider other survivor functions such as Weibull or Log-logistic functions.
However, these survivor functions are more general than exponential one because of introducing more parameters, it turns out that we may not easily obtain the convergent estimates of those general survivor functions in practice, especially when the sample period is not long enough. More properties and a further discussion of survivor functions can be found in, for example, Kiefer (1988).
2.4
Inferences and Implications
2.4.1 Three testable hypotheses concerning forecast error
Given the proposed model (6), determination of its value requires the validity (or otherwise) of the conventional theory of optimal forecast to be determined first. Hypothesis 2.1 (Optimal forecast)
If the forecast is optimal in minimizing mean squared forecast error given the
information set at the time that the forecast is made, then α0 = β0 = α1 = β1 = 0.
Other than separately considering four individual tests based on the corre-sponding estimates of these four parameters, three implied null hypotheses are also considered in our empirical study, they are
Ho : α0 = α1 = 0;
Ho : β0 = β1 = 0;
Ho : α0 = β0 = α1 = β1 = 0.
Rejecting any of them suggests that the forecast is either non-optimal or the minimum MSFE is not the only concern of the forecaster.
Besides, going beyond the model of Nordhaus (1987), the proposed model emphasizes the time-varying biases because of the important role of relative state of underlying economy in forming forecast errors. Whether the induced biases are independent of the underlying economic state when the forecast is made is of interest to determine. To this end, the following hypothesis may be tested. Hypothesis 2.2 (State-independent bias)
If all the potential biases are independent of the state of the economy, then α0 =
α1, β0 = β1, γα0 = γα1 and γβ0 = γβ1.
Three implied joint tests are of particular interest: Ho : α0 = α1, γα0 = γ 1 α; Ho : β0 = β1, γβ0 = γ 1 β; Ho : α0 = α1, γα0 = γα1, β0 = β1, γβ0 = γβ1.
Rejecting the first hypothesis implies that the over-estimation and/or under-estimation biases are state-dependent; likewise, rejecting the second one indicates
state-dependent over-reaction and/or under-reaction biases. The third test is the joint test of the first and the second ones, rejecting it means that not all of these biases are state-independent. Besides, any rejection among these three tests shows that employing the model of Nordhaus (1987) to analyze the forecast error is not supported by the data.
Furthermore, if the induced biases does not disappear with the duration of the state, they are persistent. This may come form the situation that the forecaster is not aware of the ongoing economy very well even though the duration of the underlying state is longer. Accordingly, if the bias in the forecast is present, whether this bias is persistent or not may be determined by testing the following hypothesis.
Hypothesis 2.3 (Persistent bias)
(i) If the bias caused by over-estimation or under-estimation of the forecast
is persistent in relatively bad (good) state of the economy, then γ0
α = 0
(γα1 = 0).
(ii) If the bias caused by over-reaction or under-reaction to news is persistent
in relatively bad (good) state of the economy, then γβ0 = 0 (γβ1 = 0).
Regarding to the Hypothesis 2.3, some remarks are worthy of noting. First, the persistent bias during the relative state of the underlying economy can also be viewed as a kind of systematic bias documented by Fildes and Stekler (2002). Second, the proposed Hypothesis 2.3 is somewhat different from the recent work of Coibion and Gorodnichenko (2012, hereafter CG). In CG’s model, the forecast error may occur owing to forecaster’s limited information even he/she is rational. This information rigidities will be lessened as the underlying state has lasted for longer. Thus, they argue “forecast errors converge back to zero over time, as agent’s information sets progressively incorporate the new information.” On the contrary, the violation of Hypothesis 2.3, says that, provided that the forecast is non-optimal, the forecast errors will converge back to zero over time as the biases disappear with the duration of the state; that is, the biases of the forecast still exist with the finite length of the duration, which violates the implication of the optimal forecast.
2.4.2 Bias-corrected forecast
Recall that the main aim of this paper is to investigate whether the
one-step-ahead forecast yt|t−1f is optimal given the information set Ωt−1 at time t − 1. We
thus propose the model (6) induced from the moment condition (1) to detect the possible time-varying and state-dependent biases. Once the relevant parameters of this model for the forecast error can be consistently estimated, an interesting
by-product at time t − 1 is the so-called bias-corrected forecast, ybc
t|t−1 say. It can
be made as
yt|t−1bc = yt|t−1f − ˆet, (7)
where ˆetis an estimate of etbased on model (6) provided the information set Ωt−1.
Notably, at time t − 1, even though et (= y
f
t|t−1− yt) is determined ex post, the
bias-corrected forecast ybc
t|t−1 is still readily obtained because all the explanatory
variables in model (6) are observable when the one-step-ahead forecast yt|t−1f is
published by the forecaster. This property is particularly useful in the real-time analysis because we can obtain the corresponding bias-corrected version of a forecast immediately once the forecast is published. The predictability of bias-corrected forecast yt|t−1bc can not be worse than yft|t−1 if the accuracy of a forecast is measured as the proportion of the volatility in the realization that the forecast correctly accounts for. Accordingly, the following always holds.
1 − var yt− ybct|t−1 var(yt) ≥ 1 − var yt− yt|t−1f var(yt) (8) since yt= yt|t−1f − et= (yt|t−1f − ˆet) − (et− ˆet) = ybct|t−1− (et− ˆet).
2.5
Remarks
While comparing above framework and the proposed model with the existing related works in the literature, there are some remarks.
Other than testing the optimality of yft|t−1 via the moment condition (1) of
forecast error, we may directly compare yt|t−1f with the optimal forecast IE[yt|Ωt−1].
If yft|t−1is optimal given Ωt−1, then we would expect (λ, δ) = (0, 1) in the following
regression model:
yt = λ + δy
f
t|t−1+ ut.
This is known as the model of Mincer and Zarnowitz (1969, hereafter MZ) and is also popular in the literature. Section 3.3.1 will empirically compare the per-formances of this model and some of its variants with the proposed model.
For detecting the time varying biases, Sinclair et al. (2010), Sinclair et al. (2015) and Messina et al. (2015) also modify the Holden-Peel, MZ, and Nordhaus’s mod-els as followed:
et = µ0× (1 − st) + µ1× st+ ut,
yt= λ0× (1 − st) + λ1× st+ δyt|t−1f + ut,
et= [α0+ β0· FRt−1] × (1 − st) + [α1 + β1· FRt−1] × st+ ut,
where the dummy variable st reflects the state of the economy at time “t”. In
essence, they investigate whether the forecasts incorporate knowledge about the “future state” of the economy at the time the forecast is made. However, this
paper tests the moment condition (1) by introducing ˆst−1 in model (4) and thus
(6) to infer if the forecast is optimal. Therefore, the spirit of these models is
different from the proposed. Besides, the values of their state variable st are
identified ex post and are infeasible at the time when the forecast is made; for
example, the values of st are determined based on the NBER dated recession in
Messina et al. (2015).
3
Empirical Study of Forecasts in Taiwan
3.1
Data Description and Proxy for State Variable
In this section, an empirical study of forecasts in Taiwan is conducted based on the proposed framework. The target of interest is the quarterly real output growth rate of Taiwan. The GDP growth forecasts and realized values from 1986Q4 to 2012Q3 are sequentially taken from Quarterly National Economic Trends, routinely published in every February, May, August and November, by the Directorate General of Budget, Accounting and Statistics (DGBAS) in Tai-wan; there are 105 real-time data vintages. Based on the identification of the multiple-quarter-ahead forecast in Chang et al. (2011), the DGBAS has roughly known the past realized quarterly GDP at the time when the one-quarter-ahead forecast is published; the realized data used here are released approximately 6
to 7 weeks after the quarter to which they refer.7 This feature of Taiwan data
7According to the definition of Chang et al. (2011), if the target is the GDP growth in some
quarter, the corresponding DGBAS’s one-quarter-ahead forecast is identified as the forecast published in the second month of that quarter. For example, in February (the middle of Q1), the published forecast for Q1 is recorded as the “one-quarter-ahead” forecast for Q1 GDP
87 88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 -11 -6 -1 4 9 14 RealizedForecast
Realized Taiwan GDP growth DGBAS forecast Realized Forecast
(a) Realized Taiwan GDP growth and its one-quarter-ahead forecast
87 88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 -6 -4 -2 0 2 4 6 8 Realized Greenbook SPF
DGBAS forecast error
based on deviations from 4-quarter moving average
(b) Forecast error and state of the economy
Figure 1: Time series plots of Taiwan real GDP growth, DGBAS forecast and forecast error. The shaded areas of (b) satisfy ˆst= 0 based on deviations from 4-quarter moving average.
makes it possible to construct the proxy for state variable ˆst−1 by using realized
yt−1 directly without estimating yt−1 in advance; cf. (5).
Figure 1 plots the time series data. The upper figure (a) plots the realized
GDP growth rate {yt} and its one-quarter-ahead forecast {yt|t−1f } while the lower
figure (b) plots its corresponding forecast error {et} and the shaded area
repre-sents the periods of relatively bad state for which the proxy is constructed based on deviations from 4-quarter moving average, i.e., D = 4 in (5).
growth rate; likewise, the forecasts for Q2, Q3 and Q4 announced in Q1 are identified as the two-, three-, and four-quarter-ahead forecasts, respectively; see Chang et al. (2011, pp.1070) for details. By the way, at the time releasing the one-quarter-ahead forecast for Q1 GDP growth in February, DGBAS also publishes the initial realized GDP growth of Q4 in the previous year. That’s why we say in this paper that the DGBAS has roughly known the past realized GDP while the one-step-ahead forecast is published.
From Figure 1(a), the Taiwan government seems unable to adjust its predic-tion immediately in response to the news since the forecast generally falls behind the realized Taiwan GDP growth rate, especially during volatile periods. Figure 1(b) demonstrates that the DGBAS forecast error is likely correlated with the state of the economy. That is, the DGBAS tends to over-predict the GDP growth rate during decelerating GDP environments and to under-predict it during ac-celerating ones. Moreover, the magnitude of DGBAS forecast error is observed to vary with the duration of the state; the absolute error initially increases and then decreases in most cases.
3.2
Results of Conventional Unbiasedness and Efficiency Analyses
To test whether the Taiwan forecast is optimal or not, we first employ some conventional models proposed in the literature; Holden-Peel regression model can be used to test for the unbiasedness while both of the MZ and Nordhaus regressions are considered for testing the optimality of a forecast.
Table 1 reports the estimation results. Estimation results of typical
Holden-Peel regression, i.e., ˆµ in model (a1), show that the forecast is biased, and the
Ljung-Box Q-statistics indicates that regression’s residuals are significantly au-tocorrelated. This implies that the forecast is inefficient since the forecast error is predictable by its past values. Meanwhile, both the conventional MZ regres-sion (b1) and Nordhaus model (c1) also indicate that the forecast is biased and inefficiency since the forecaster under-reacts to new information and thus fails to incorporate all new information in the forecast. Moreover, the results of state-dependent Nordhaus’s model (c2) further show the time varying biases in different state of the economy.
3.3
Results of Proposed Model
The dynamics of time varying biases during different periods of the state of the economy now are analyzed using the proposed model (6).
Panel (A) in Table 2 presents the estimates, ˆα0, ˆα1, ˆβ0, ˆβ1, ˆγα0, ˆγα1, ˆγβ0 and ˆγβ1,
say, that are obtained using the nonlinear least squares (NLS) estimation method; Panel (B) presents the results of the testing of Hypothesis 2.1 and Hypothesis 2.2
with χ2 statistics by Wald joint tests and of Hypothesis 2.3 with t statistics by
Table 1: Conventional Unbiasedness and Efficiency Analyses
Panel (A) Holden-Peel Regression Model (a1): et= µ + ut
Model µˆ Q(1) Q(2) Q(3) Q(4)
(a1) −0.303∗ 0.000∗∗∗ 0.000∗∗∗ 0.000∗∗∗ 0.000∗∗∗
(−1.694)
Note: Number in parentheses is robust t statistics based on heteroscedasticity-consistent estimates for corresponding variances; Symbols ***, **, and * denote rejection at the significance level of 1%, 5%, and 10%, respectively; Q(r) denotes the Ljung-Box Q-Statistic with its corresponding probabil-ity value that is used to test the null hypothesis of no autocorrelation of ut for a specified order of
autocorrelation lags r.
Panel (B) Mincer-Zarnowitz Regression Model (b1): yt= λ + δyft|t−1+ ut
Model λˆ ˆδ Joint Test R¯2
(b1) −0.696 1.197∗∗∗ 0.000∗∗∗ 0.819
(−1.357) (14.447)
Note: Number in parentheses is robust t statistics based on heteroscedasticity-consistent estimates for corresponding variances; Symbols ***, **, and * denote rejection at the significance level of 1%, 5%, and 10%, respectively; The p-value of joint test for the null hypothesis Ho: λ = 0, δ = 1.
Panel (C) Nordhaus Regression Model (c1): et= α + β · FRt−1+ ut Model (c2): et= [α0+ β0· FRt−1] × (1 − ˆst−1) + [α1+ β1· FRt−1] × ˆst−1+ ut Model αˆ αˆ0 αˆ1 βˆ βˆ0 βˆ1 R¯2 (c1) −0.433∗∗∗ −0.642∗∗∗ 0.319 (−3.064) (−4.620) (c2) 0.050 −0.886∗∗∗ −0.499∗∗∗ −0.546∗∗ 0.361 (0.261) (−4.299) (−2.814) (−2.519)
Note: Number in parentheses is robust t statistics based on heteroscedasticity-consistent estimates for corresponding variances; Symbols ***, **, and * denote rejection at the significance level of 1%, 5%, and 10%, respectively.
Panel (A) of Table 2 shows the errors in the DGBAS forecast are attributable to biases of under-estimation and under-reaction to news in different state of the economy. In particular, with respect to the bias that is caused by
over/under-reaction to new information, insignificant ˆβ1 but significant ˆβ0 indicate that the
forecaster reacts to news more accurately during accelerating periods than during decelerating ones; under-reaction to the news is thus an important source of forecast error during decelerations. With respect to the bias that is generated
by over-/under-estimation, the estimate ˆα1 is significantly negative whereas ˆα0
differs insignificantly from zero. This indicates that the DGBAS under-predicts GDP growth only in accelerating GDP environments. It is also worth noting that
Table 2: Estimation and Hypothesis Testing Results
Panel (A) Estimation Results
Decelerating periods Accelerating periods (I) Bias due to over-estimation or under-estimation
ˆ α0 0.929 αˆ1 −1.061∗∗ (0.402) (−2.154) ˆ γ0 α 0.861 γˆα1 0.071 (0.394) (0.466)
(II) Bias due to over-reaction or under-reaction ˆ β0 −2.913∗∗ βˆ1 −0.267 (−2.356) (−0.862) ˆ γ0 β 0.631 ∗∗∗ ˆγ1 β −0.197 (3.165) (−0.866) ¯ R2 0.440
Note: Number in parentheses is robust t statistics based on heteroscedasticity-consistent estimates for corresponding variances; Symbols ***, **, and * de-note rejection at the significance level of 1%, 5%, and 10%, respectively.
Panel (B) Hypothesis Testings
Statistics p-value (I) Hypothesis 2.1 (Optimal forecast) χ2
(a) Ho: α0= α1= 0 4.800∗ 0.091
(b) Ho: β0= β1= 0 6.296∗∗ 0.043
(c) Ho: α0= α1= 0, β0= β1= 0 14.446∗∗∗ 0.006
(II) Hypothesis 2.2 (State-independent bias) χ2
(a) Ho: α0= α1, γα0 = γ1α 1.729 0.421
(b) Ho: β0= β1, γβ0= γ1β 7.484∗∗ 0.024
(c) Ho: α0= α1, γα0 = γα1, β0= β1, γβ0= γβ1 8.940∗ 0.063
(III) Hypothesis 2.3 (Persistent bias) t
(a) Ho: γα1= 0 0.466 0.641
(b) Ho: γ0β= 0 3.165
∗∗∗ 0.002
Note: Joint Wald test with statistics χ2is employed for testing Hypothesis 2.1
and 2.2, while individual test with t statistics is for Hypothesis 2.3; Symbols ***, **, and * denote rejection at the significance level of 1%, 5%, and 10%, respectively.
forecast errors can be eliminated by considering the proposed biases.
The testing results in part (I) and (II) of Panel (B) show the DGBAS forecast is not optimal and suffers from state-dependent biases, which also suggests that the original model (3) of Nordhaus (1987) may not be suitable to official forecasts in Taiwan. Since the biases significantly revealed in Panel (A), whether they are persistent is further tested in part (III) of Panel (B). The p-values for the
the data while γα1 = 0 is not. It implies that (i) the DGBAS persistently publishes an average under-estimate of GDP growth of 1.061% during accelerating periods; (ii) the DGBAS under-reacts to new information during decelerating periods and
this bias reveals, on average, only for 1.585 (= 1/0.631) quarters,8 the degree of
under-reaction is deminishing with duration.
3.3.1 Bias-corrected Forecasts
Since the forecast errors of DBGAS, which are attributable to possible over-/under-estimation or over-/under-reaction of biases across the state of the econ-omy, have been identified after estimating the model (6), the bias-corrected fore-casts can now be made via the formula (7),
yt|t−1bc = yt|t−1f − ˆet,
where yt|t−1f is the forecast made by the DGBAS at time t − 1, and ˆet is the
proposed estimate of et made using the model (6). As mentioned, this
bias-corrected forecast is particularly useful for the real-time analysis because a better forecast can be obtained immediately once a biased forecast is released.
Figure 2 plots the time-series patterns of the bias-corrected forecast ybc
t|t−1,
along with the realized Taiwan GDP growth and the DGBAS forecasts from
1986Q4 to 2012Q3. Roughly, bias-corrected forecast ybc
t|t−1 is much closer to the
realized data than the DGBAS forecast yt|t−1f , especially during decelerating
peri-ods after 2000. On average, an 89.497% of the variation of realized GDP growth
can be explained by yt|t−1bc whereas the DGBAS forecast explains only 79.857%;
cf. (8).
Beyond the above (in-sample) performance of the proposed bias-corrected forecast, the effectiveness of this corrected forecast in pseudo out-of-sample ex-periments is now determined. Two competing classes of models are compared below. The first class contains three MZ bias-adjusted models based on
modifi-8Recall that γ0
β in exponential survivor functionals measures the fixed probability that the
forecaster leaves the state of suffering from the under-reaction during decelerating periods, the expected duration of under-reaction equals 1/γ0
87 88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 -11 -6 -1 4 9 14 ACCELNO Realized Forecast Log-logistic
Figure 2: Time series plots of realized Taiwan GDP growth, DGBAS forecast and bias-corrected forecast yt|t−1bc . The shaded area represents decelerating periods where ˆst= 0 for all
t.
cation of the work of Ang et al. (2007). They are MZ1: yt = λ + δyt|t−1f + εt; MZ2: yt = λ10× (1 − ˆst−1) + λ11× ˆst−1+ δ1yt|t−1f + εt; MZ3: yt = h λ20+ δ20yft|t−1 i × (1 − ˆst−1) + h λ21+ δ21yt|t−1f i × ˆst−1+ εt,
where MZ1 is the conventional MZ regression, while MZ2 and MZ3 are two
generalizations that depend on the proxy for the economic state ˆst−1.
The second class is composed of the linear/nonlinear autoregressive time series models. They are the typical linear autoregressive time series model:
yAR(p): yt= c +
p
X
i=1
φiyt−i+ εt,
where 1 ≤ p ≤ 5, and the nonlinear exponential smooth transition autoregressive (ESTAR) model: yESTAR(q): yt= a0+ q X j=1 ϕ0jyt−j + a1+ q X j=1 ϕ1jyt−j ! G(γ, µ, yt−1) + εt, where G(γ, µ, yt−1) = [1 + exp{−γ(yt−1− µ)}] −1 , and 1 ≤ q ≤ 5.
Fifty-one pseudo out-of-sample periods from 2000Q1 to 2012Q3 (about a half of whole sample period) are considered. Recursive estimation scheme is utilized for all regressions on the samples, beginning in 1986Q4 but ending from 1999Q4 to 2012Q2. That is, the first estimation begins with data from 1986Q4 to 1999Q4, and a one-step-ahead forecast for 2000Q1 is made based on these estimates; the second estimation and the resulting one-step-ahead forecast for 2000Q2 is based on the data from 1986Q4 to 2000Q1, and so on and so forth. Hence, the final forecast for 2012Q3 is made based on the data ends in 2012Q2. Notably, the optimal lag lengths p for yAR(p) model and q for yESTAR(q) are also recursively determined by applying Baysian information critiron (BIC) in each sample period. The performance is measured in terms of mean square error (MSE) and mean absolute error (MAE), and Diebold and Mariano’s (1995) test, (the DM test herein) is implemented further to compare the performance of these six models to that of DGBAS. The null hypothesis of the DM test claims no difference between the forecast accuracy of the competing forecast with that of the DGBAS forecast, while the alternative is that the competing forecast is superior. Two
test statistics, one for the squared errors difference (DMS) and another for the
absolute errors difference (DMA) are computed.
Table 3 summarizes the results. First, DM tests do not support the hypoth-esis that the three MZ-type adjusted forecasts and the two autoregressive-type forecasts are significantly superior to the DGBAS forecast, even though some of their MAEs and MSEs are slightly lower than those of DGBAS. Second, the proposed bias-corrected forecast is better than those made using all other mod-els because it has the lowest MSE and MAE. The bias-corrected forecast has a 38.67% lower MSE than the DGBAS forecast and a 16.46% lower MAE as it eliminates possible biases of over-/under-estimation and over-/under-reaction in different state of the economy. Furthermore, both DM tests significantly reject the null hypothesis in favor of the alternative hypothesis that the bias-corrected forecast is superior to the official forecast of Taiwan.
3.4
Robustness Checks for State Proxy and Long Horizon Forecasts
We also conduct robustness checks by using two alternative state proxies where 6-quarter and 8-6-quarter MA in (5) are adopted. In essence, the resulting inferences are the same.
ad-Table 3: Out-of-sample Performance Comparisons
DGBAS Bias-corrected model MZ bias-adjusted models Autoregressive models
yft|t−1 ybct|t−1 MZ1 MZ2 MZ3 yAR(p) yESTAR(q) MSE 5.669 3.477 5.602 4.738 5.283 5.952 7.964 MAE 1.762 1.472 1.772 1.615 1.676 1.925 2.036 DMS −2.454∗∗∗ −0.087 −1.175 −0.396 0.252 1.108 (p-value) (0.007) (0.465) (0.120) (0.346) (0.600) (0.866) DMA −2.115∗∗∗ 0.073 −1.009 −0.508 0.731 0.939 (p-value) (0.017) (0.529) (0.156) (0.306) (0.768) (0.826) Note: The null and alternative hypotheses for DM tests are H0: “there is no difference in forecast accuracy of
the competing forecast and DGBAS forecast” and H1: “the competing forecast is superior to DGBAS forecast”,
DMSis the corresponding statistics calculated by square errors difference while DMAis based on absolute errors
difference. Symbols ***, **, and * denote rejection at the significance level of 1%, 5%, and 10%, respectively.
dition to the one-step-ahead forecast errors. The model for the h-step-ahead forecast is established as et,h = h α0exp −γ0 α· ˆd0t−h + β0exp −γ0 β· ˆd 0 t−h FRt−h i × (1 − ˆst−h) +hα1exp −γ1 α· ˆd1t−h + β1exp −γ1 β · ˆd1t−h FRt−h i × ˆst−h+ ut,h,
where et,h = yft|t−h− yt and FRt−h = yt|t−hf − yft|t−h−1. We analyzed the 2- and
3-step-ahead forecasts. In brief, the estimation and testing results show that the state-dependent biases in 2-step-ahead DGBAS forecast are weak while those of in 3-step-ahead DGBAS forecasts are statistically significant and persistent. Owing to space limitations, we do not report results in detailed.
4
Concluding Remarks
This paper is aimed at investigating whether the forecast is optimal given the available information when the forecast is made. Going beyond the papers that study forecast errors based on the model of Nordhaus (1987), we apply a time varying procedure to forecast revisions and further account for the possibility that the duration of the state may also affect the bias. Three testable hypothe-ses are presented to help researchers test the optimality of forecasts, with the ultimate goal of determining whether these biases depend on the underlying eco-nomic state and whether they are persistent with duration of the state. The corresponding bias-corrected forecasts can then be made based on these results. Briefly, this framework is novel and implementable using conventional estimation and hypothesis methods.
In the empirical part, we apply the proposed framework to investigating Tai-wan DGBAS forecasts for GDP growth rates. We find that the one-quarter-ahead forecast is not optimal; it indeed suffers from state-dependent biases—persistent under-estimation bias at the relatively good state and under-reaction bias which decays with duration at the relatively bad one. Eliminating these biases in DG-BAS forecast can reduce over 44.0% variation of forecast errors, and pseudo out-of-sample experiments further support the fact that the resulting bias-corrected forecast is markedly better than that made by Taiwan’s government and those made using other competing models.
References
Amir, E. and Y. Ganzach (1998). Overreaction and underreaction in analysts’ forecasts, Journal of Economic Behavior & Organization, 37(3), 333–347. Ang, A., G. Bekaert and M. Wei (2007). Do macro variables, asset markets,
or surveys forecast inflation better? Journal of Monetary Economics, 54, 1163–1212.
Artis M. and M. Marcellino (2001). Fiscal forecasting: the track record of the IMF, OECD and EC, Econometrics Journal, 4(1), 20–36.
Ashiya, M. (2003). Testing the rationality of Japanese GDP forecasts: the sign of forecast revision matters, Journal of Economic Behavior & Organization, 50(2), 263–269.
Ashiya, M. (2007). Forecast accuracy of the Japanese government: its year-ahead GDP forecast is too optimistic, Japan and the World Economy, 19(1), 68– 85.
Bry, G. and C. Boschan (1971). Cyclical Analysis of Time Series: Selected Pro-cedures and Computer Programs, National Bureau of Economic Research, New York.
Chang, C.-L., P. H. Franses, and M. McAleer (2011). How accurate are govern-ment forecasts of economic fundagovern-mentals? the case of Taiwan, International Journal of Forecasting, 27, 1066–1075.
Chauvet, M. and J. T. Guo (2003). Sunspots, animal spirits, and economic
fluctuations, Macroeconomic Dynamics, 7, 140–169.
Christiano, L. J. and T. J. Fitzgerald (2003). The band pass filter, International Economic Review, 44(2), 435–465.
Coibion, O. and Y. Gorodnichenko (2012). What can survey forecasts tell us about information rigidities? Journal of Political Economy, 120(1), 116– 159.
Diebold, F. and R. Mariano (1995). Comparing predictive accuracy, Journal of Business & Economic Statistics, 13(3), 253–63.
in Germany, International Journal of Forecasting, 17(2), 181–201.
Ehrbeck, T. and R. Waldmann (1996). Why are professional forecasters biased? Agency versus behavioral explanations, Quarterly Journal of Economics, 111(1), 21–40.
Fildes, R. and H. Stekler (2002). The state of macroeconomic forecasting, Journal of Macroeconomics, 24, 435–468.
Gavin, W. T. and R. Mandal (2003). Evaluating FOMC forecasts, International Journal of Forecasting, 19(4), 655–667.
Granger, C. (1996). Can we improve the perceived quality of economic forecasts? Journal of Applied Econometrics, 11, 455–73.
Grunberg, E., and F. Modigliani (1954). The predictability of social events, Journal of Political Economy, 465–478.
Harding, D. and A. Pagan (2001). Dissecting the cycle: a methodological inves-tigation, Journal of Monetary Economics, 49(2), 365–381.
Holden, K. and D. A. Peel (1990). On testing for unbiasedness and efficiency of forecasts, Manchester School, 58, 120–127.
Joutz, F. and H.O. Stekler (2000). An evaluation of the predictions of the Federal Reserve, International Journal of Forecasting, 16, 17–38.
Kiefer, N. M. (1988). Economic duration data and hazard functions, Journal of Economic Literature, 26(2), 646–679.
Loungani, P. (2001). How accurate are private sector forecasts? Cross-country evidence from Consensus forecasts of output growth, International Journal of Forecasting, 17(3), 419–432.
Messina, J. D., T. M. Sinclair, and H. Stekler (2015). What can we learn from revisions to the Greenbook forecasts? Journal of Macroeconomics, 45, 54– 62.
Mincer, J. and V. Zarnowitz (1969). The evaluation of economic forecasts, in J. Mincer (Ed.), Economic Forecasts and Expectation, National Bureau of Research, New York.
Nordhaus, W. D. (1987). Forecasting efficiency: concepts and applications, Re-view of Economics and Statistics, 69(3), 667–674.
Patton A. and A. Timmermann (2012). Forecast rationality tests based on multi-horizon bounds, Journal of Business & Economic Statistics, 30(1), 1–17. Sinclair, T., F. Joutz, and H. O. Stekler (2010). Can the Fed predict the state of
the economy? Economics Letters, 108(1), 28–32.
Sinclair, T., H. O. Stekler and W. Carnow (2015). Evaluating a vector of the Fed’s forecasts, International Journal of Forecasting, 31(1), 157–164. Swanson, N. R. and D. van Dijk (2006). Are statistical reporting agencies
get-ting it right? Data rationality and business cycle asymmetry, Journal of Business & Economic Statistics, 24, 24–42.
Zarnowitz, V. (1992). The record and improvability of economic forecasting, NBER Chapters, in: Business Cycles: Theory, History, Indicators, and Forecasting, 519–534, National Bureau of Economic Research, Inc.
