Workshop DGECFIN, Bruxelles, 16 November 2010
Constructing a conditional GDP fan chart with an application to French business survey data
Matthieu CORNEC INSEE Business Surveys Unit
Draft, version 11/24/2010
Abstract
Among economic forecasters, it has become a more common practice to provide point projection with a density forecast. This realistic view acknowledges that nobody can predict future evolution of the economic outlook with absolute certainty. Interval confidence and density forecasts have thus become useful tools to describe in probability terms the uncertainty inherent to any point forecast (for a review see Tay and Wallis 2000). Since 1996, the Central Bank of England (CBE) has published a density forecast of inflation in its quarterly Inflation Report, so called “fan chart”. More recently, INSEE has also published a fan chart of its Gross Domestic Production (GDP) prediction in the Note de Conjoncture. Both methodologies estimate parameters of exponential families on the sample of past errors. They thus suffer from some drawbacks. First, INSEE fan chart is unconditional which means that whatever the economic outlook is, the magnitude of the displayed uncertainty is the same. On the contrary, it is common belief among practitioners that the forecasting exercise highly depends on the state of the economy, especially during crisis. A second limitation is that CBE fan chart is not reproducible as it introduces subjectivity. Eventually, another inadequacy is the parametric shape of the ditribution. In this paper, we tackle those issues to provide a reproducible conditional and semi-parametric fan chart. For this, following Taylor 1999, we combine quantile regression approach together with regularization techniques to display a density forecast conditional on the available information. In the same time, we build a Forecasting Risk Index associated to this fan chart to measure the intrinsic difficulty of the forecasting exercise. The proposed methodology is applied to the French economy. Using balances of different business surveys, the GDP fan chart captures efficiently the growth stall during the crisis on an real-time basis. Moreover, our Forecasting Risk Index increased substantially in this period of turbulence, showing signs of growing uncertainty.
Key Words: density forecast, quantile regression, business tendency surveys, fan chart.
JEL Classification: E32, E37, E66, C22 Constructing a conditional GDP fan chart with an application to French business survey data
1. Introduction
Usually, institutions display economic point forecasts. However, the forecast is not free of uncertainty: assuming that forecasts are not biased, they may be considered as the expected figure given the available information. Indeed, many shocks can affect the forecast in regard to oil prices, exchange rates, interest rates, and other variables. Another reason is that the behaviors of economic agents can only be estimated imprecisely over the past (when they do not change). The potential scenarios are therefore numerous: forecasters thus condense their forecast into a single, “baseline” scenario.
In the end, readers may lose sight of the uncertainty inherent in this type of exercise. Among economic forecasters, it has therefore become a more common practice to provide point projection with a density forecast. The longest running series of macroeconomic density forecasts dates back to 1968, when the ASA and the NBER initiated a survey of forecasters. For a detailed historical review, we refer to Tay and Wallis (2000). In particular, to dispel their risk, the Central Bank of England (cf. Britton et al 98) and INSEE display a “fan chart” to provide a concise illustration of the uncertainty affecting point forecasts. This realistic view recognizes that future evolution of the economic outlook cannot be predicted with absolute certainty. Confidence intervals and density forecasts have appeared as useful tools to describe in probability terms the uncertainty inherent to any point forecast (for a review, see Tay and Wallis 2000).
Both methodologies suffer from some drawbacks. First, the INSEE fan chart is unconditional which means that the magnitude of the displayed uncertainty is the same, whatever the economic outlook is. On the contrary, it is a common belief among practitioners that the forecasting exercise highly depends on the state of the economy, especially during crisis. Secondly, CBE’s fan chart is conditional but this conditionality comes from the subjective assessments by the members of the Monetary policy comittee, and is therefore not reproducible. Eventually, another limitation is the parametric shape of the distribution, which is usually assumed to be exponential, that is without fat tails. In this paper, we tackle these issues to provide a reproducible methodology to build a non parametric conditional density forecasts.
This paper is organized as follows: in section 1, we review the main methods to describe uncertainty used by both INSEE and CBE, namely confidence intervals and density. We introduce our main notations and we give a brief description of French business surveys in section 2. In section 3, we introduce our methodology to derive conditional density forecasts and to construct our Forecasting Risk Index. Eventually, the proposed methodology is applied to the French economy in section 4. Uncertainty description
A common way to describe uncertainty is to consider the future evolution of the economy (i.e. GDP growth rate) as the realization of a continuous random variable. The uncertainty is then fully characterized by the random variable density. A peaky density means a small uncertainty. On the contrary, a large uncertainty is translated into a loose distribution.
2 Constructing a conditional GDP fan chart with an application to French business survey data
Confidence intervals
Confidence intervals may be the first simple method to describe the point forecast uncertainty (for a review see Tay and Wallis 2000). The principle is the following : the economic forecaster provides an interval together with his point forecast. The future observed value is then supposed to lie within this interval with a specified probability. For example, if the forecaster provides 95%-confidence intervals, the observed value is supposed to lie around 95% of the time into these intervals. It is also common to provide confidence intervals at different probability levels, i.e. 20%, 50%, 99% confidence intervals.
In the case of confidence intervals, the length of the interval measures the level of uncertainty. For example, we usually expect uncertainty to grow as the forecasting horizon increases. This is displayed by an increasing length of confidence intervals for a specified level of probability (cf. figures 1 and 2).
However, if their simplicity makes confidence intervals very attractive, they do not completely describe uncertainty.
Density forecasts
Density forecasts have thus become more and more appealing since they fully describe uncertainty. Notice that they can also be used to easily derive confidence intervals based on the appropriate distribution quantiles. In the case of density, the level of uncertainty is measured by the inverse of the sharpness of the distribution. Many indicators have been proposed to characterize the level of uncertainty, among them variance and entropy (see. Lugosi).
Since 1996, the Central Bank of England has a published a density forecast of inflation in its quarterly Inflation Report, so called “fan chart” (cf. figure 1). More recently, INSEE has also published a fan chart of its GDP prediction in the Note de Conjoncture (cf. figure 2).
3 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 1 CBE inflation fan chart
Figure 2 INSEE GDP fan chart
4 Constructing a conditional GDP fan chart with an application to French business survey data
INSEE and CBE density estimation
Both methodologies assume that the error density belongs to exponential families. Estimations of the parameters are based on the sample of past forecast errors.
INSEE fan chart displays a normal distribution:
1 (y ) 2 f (y) exp( ). 2 2 2
Thus INSEE perceives the possible GDP outcomes symmetrically dispersed around the central most probable value. This is the famous bell-shaped curve (cf. figure 3 for the density of a standard normal).
Figure 3 density of a standard normal distribution
Bell curve 4 . 0 3 . 0 y t i s 2 . n 0 e D 1 . 0 0 . 0
-3 -2 -1 0 1 2 3
x
Central Bank of England perception is that in some circonstances the forecast error is more likely to be in one direction than the other. In statistical terms, their fan chart distribution is skewed. That is why they chose a particular form of statistical distribution called “two-piece” normal (cf. figure 4):
5 Constructing a conditional GDP fan chart with an application to French business survey data
1 1 x 2 2 . f (y) exp( 2 (x ) ( )(x ) ) 2 2 x
Eventually, CBE’s methodology allows the injection of subjectivity by deforming the density shape.
Figure 4: example of a skewed distribution by CBE
Even if both institutes display density forecasts, a fundamental philosophical difference remains between both methodologies. INSEE methodology attempts to capture an intrinsic uncertainty.
“assuming no change in the volatility of growth figures and the methodology used by INSEE forecasters during the period, the distribution of forecasting errors calculated from past exercises is a reliable indicator of the distribution of future errors” (cf. Note de conjoncture)
“we have chosen not to depart from the historic variance of forecasting errors, as the injection of a dose of subjectivity seems hard to justify ”( cf. Note de conjoncture)
6 Constructing a conditional GDP fan chart with an application to French business survey data
On the contrary, CBE’s fan chart somehow measures the CBE subjective uncertainty about: The aim of the fan chart has been to convey to the reader a more accurate representation of the Bank’s subjective assessment of medium-term inflationary pressures. It is therefore a forward-looking view of the risks to the forecast, not an extrapolation of past uncertainty. For more insights about this debate, we refer to B. de Finetti (1975) and references therein. Drawbacks
Conditional versus unconditional
The first practical and important consequence is that the INSEE fan chart is unconditional. Thus, whatever the economic outlook is, the magnitude of the displayed uncertainty is the same. It means that on a long-term basis it is supposed to be correct on average. In other words, during a recession, the usual unconditional central confident interval can be significantly wrong. Let’s take a simple example by comparing forecasting with flying a plane: when crossing a turbulence area, flying gets more difficult. In the same way, during a crisis period, the intrinsic uncertainty of the forecasting exercise increases. Instead of unconditional forecasting, conditional forecasting which can change from one period to another should be favoured to describe uncertainty. We illustrate the interest of conditional forecasting versus unconditional forecasting by a simple toy model.
Toy Model
Let Zt be a random variable that describes the state of the economy at time t and that can only take two values: A for acceleration with probability p , D for deceleration 1 p .
Let Yt be a random variable describing the GDP growth rate.
To make it simple, we suppose the conditional distributions of Yt to be such that: 2 E (y Z ) ,Var(y Z ) Z with (the growth rate is smaller during recession) Yt t Zt t t D A 2 2 A D (but the volatility is higher).
Then the description of uncertainty by an unconditional forecaster is E(Var(yt I t )) which is 2 2 equal to p D 1( p) A .
On the contrary, for a conditional forecaster, uncertainty as measured by Var(yt I t ) would be 2 2 equal to D during recession and to A during acceleration. We can deduce that on a long-term average the unconditional description is correct. However, at each date t , knowing the information Zt , it is either too small or too big.
7 Constructing a conditional GDP fan chart with an application to French business survey data
2 2 2 2 Indeed, since A p D 1( p) A D , the unconditional description of uncertainty is either too loose (during acceleration period) or too sharp (during crisis). The unconditional forecast error neglects the information embedded in it.
On the contrary, since CBE’s methodology allows the injection of subjectivity from one period to another, their fan chart is thus conditional. However, their method is not reproducible as it is a measure of CBE’s subjectivity.
Exponential distribution versus nonparametric
To check a density forecast, we can use Talagrand’s diagram. The principle is the following: ˆ Ft (Yt ) is supposed to be a sequence of independent identically distributed uniform variables on ˆ ˆ 1,0 with Ft the forecasted cumulative distribution function. The histogram of Ft (Yt ) is called Talagrand’s diagram and is supposed to be a straight line if our forecast is correct (cf. Dowd, K., 2004). Figure 5 displays Talagrand’s diagram for the INSEE fan chart. Instead of a straight line, it exhibits a concave profile indicating that the probabilities of right extreme risks are overestimated.
8 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 5: Talagrand’s diagram of the INSEE fan chart
INSEE Talagrand diagram 0 2 5 1 y c n e u q e r 0 F 1 5 0
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
The aim of this paper is to suggest a reproducible methodology to build conditional density forecasts. In the same time, we will release the constraint of the parametric distribution shape.
Notations and business surveys description
9 Constructing a conditional GDP fan chart with an application to French business survey data
Recall that Zt describes the state of the economy in our toy model. The role of Z t will be played by business surveys. Indeed, business surveys are a useful source of information when forecasting, for they present three types of advantages: (1) they provide reliable information coming directly from the economic decision makers, (2) they are rapidly available (about a month after the questionnaires are sent), on a monthly, bimonthly or quarterly basis, and (3) they are subject to small revisions (each publication presents a generally negligible revision, only on the preceding point). The disseminated statistics compiled from these surveys are usually balances of opinions. Since there are a large number of available survey variables (cf. table 1), composite indexes (CI) have been developed over the years to provide suitable summaries by extracting the common trend, and suppressing the undesirable "noise" of numerous data. In particular, the French composite indicator (cf. figure 6) gives an assessment of the global climate of the whole French economy.
Figure 6: French composite indicator and year-on-year GDP growth rate
French Composite Indicator and GDP growth rate
130 6,5
120 5,0
110 3,5
100 2,0
90 0,5
80 -1,0
70 -2,5
60 -4,0 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010
CI GDP growth rate
10 Constructing a conditional GDP fan chart with an application to French business survey data
Table 1: balances of opinions used in the French CI (table from Bardaji 09)
1 Denote the French CI by (it )t . Recall that our goal is to forecast the first release of the quarterly
French GDP growth rate, denoted by yt . The first release will be published only 45 days after the end of the current quarter. Usually, economists also forecast the next quarters. For the sake of simplicity, we will restrict ourselves to the forecast of the current quarter. Finally, we define yt : (y1 ,, yt ) . Our quarterly historical data of French GDP first release starts from 1988 Q1.
1 To be more precise, we define it as the mean of the three last known monthly releases when forecasting takes place.
11 Constructing a conditional GDP fan chart with an application to French business survey data
Methodology
Adapting the sequential framework in Biau and Patra (2009): at each quarter t 1T , we observe business surveys data zt but the first release of the q-o-q GDP growth rate yt is unknown. To model uncertainty, we consider (zt )t and (yt )t as the outcome of random variables (Zt )t and (Yt )t such that the process (Zt ,Yt )tZ is jointly continuous stationary ergodic.
In a sequential version of the density prediction problem, the forecaster is asked to guess the next conditional density of Yt of a sequence of random variables Y1 ,...,Yt1 with knowledge of the past observations yt1 y1,, yt1 and zt z1,, zt . In other words, the observations y1, y2 and z1, z2 are revealed one at a time, beginning with (y0 , z1 (), y1, z2 ),
Quantile regression techniques provide a suitable semi-parametric tool to achieve a proper density forecast. We introduce here the main outlines of the quantile regression methodology. Brief introduction to quantile regression
(Koenker and Bassett 78) developed a theory for the estimation of the quantiles of a variable Y based on past observations.
The starting point is to notice that many probability quantities can be characterized by a minimization problem. For example, the expectation of a L2 random variable satisfies
2 E(Y ) arg min mR E(Y m) .
In the same way, the median corresponds to
med(Y ) arg min mR E Y m .
The natural question is then: can the quantile function QY ( :) inft R : FY (t) be described by a minimization problem? (Koenker and Bassett 78) generalizes the observation that 1 minimizing the L1 loss yields to the median. To that end, they transform the L loss into a suitable loss function called pinball loss function or also check loss (cf. figure 7).
(y :) y 1( )y .with y : max( ,0 y), y : max( ,0 y)
12 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 7: pinball loss
Then they obtain (for a proof see Biau and Patra (2009)):
QY ( ) arg min mR E (Y m)
These ideas extend readily to conditional quantiles QY ( z :) inft R : FY ( zt ) .
We can then verify that
QY ( z)argmin m(.) E (Y m(z)) X z)
(Koenker and Bassett 78) considered m(z) of a linear form z ' and the coefficients are t ' estimated by minimizing (yi z i ) on past sample observations. Their estimated quantile i1 ˆ ' ˆ curve is then defined by QY ( z :) z i .
To justify quantile regression, assume the data to be generated by the following model with heteroscedasticity:
Yt t (Z t ) t (Z t ) t
with t (Z t :) E(Yt Z t ) and t (Z t :) V (Yt Z t ) , and t an error term independent of Z t .
13 Constructing a conditional GDP fan chart with an application to French business survey data
Based on the aforementioned model, the conditional quantile of Yt given Z t has the form
Q ( Z ) (Z ) (Z )Q () . Y t t t t t t
(Koenker and Bassett 82) showed that if t (Z t ) and t (Z t ) are linear functions of Zt then quantile regression estimates are asymptotically consistent. The appeal of quantile regression is that past observations of the quantiles are not required.
Reference Model
The first step is to find a proper set of explicative variables to both: -reduce the uncertainty since
Var(Yt ) E(Var(Yt Z t )) Var(E(Yt Z t )) E(Var(Yt Z t ))
In other words, the square error of an uninformed forecaster is equal to the error of an informed forecaster plus a residual term (knowledge term).
-and explain correctly the remaining uncertainty (i.e. Var( t /) Var( t (Z t )) small).
As mentioned above, there is a huge number of balances of opinions (hundreds). Thus, we face the curse of dimension. The French composite indicator (FCI) is a way to reduce the dimension by assessing the global climate of the whole French economy.
In a simplified framework, FCI would be roughly equal to GDP growth rate Yt I t .
However, in these surveys, entrepreneurs are asked to give a qualitative appreciation on the occurred or expected changes of some variables of interest (output, order book, foreign order book, inventories, . . .) through the three following categories: increase, no change, decrease. There is thus a difference between qualitative answers from microeconomic surveys and quantitative macroeconomic measures. To fill the gap, we suggest a model of the following form:
Yt Z t (Z t )ut
' with Z t : ,1( Yt1 , I t ,I t I t ) , : ( 0 ,1 , 2 , 3 ) and ut a white noise independent of the available information.
14 Constructing a conditional GDP fan chart with an application to French business survey data
Point forecast
Once the model set, the forecaster needs first a point forecast. It is usual (though not optimal for prediction as we do not need the estimator to be unbiased) to estimate by Ordinary Least Squares (OLS) and we get:
ˆ ' Yt ˆ Z t
' 1 ' with ˆ : (Z t Z t ) Z t Yt .
The unconditional variance E(Var(Yt Zt )) of the error term can then be estimated by
ˆ 1 ˆ 2 E(Var(Yt I t )) : Yt Yt . T t
Conditional confidence intervals
A simple way to derive conditional confidence intervals for Yt is to use quantile regressions. In order to construct a confidence interval with probability at least 90%, it is sufficient to estimate the quantile curve at levels 5% and 95% by quantile regression on the past available observation (y , z ),...,(y , z ) : Qˆ .0( 05 z) and Qˆ .0( 95 z) . Our confidence interval at quarter t can be written 0 1 t1 t Yt Yt as
90% CI t : [Qˆ .0( 05 z ),Qˆ .0( 95 z )] Yt t Yt t
Conditional Density forecasts
Recall that the quantile curve Q ( z ) completely describes the distribution. Thus, it would Yt t be sufficient to give the estimated quantile curve Qˆ ( z ) . Instead, it is a common practice Yt t ˆ among forecasters to display a density forecast y ft (y zt ) .
We give here a simple heuristic to derive a density forecast from the estimated quantile curve Qˆ ( z ) . Yt t
15 Constructing a conditional GDP fan chart with an application to French business survey data
Recall that Q (U z )with U a uniform distribution is distributed as Y z . Moreover, Yt t t t
ˆ 1 y Yt Y Y z f (y z :) K( ) (with K a kernel and t ) is a consistent estimator of f (y z) Yt t Y Th t h under suitable conditions.
Box 1
Heuristic path to estimate fY (y z)
Set N a large number (a practical choice is 100). i Compute at each u : 1; i N , y : Qˆ(u z) i N i i 1 y y Eventually, compute ˆ i with a Epanechnikov kernel and the fY (y z :) K( ) K h Th t h bandwidth
Forecasting Risk Index: FRI
In this section, we aim at building a Forecasting Risk Index for each quarter t, associated to this fan chart to measure the intrinsic difficulty of the forecasting exercise. At quarter t, the difficulty of forecasting is linked to the sharpness of the distribution: the sharper the distribution, the easier forecasting is. We must thus set a quantitative indicator to measure how much the distribution is spread. In the literature, classical measures are variance or entropy. As far as the root mean square error is concerned, the L2 norm (root of variance) suits our needs. The definition of conditional variance is given by:
Var(Y z)
We estimate Var(Y z) by Vˆar(Y z :) Var (Y ) fˆ z
We are now in position to define our Forecasting Risk Index for each quarter t as
FRI : Vˆar(Y z ) t t t
16 Constructing a conditional GDP fan chart with an application to French business survey data
When the point forecast is based on our reference model, the expected forecasting error is approximately equal to the value of the Forecasting Risk Index.
Out-of-sample Validation
Recall the forecaster is asked to guess the next conditional density of Yt of a sequence of random variables Y1 ,...,Yt1 with knowledge of the past observations yt 1 y1 ,, yt1 and zt z1,, zt .
ˆ ' Thus, all previous quantities must be computed on a real time basis: Yt ˆ t Z t with ' 1 ' 90% ˆ : (Z Z ) Z Y , CI t , fˆ (y z ) t t t t t Yt t
Interval forecast
90% If the confidence intervals for Yt are correct, yt should lie in CI t around 90% of the time.
Density forecast
To check our assumptions on real time basis, we can notice that Yt is supposed to follow
Yt fˆ (y z ) . Thus the cumulative function Fˆ (Y x :) fˆ (y z )dy should be independent random Yt t t t t t t ˆ variables distributed as uniform variable on ]1,0[ . The histogram of (Ft (Yt zt )) t can be plotted and compared with a uniform density: this is the classical Talagrand’s diagram. Other more sophisticated tests might be considered (see Berkowitz 2003, Clement 2004, Wallis 2003 and references therein).
17 Constructing a conditional GDP fan chart with an application to French business survey data
Numerical results
Point forecasts
Zt : ,1( Yt1, It ,It It ) () Parameters estimation by OLS (from 1988 Q1 to 2010Q1) leads to the following chart:
Table 1 Parameters estimation
Estimate Std. Error T value p-value Intercept 0.61 0.05 11.84 ***
-0.41 0.09 -4.42 *** Yt1
0.09 0.01 6.89 *** It
0.10 0.01 8.28 *** It It
Figure 8 displays the out of sample forecasts together with the first GDP release. The residual standard error is 0.31.
Var(Y ) E(Var(Y Z )) Var(E(Y Z )) Recall that we have t t t t t . In other words, the square error of an uninformed forecaster is equal to the error of an informed forecaster plus a residual term Var(Y ) .0 24 E(Var(Y Z )) .0 12 (knowledge term). We estimate t and t t . Thus, our reference model gives us a 50% gain of accuracy for the L2 loss.
18 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 8: out of sample point forecasts
Out-of-sample forecasts 0 . 1 5 . 0 ) % (
e t a r
0 . h t 0 w o r g
P D G 5 . 0 - 0 . 1 -
cpib_pr OoS Forecasts
1995 1997 1999 2001 2003 2005 2007 2009
19 Constructing a conditional GDP fan chart with an application to French business survey data
Conditional confidence intervals
In table 2, we obtain the estimates (from 1988 Q1 to 2010Q1) for the quantile regression coefficients. Interestingly, the coefficient of the acceleration term is almost zero for the 95% quantile whereas it is significant for the 5% quantile. In other words, troughs are much worse than peaks are great.
Table 2 Parameters estimation
theta = 0.05 Theta = 0.95 Intercept 0.17 1.23
-0.35 -0.46 Yt1
0.09 0.06 It
0.10 -0.00 It It
Figure 9 displays GDP first releases together with 90% confidence intervals of its last forecast. Notice that the length of the interval depends on the forecasting date. The percentage of GDP first releases outside our 90%-confidence intervals is 13% estimated on our historical data.
20 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 9: Out of sample 90% confidence intervals
Out-of-sample forecasts 5 . 1 0 . 1 5 . 0 ) % (
e t a r
0 h . t 0 w o r g
P D 5 . G 0 - 0 . 1 -
cpib_pr
5 OoS Quant forecasts 5 % . 1
- OoS Quant forecasts 95 %
1995 1997 1999 2001 2003 2005 2007 2009
21 Constructing a conditional GDP fan chart with an application to French business survey data
Conditional Fan chart
Figures 10 show density forecasts (methodology from box1) together with GDP first release (vertical line) at different quarters (from 2008 Q2 to 2010Q2).
Figure 10: density forecasts and GDP first release (vertical line)
As mentioned above, we draw Talagrand’s diagram for our new fan chart (cf. figure 11). In comparison with INSEE diagram (cf. figure 11), we can see that our new method leads to a better estimate of probability tails. The bumpy fan charts come from the semi-parametric methodology as no unimodal distribution has been imposed. This raised the philosophical question if the error distribution should be unimodal, parametric or nonparametric. If necessary, it is always possible to display a unimodal density but with a conditional variance. For this, it is sufficient to plug the forecasting risk index, as the conditional standard error of a gaussian density forecast.
22 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 11: Talagrand’s diagram for the new density forecast
Talagrand diagram for the new fan chart 5 1 y c n 0 e 1 u q e r F 5 0
0.0 0.2 0.4 0.6 0.8 1.0
Forecasting Risk Index
Figure 12 displays the Forecasting Risk Index. On an out-of-sample basis, the FRI exhibits clear signs of turbulence during the previous crisis. This gives an early signal of growing uncertainty. For example, when FRI is equal to 0.6 in 2008Q4, it can be interpreted as an expected error of 0.6 for a point forecast based on our reference model. It is worth recalling that our reference model is only based on surveys data. Thus, any uncertainty that may be not reflected by business surveys such as uncertainty on oil prices or exchange rates will not be displayed by our forecasting risk index. However, Talagrand’s diagram shows that our fan charts enjoy nice empirical properties. Moreover, in a bayesian manner, an economic forecaster could always start from the uncertainty as measured by the surveys data and add his own subjective assessment of uncertainty.
23 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 12: Forecasting Risk Index
Forecasting Risk Index 7 . 0 6 . 0 5 . ) 0 e c n a i r a v ( t 4 r . q 0 s 3 . 0 2 . 0
1995 2000 2005 2010
Time
Focus on the crisis period
Our new GDP density forecast captures efficiently the growth stall during the crisis on a real time basis. Indeed, the first release of 2008 Q4 (-1.3% in volume) lies inside the confidence interval (cf. figure 13). On the contrary, it was almost considered as an outlier by the INSEE fan chart (cf. figure 15). It is also interesting to compare results during the rebound of 2009 Q2. The first release (+0.3%) was in our confidence interval but it was far outside the INSEE fan chart (cf. figure 16). It could be surprising that our confidence interval captures efficiently the growth stall during the crisis on an real- time basis. Indeed, before 2008 Q4, the minimum q-o-q GDP growth rate of our historical data was - 0.6%, far above the -1.3% of 2008 Q4. This feature is made possible both by the linear form of every quantile regression and by new extreme business surveys values during the last crisis.
24 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 13: Out-of sample confidence intervals during the last crisis
Out-of-sample forecasts 5 . 1 0 . 1 5 . 0 ) % ( e t a r
0 h . t 0 w o r g
P D 5 . G 0 - 0 . 1 -
cpib_pr
5 OoS Quant forecasts 5 % . 1
- OoS Quant forecasts 95 %
2006 2007 2008 2009 2010
25 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 14: density forecasts during crisis
Chart on 2008 Q2 Chart on 2009 Q1 5 . 1
GDP First release 2 GDP First release . 1 0 . y y 1 t t 8 i i . s s 0 n n e e 5 . D D 4 . 0 0 0 0 . . 0 0
0.0 0.5 1.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5
N = 99 Bandw idth = 0.1022 N = 99 Bandw idth = 0.1098
Chart on 2008 Q3 Chart on 2009 Q2 2
GDP First release . GDP First release 1 8 . 0 y y t t 8 i i . s s 0 n n e e 4 . D D 4 0 . 0 0 0 . . 0 0
-0.5 0.0 0.5 1.0 1.5 -0.5 0.0 0.5 1.0 1.5 2.0
N = 99 Bandw idth = 0.1682 N = 99 Bandw idth = 0.1531
Chart on 2008 Q4 Chart on 2009 Q3 5 . GDP First release 1 GDP First release 8 . 0 0 . y y t t 1 i i s s n n 4 e e . 5 0 . D D 0 0 0 . . 0 0
-2 -1 0 1 -1.0 -0.5 0.0 0.5 1.0
N = 99 Bandw idth = 0.1729 N = 99 Bandw idth = 0.1088
26 Constructing a conditional GDP fan chart with an application to French business survey data
Figure 15: INSEE fan chart on 2008 Q3
GDP first release 2008Q4
Figure 16: INSEE fan chart on 2009 Q1
GDP first release 2009Q2
27 Constructing a conditional GDP fan chart with an application to French business survey data
Conclusion
In this paper, we developed a reproducible methodology to build conditional density forecasts based on business surveys data. Even though results are not fully comparable since the forecasting device is not exactly the same, this methodology seems able to better catch the uncertainty pattern associated with forecasts of French GDP. In particular, we derive a Forecasting Risk Index from the conditional density forecasts, which aims at quantifying the intrinsic uncertainty of a point forecast. Interestingly, our index methodology leads to uncertainty that increases on recessions, a characteristic that Insee forecasts have witenessed in the recent crisis.
References
Bardaji, J., Clavel L. and Tallet, F. (2010). Constructing a Markov-Switching Turning Point Index Using Mixed Frequencies with an Application to French Business Survey Data, Journal of Business Cycle Measurement and Analysis, forthcoming.
Berkowitz, J. (2001), “Testing Density Forecasts, With Applications to Risk Management”, Journal of Business and Economic Statistics, 19: 465-474.
Biau, G. and Patra, B. (2009). Sequential quantile prediction of time series.
Britton, Erik, Paul Fisher, and John Whitley (1998), .The Inflation Report Projections: Understanding the Fan Chart, Bank of England Quarterly Bulletin, 38(1), 30-37
B. de Finetti. Theory of probability. Vol. 1-2. John Wiley & Sons Ltd., Chichester, 1990. Reprint of the 1975 translation.
Clements, M. P. (2004), “Evaluating the Bank of England Density Forecasts of Inflation», Economic Journal, 114, 844–866.
Diebold, F.X., Tay, A.S. and Wallis, K.F. (1997). Evaluating density forecasts of inflation: the Survey of Professional Forecasters. Discussion Paper No.48, ESRC Macroeconomic Modelling Bureau, University of Warwick and Working Paper No.6228, National Bureau of Economic Research, Cambridge, Mass.
Dowd, K., (2004). The inflation fan charts: An evaluation. Greek Economic Review 23, 99–111.
INSEE Note de Conjoncture for June 2008, pages 15 to 18
Koenker, Roger W. and Gilbert W. Bassett, (1978). Regression quantiles. Econometrica 46, 33-50
Koenker RW and Bassett GW (1982). Robust tests for heteroscedasticity based on regression quantiles. Econometrica 49, 32-51
G. Lugosi, Concentration-of-measure Inequalities, Lecture Notes.
28 Constructing a conditional GDP fan chart with an application to French business survey data
Taylor, J.W. (1999b). A quantile regression approach to estimating the distribution of multiperiod returns, Journal of Derivatives, 7 Fall, 64-78.
Tay, A.S. and Wallis, K.F. (2000). Density forecasting: a survey.Journal of Forecasting, 19, 235-254.
Wallis, K. F. (2003), “Chi-squared Tests of Interval and Density Forecasts, and the Bank of England’s Fan Charts”, International Journal of Forecasting, 19: 165-175.
29