Do We Need Better Econometric Models?

I have made a number of arguments in the last post (here) suggesting that we need better econometric models. One of the arguments was that econometric forecasting models did a poor job of predicting the Subprime Mortgage Crisis (the Financial Crisis of 2007-2008).

Yesterday the Federal Reserve Open Market Committee (FOMC) released transcripts of closed-door policy meetings in 2007 (here), at the start of the Subprime Mortgage Crisis. In the quote below from the Washington Post (here) we see that the models used by the FED staff economists were not predicting the upcoming crisis. The reasons why these models failed is a major concern of this blog and should be a major topic of discussion for the economics profession.

In December 2007, the month that the recession is now known to have begun, Fed officials were working from economic projections that would prove wildly inaccurate. They forecast sluggish but sustained growth in 2008 followed by a bounceback in 2009. Staff economist Dave Stockton acknowledged that his was a more optimistic view:

“Our forecast could admittedly be read as still painting a pretty benign
picture: Despite all the financial turmoil, the economy avoids recession and, even with steeply higher prices for food and energy and a lower exchange value of the dollar, we achieve some modest edging-off of inflation.”

Dave Stockton, who was the Chief Economist and Director of Research and Statistics at the Federal Reserve Board from 2000-2011, was basing his forecasts on those being produced by the FED econometric models. The burning question is why those models performed so poorly during the financial crisis and whether anything has been done about it after the crisis.

Typically, econometric models predict from one period to the next, whether that is months, quarters of years. The models are basically a collection of structural equations, some of which are dynamic, that is, have a time component. For example, the simple model Q(t) = A Q(t-1) + B X(t-1) + E would look similar to the dynamic equations where Q is some variable, t represents a time point, A represents a coefficient matrix for the endogenous variables, B represents a coefficient matrix for the exogenous variables (X) and E represents error.

The problem here is that when the model gets to time t, the predictions for Q contain E. If E is non-random (systematic positive shocks from the liar loans in the financial sector, for example), Q keeps accumulating errors and creates a bubble. It's ability to predict Q(t) is compromised by adding up errors from the developing Subprime Mortgage Bubble and makes everything look pretty rosy, as is evident from Mr. Stockton's comments.

The way in which econometric models are used, in this case simulating from one period to the next, is one of their major problems. The other problem is the theory (or lack of it) underlying the models. Both of these arguments will take many more posts to develop.