Economic forecasting is an art, not a science. Economic outcomes are often influenced by unanticipated events, and data may be inadequate, particularly in developing countries.
THE DISAPPOINTING growth performances of some industrial and developing countries in 1995–96, and the successive revisions of the IMF staff’s growth estimates for several regions have highlighted the margins of uncertainty surrounding economic projections. Revision of economic forecasts is not an uncommon occurrence, however, as those who have tried to predict the paths of business cycles—particularly the turning points—know all too well. A recent study by Professor Michael J. Artis of the European University Institute in Florence, Italy (see references) examines the IMF’s economic forecasting record from 1971 to 1994.
In principle, the projections in the World Economic Outlook (WEO)—like most official forecasts—are based on the assumption that economic policies will not change during the forecast period. Maintaining such an assumption, however, is difficult because much of the market information available at any point in time (including indicators such as interest rates, exchange rates, and business expectations) reflects, among other things, the anticipation that policy variables may change in the future. For this reason, Artis’s study followed the general practice of treating “unchanged policy” projections as “unconditional.”
On this basis, the WEO’s year-ahead forecasts of growth for the industrial countries as a group have been either too high or too low by about 1 percentage point, on average—a significant degree of error when measured against the actual average absolute growth rate of 2.75 percent. Inflation has been over- or underpredicted by about 0.75 percent a year, on average, a relatively smaller margin of error when measured against an actual average inflation rate of 5.75 percent. The WEO’s most accurate forecasts of growth rates appear to have been those for France and the United States, of the major industrial countries (Chart 1); for inflation, however, the WEO has had most success with its forecasts for Germany.
Paula R. De Masi, a US national, is an Economist in the World Economic Studies Division of the IMF’s Research Department.
Chart 1Real GDP growth in industrial countries World Economic Outlook forecasts compared with actual 1
Source: IMF, World Economic Outlook, October 1996, World Economic and Financial Surveys (Washington).
1World Economic Outlook forecasts are for the year ahead; actual represents the first settled estimates available.
Average forecast errors, however, represent only one of many criteria by which to judge the WEO’s forecasting record. Errors generated by a good forecast procedure should be unbiased and serially uncorrelated (i.e., errors should not be related systematically over time), and have no other property indicating that there is information in the data that could be used to improve the forecasts—in other words, the forecasts should be efficient. Judged by these criteria, the WEO forecasts of output growth for the industrial countries are broadly satisfactory. There is little evidence of bias or serial correlation and, on the whole, the forecasts are efficient. When the data for the industrial countries are pooled, some evidence of bias emerges, but this is primarily due to errors generated in the first half of the sample period. For inflation, however, the evidence is mixed. Although the forecasts are generally unbiased and efficient, they appear to suffer from serial correlation, suggesting that it takes time for forecasters to recognize that the inflation environment has changed.
Forecasting output and inflation for the industrial countries has not become any easier over time. In his analysis of two sub-samples of the forecast errors (pre- and post-1983), Artis shows that there is not a great deal of difference in the accuracy of forecasts between the two periods relative to forecasts based on “naive” methods, such as the assumption of a random walk (the direction of movement from year to year is determined by chance). Even though the first subsample includes the unusual economic disruptions associated with both of the major oil shocks, the environment of the second subsample did not prove any easier for forecasting.
Artis’s study confirms that the greatest area of weakness in forecasts for the industrial countries is predicting turning points in the business cycle. During the most recent cycle, forecasters had considerable difficulty anticipating the strength of the upswing and the duration of the subsequent slowdown. For inflation, although a number of statistical tests reveal that the accuracy of the forecasts did not improve, there was a large decline in the average absolute value of errors, reflecting a decline in the actual average rate of inflation.
The WEO forecasts for the major industrial countries are not the only forecasts to contain large errors with respect to the turning points of the business cycle. In a study carried out in 1988, Artis compared WEO forecasts with forecasts by the Organization for Economic Cooperation and Development (OECD) and by official forecasters from a variety of countries. The major forecasting errors were present in all forecasts. In Artis’s 1996 study, a comparison of WEO forecasts with the private sector Consensus Forecasts reveals that, over the last business cycle (1990–94), the forecasting errors were generally the same. A scatter plot of the Consensus Forecasts and WEO forecast errors of output growth for the major industrial countries is shown in Chart 2. Most of the observations fall on, or close to, the 45-degree line, suggesting that it the two forecast error records are similar. Moreover, many of the errors fall in the upper right quadrant of the chart, indicating that both forecasts tended to overestimate growth. The WEO forecasts, however, seem to have been slightly more optimistic than the Consensus Forecasts. A similar scatter plot for the inflation forecasts (Chart 2) shows little overall difference between the two sets of forecast errors.
Chart 2Major industrial countries: comparison of World Economic Outlook and Consensus Forecasts errors 1
Sources: IMF, World Economic Outlook, October 1996, World Economic and Financial Surveys (Washington); and Consensus Economics Inc., Consensus Forecasts.
1 Forecast errors are for the period 1990–94 and are defined as year-ahead forecast values minus actual realized values. Each observation shows the Consensus Forecasts and World Economic Outlook forecast errors for one of the seven major industrial countries for forecasts constructed at approximately the same time.
Forecasting movements in economic activity is even more difficult for the developing countries. In many countries, the data on which the forecasts are based are not timely and are of poor quality. In addition, many developing economies experience relatively greater volatility than the industrial economies, especially in countries with acute domestic or external imbalances or in those that are subject to fluctuating commodity prices or major shifts in investor sentiment. Even though an analysis of the aggregate performance of the developing countries should reduce the influence of these factors, the forecast performance is considerably less satisfactory than for the industrial countries.
The average forecast errors for 1977–94 for output growth and inflation differ considerably across regions of the developing world but are relatively large in comparison with their average absolute actual values. They are particularly large in the Middle East region, where uncertainty in oil markets and events such as the Persian Gulf war have complicated forecasting efforts, and in Latin America and the Caribbean, where output and inflation have tended to be more volatile than in other regions and developments have been dominated by a few large countries. It is important to note, however, that a significant source of forecast errors for the developing countries is the fact that the projections published in the WEO are consistent with those underlying IMF-supported policy programs, which are naturally based on the assumption that these programs will be implemented and successful.
Additional analysis of these forecast errors reveals that for output growth, there is evidence of positive bias for both Africa and Latin America and the Caribbean. In contrast, for inflation, there is evidence of a negative bias in most developing regions, and of serial correlation in Africa and Latin America and the Caribbean. Overall, these results, when compared with those for the industrial countries, suggest that it is much more difficult to forecast both output and inflation for the developing countries, and that historically there has been some bias in these forecasts.
The WEO forecasting record for both the industrial and developing countries suffers from a variety of weaknesses, and evidence suggests that forecasting has not become any easier over the sample period. Although certain factors seemed to promise increased accuracy—the cumulative experience of forecasting; greater timeliness of forecasts, thanks to advances in data pro-cessing; and growing competition in the field of economic forecasting—the structure of the world economy has been changing rapidly. As a result, new and sometimes unpredictable sources of error are being introduced, further complicating the fore-caster’s job.
Artis concludes that recent attempts to predict cyclical turning points have been especially disappointing, although, to be fair, the record needs to be assessed against the difficulty of the task. The oil price shocks in 1973 and 1979 and their impacts on the world economy were the driving force behind business cycle developments in the 1970s and early 1980s, and forecasters may be forgiven for not having foreseen oil price increases. The period since the mid-1980s has been different, however, with supply shocks playing less of a role. Notwithstanding “exogenous” events, such as the drop in oil prices in the mid-1980s, the Persian Gulf war, the unification of Germany, and the collapse of central planning, the prolonged global upswing of the mid-to-late 1980s and the subsequent slowdown appear to have been “endogenous,” and forecasting errors are harder to justify. If it is true that the recent business cycle largely followed the natural momentum of a more integrated and less regulated world economy, forecasters will do well to learn from it. It is too early to say how successful this learning process will be.
MichaelJ. Artis1988“How Accurate is the World Economic Outlook? A Post-Mortem on Short-Term Forecasting at the International Monetary Fund,”Staff Studies for the World Economic Outlook (Washington: International Monetary Fund) pp. 1–49.
MichaelJ. Artis1996“How Accurate are the IMF’s Short-Term Forecasts? Another Examination of the World Economic Outlook,”IMF Working Paper No. 96/89 (Washington: International Monetary Fund).
Paula R. De Masi