Chapter

Chapter 2 Content and Quality

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
September 2006
Share
  • ShareShare
Show Summary Details

We now turn to our evaluation of the content and quality of the IMF’s multilateral surveillance outputs. We begin by discussing the selection of issues analyzed as part of multilateral surveillance, showing how they reflect the IMF’s comparative advantage. We then consider the relevance and timeliness of the issues selected for analysis by using as a benchmark the importance attached by national policymakers in the G-7 and G-20 process. We likewise assess the relevance and timeliness of the risks identified in multilateral surveillance by comparing them to the risks identified by other international and national bodies. We look at the integration of multilateral and bilateral surveillance by analyzing the IMF’s policy advice and macroeconomic forecasts. And we assess the integration of macroeconomic and capital market analysis by identifying how multilateral surveillance combines these alternative approaches to analyze relevant issues.

Comparative Advantage and Topic Selection

The IMF’s comparative advantage

A broadly defined multilateral surveillance output can conceivably include the analysis of different types of issues, with each addressed to a particular audience or intended to satisfy a recognized need. These issues might include:

  • (1) economic linkages, policy spillovers, and global risks;
  • (2) global economic and market developments;
  • (3) developments in systemically more important economies;
  • (4) prospects in systemically more important economies;
  • (5) developments in systemically less important economies;
  • (6) prospects in systemically less important economies;
  • (7) emerging global policy issues; and
  • (8) cross-country comparisons of policy experiences.

Some of these issues are uniquely multilateral surveillance in nature, in that information on them cannot be obtained simply by aggregating information from bilateral surveillance. Others have a multilateral-surveillance orientation even if the information is obtainable from bilateral surveillance, given their relevance for the global economy or their cross-country focus. Still other issues have their roots largely in bilateral surveillance. With some issues, including those rooted in bilateral surveillance, the IMF may be better positioned than other institutions to provide information because of its greater analytical resources; with other issues, information may be plentiful and the IMF would be just one of many providers.

In light of these considerations, the categories of issues listed above can be depicted in a diagram in which the “multilateral/bilateral surveillance” scale is drawn as the horizontal axis and the “less/more competition from others” scale as the vertical axis (Figure 2.1). In this diagram, the IMF’s contribution to multilateral surveillance increases as one moves from the south-west corner toward the north-east. While these are all legitimate topics of analysis, given the scarce resources, it would make more sense for the IMF to give comparatively more attention to those topics on which it could better contribute to the goals of multilateral surveillance. A trade-off may be involved in choosing topics. For example, the IMF is uniquely positioned to do cross-country comparisons but these have a bilateral (country-specific) focus. Global economic and market developments are an important topic of multilateral surveillance, but equally good information is readily available elsewhere.

Figure 2.1.Potential Topics for Multilateral Surveillance Outputs

What is useful to one audience may not beuseful to another; thus, usefulness does not by itself establish the IMF’s comparative advantage. For example, with regard to a question on the components of the WEO, about half of the national authorities surveyed considered the review of recent economic developments—on which information is readily available elsewhere—to be most useful, about the same as those who considered the discussion of prospective policy issues as most useful (when the sample is restricted to industrial countries, the percentage was less than 25 percent; for the G-7 countries only, no respondent considered the review of recent economic developments to be most useful). With regard to capital market surveillance, although almost 80 percent of respondents considered the GFSR’s review of recent market developments to be most useful, a number of market participants interviewed by the IEO stated that the GFSR offered little new market-related information.

Selection of topics for the WEO

With this background, the IEO evaluation team assessed the selection of topics for the WEO. A close look at the WEO is particularly warranted because it forms the intellectual foundation for much of the IMF’s other multilateral surveillance work.

In discussing topic selection in the WEO, it bears noting that Chapter I differs quite a bit from the rest of the report. Chapter I traditionally presents an overview of economic developments and prospects across major groupings of IMF member countries. Chapter II addresses a topical issue of current interest, such as the house price boom or the role of remittances, while Chapter III analyzes an issue of more medium-term importance, such as population aging. In addition, five or so boxes typically included in Chapter I also discuss topical issues, albeit much more succinctly. Because the Chapter I topics are standard and recurring, we focus initially on topics in the boxes and analytical chapters.

As expected, the WEO’s analytical chapters (including appendices and subchapters) and boxes address a variety of topics, but the overwhelming majority reflect the two areas of the IMF’s greatest comparative advantage: linkage-related and cross-country issues (Figure 2.2). Of the 245 topics covered in the analytical chapters and boxes during 2000-05, 96 topics were related to economic linkages across countries, 106 related to cross-country themes, and the remaining 43 related to other issues. Thus, the WEO’s analytical sections seem to serve two purposes: to present analyses of issues that have implications for economic linkages across countries, and to present analyses of cross-country themes that draw on the experiences of different countries. We consider the selection of these topics to be broadly appropriate.

Figure 2.2.The Composition of WEO Topics, 2000-05

A more detailed breakdown indicates that almost a quarter of the 96 linkage-related topics were concerned with trade, foreign direct investment (FDI), and external balances (Figure 2.3). The second most frequent multilateral surveillance theme in the WEO related to financial markets (16 percent). Compared with the GFSR, however, many of the financial market issues discussed in the WEO had a macroeconomic focus and were often covered in a text box, rather than a full chapter or appendix. Other important themes were economic growth (14 percent) and broadly defined monetary issues (11 percent), which included interest rate policy, inflation, and deflation. Strikingly rare was dedicated coverage of exchange rate issues, with only six observations over six years.1

Figure 2.3.The Breakdown of WEO Linkage-Related Topics, 2000-05

(In percent)

For example, conspicuously missing was an analysis of China’s exchange rate, which in recent years has figured prominently in international policy debate. The issue was discussed in Article IV consultations with China, but the IMF did not use the WEO to discuss whether the renminbi (or any other Asian currency for that matter) was undervalued and, if so, what the alternative paths to adjustment might be and their implications for the adjustment of global imbalances.2 The omission is all the more striking when one considers that the WEO in 2004 included such topics as “China’s Emergence and Its Impact on the Global Economy” and “Learning to Float: The Experience of Emerging Market Countries Since the Early 1990s.”

Selection of topics for the GFSR

The format of the GFSR was fixed in 2003, when the publication became semiannual. The report opens with an overview (Chapter I), which is followed by a review of recent developments and prospective risks in mature and major emerging markets (Chapter II)and then by a discussion of structural topics of medium-term nature in its two analytical chapters (Chapters III and IV).3Chapter II includes several boxes on special themes and may also occasionally include a section on structural issues of current interest. In assessing the selection of topics for the GFSR, we concentrate below on the boxes included in the review chapters and the topics discussed in the analytical chapters.

We find that, unlike the WEO, virtually every topic in the GFSR was related to economic linkages in a broad sense.4 This is to be expected, given its focus on global financial market developments that affect the volume and direction, as well as the risks associated with international capital flows. Much space is devoted to discussing developments and risks in individual countries, including some smaller emerging markets; some discussions have a specific geographic focus. Yet, clear implications are drawn from these for capital inflows or outflows, international linkage of interest rates, and other international transmission channels.

During 2002-05, roughly one-third of GFSR topics related to advanced markets, another third to emerging markets, and the remaining third to other issues with no clear geographic focus, including crisis resolution issues. As to the breakdown of topics by subject, 37 percent involved risk-related issues (e.g., risk transfers and risk management), 32 percent related to the development of market institutions, and 22 percent to capital flows (Figure 2.4). Many boxes had pedagogical functions, explaining methodological issues involved in monitoring financial markets (see also Table 3.1).

Figure 2.4.The Breakdown of GFSR Topics, 2002-05

(In percent)
Table 3.1.Major GFSR Topics, 2003-05
IssueDevelopments and Prospects1Analytical Chapters2
March 2003
  • Developments and risks in mature and emerging markets.
  • Analysis of portfolio shift from risky assets to cash in mature markets and its implications for financial stability.
  • “Local securities and derivatives markets in emerging markets”—developments and policy measures to promote their further development.
September 2003
  • Developments and risks in mature and emerging markets.
  • Possible risks posed by Fannie Mae and Freddie Mac for financial stability, given their size.
  • How hedging in the mortgage market can amplify interest rate movements.
  • Implications of the prospect of rising U.S. interest rates for emerging markets.
  • “Financial asset price volatility”—how to control factors that amplify asset price volatility, including enhanced transparency, better risk management, and improved market infrastructure.
  • “Volatility of private capital flows to emerging markets”—determinants of the pattern and volatility of capital flows and the policy responses that followed the Asian crisis.
April 2004
  • Developments and risks in mature and emerging markets, with a focus on whether asset price levels are justified by fundamentals, and also including corporate governance.
  • “Risk transfer and the insurance industry”—the transfer of risk from banks to insurance companies in major countries, and its implications for risk management in the insurance sector.
  • “Institutional investors in emerging markets”—analysis of how various nonbank institutional investors make asset allocation decisions vis-à-vis emerging market securities, and its implications for the volatility of capital flows to emerging markets.
September 2004
  • Developments and risks in mature and emerging markets, including an update on the insurance industry, the hedge fund industry, and energy trading markets.
  • “Risk management and the pension fund industry”—the challenges faced by pension funds in major countries, measures to enhance their risk management practices, and the implications for the transfer of risk to the household sector.
  • “Financing flows and global imbalances.”
  • “Emerging markets as net capital exporters”—the associated accumulation of net international reserves and how this facilitated the financing of large U.S. current account deficits.
April 2005
  • Developments and risks in mature and emerging markets, including global imbalance issues, energy markets, and an update on the insurance industry and hedge funds.
  • Possible impact on emerging markets of a rise in U.S. interest rates.
  • “Household balance sheets”—greater assumption of risks by the household sector and its implications for risk management and the need to educate the public.
  • ”Corporate finance in emerging markets”—the rise of capital market financing in emerging markets, remaining obstacles to more diversified funding, and the associated vulnerabilities.
September 2005
  • Developments and risks in mature and emerging markets.
  • “Aspects of global asset allocation”—factors that determine asset allocation, the behavior and strategy of different types of institutional investors, growing complexity of investment funds, and how home bias has disappeared.
  • “Development of corporate bond markets in emerging market countries.”

Chapters II and III for the March 2003 GFSR;Chapter II for all others. These chapters follow a largely standard template that includes developments and issues related to bond and equity markets, the banking sector, exchange rate movements, capital flows and emerging market financing, and sectoral balance sheet developments. Only a few topics of special interest are highlighted in the table.

Chapter IV for the March 2003 GFSR;Chapters III and IV for all others.

Chapters II and III for the March 2003 GFSR;Chapter II for all others. These chapters follow a largely standard template that includes developments and issues related to bond and equity markets, the banking sector, exchange rate movements, capital flows and emerging market financing, and sectoral balance sheet developments. Only a few topics of special interest are highlighted in the table.

Chapter IV for the March 2003 GFSR;Chapters III and IV for all others.

Selection of topics in regional outlooks

Partly responding to direction from the Executive Board, IMF staff has recently intensified its efforts to incorporate regional surveillance and cross-country analysis in its country work. The most prominent of these initiatives is the publication of regional outlooks by three area departments—the African Department (AFR), the Asia and Pacific Department (APD), and the Middle East and Central Asia Department (MCD).5 Because these regional outlooks, in their current public format, are relatively new and may still be evolving, our assessment is only tentative.

The three regional outlooks published in 2005 included overviews of the regional economic picture and a few analytical pieces on policy issues of regional relevance. Some of these pieces were of high quality. At least one of them (Chapter II of the Middle East and Central Asia outlook on “Responding to Higher Oil Prices”) received wide international press coverage and contributed to public debate. Aside from these analytical contributions, the bulk of the outlooks (more than 70 percent) were largely descriptive accounts of regional economic developments obtained from Article IV reports on individual countries (Figure 2.5). The weight of descriptive material, however, differed considerably across the three outlooks.

Figure 2.5.The Composition of Topics in Regional Outlooks

(In percent)

Source: Based on regional outlooks for sub-Saharan Africa (AFR, May 2005), Asia and Pacific (APD, August 2005), and the Middle East and Central Asia (MCD, September 2005).

Regional outlooks seem to offer some value to most people, but the perception of their usefulness differs between industrial and developing countries, as well as between B-level staff and A-level staff at the IMF (Figure 2.6).6 Surveys reveal that the authorities of industrial countries and B-level staff are much less inclined to view regional outlooks as “very useful” as opposed to “somewhat useful,” while the reverse holds for the authorities of other countries and A-level staff.7

Figure 2.6.National Authorities and Area Department Staff on Regional Outlooks

(In percent)

Relevance and Timeliness

Another aspect of the content of the products of multilateral surveillance concerns the relevance and timeliness of the issues and risks identified. In this section, we first examine whether the topics discussed in the WEO have subsequently influenced the discussion agendas of the G-7 and G-20.8 We then compare the risks identified in the WEO and the GFSR to those cited in the OECD’s Economic Out-look (OEO), the BIS’s Annual Report (BAR), and the Bank of England’s Financial Stability Review (FSR). (We selected the FSR because it is the oldest and most established national publication.)

Issues analyzed by the WEO

We assess the relevance and timeliness of the topics analyzed in the WEO by using the topics discussed by the G-7 and G-20 as a benchmark. We focus on the WEO, even though the IMF’s G-7 and G-20 surveillance notes can also include special, and directly relevant, analytical pieces on policy issues. These pieces, however, are often prepared at the request of meeting organizers and do not provide insight into whether the IMF has proactively identified those topics. We rely on all the issues of the WEO and the G-7 and G-20 communiqués for the 2000-05 period.

Here, we must cite a caveat. Because the G-7 and G-20 agendas cover various types of issues that are broader than the IMF’s mandate, one cannot expect every agenda item to have been discussed in an immediately preceding WEO. The issues fall under three broad categories: “perennial” topics, unpredictable shocks, and emerging policy issues. Perennial topics are covered regularly by the WEO and the G-7 and G-20, while unpredictable shocks are also almost immediately taken up by all three. In these cases, it is not possible to say whether the appearance of such a topic in a G-7 or G-20 communiquá was prompted by the WEO’s discussion of it. In assessing the relevance of WEO topics, we focus on emerging policy issues.

A review of the documents suggests that about three-quarters of the items for discussion (140 out of 192) were perennial issues that had emerged before 2000 and were discussed regularly thereafter. These included the Heavily Indebted Poor Countries (HIPC) initiatives, reform of the international financial institutions, standards and codes of good practice, trade liberalization, and the Millennium Development Goals (MDGs). Two were related to shocks—namely, the Indian Ocean tsunami of 2004 and a post-September 11 Action Plan to combat the financing of terrorism.9 This leaves 50 items of discussion that could be considered as emerging policy issues, such as the impact of higher oil prices, the impact of the information technology (IT) revolution, remittance flows, and regional issues of strategic interest. Of the 50 items, 36 items constituted separate issues, with at least ten of these (such as issues related to oil prices and the Doha round of multilateral trade negotiations) discussed on successive occasions, sometimes for more than a year. We compared the G-7 and G-20 communiqués with the WEO to see if the WEO discussion of these 36 issues preceded their appearance in the relevant communiqués (Figure 2.7).

Figure 2.7.G-7 and G-20 Emerging Policy Issues in IMF Multilateral Surveillance, 2000-05

(In percent)

More than 40 percent of the emerging issues discussed by the G-7 and G-20 were addressed in IMF multilateral surveillance either prior to or around the same time as the G-7 and G-20 discussion (identified as “early” in the figure). This, however, understates the performance of IMF multilateral surveillance. About 20 percent of the emerging issues not covered—such as developments in Brazil and Turkey—were being addressed by the IMF through other channels, including bilateral surveillance (designated as “addressed through other channels”). Most of the other issues not covered—such as nuclear safety, an Education Action Plan, and security-related issues—were outside the expertise of the IMF. Only five issues (14 percent of total) were addressed “late” by IMF multilateral surveillance. These included some financial-flows-related issues (such as remittances) and oil data transparency. All in all, IMF multilateral surveillance appears to have performed reasonably well in its timely selection of relevant topics; it successfully identified and analyzed emerging issues 75 percent of the time before they were taken up by the G-7 and G-20 forums.

Risks identified by the WEO and the GFSR

There is no absolute yardstick to assess whether IMF multilateral surveillance identified relevant global risks in a timely manner. Here, we compare the risks identified by the WEO and the GFSR during 2003-05 to those identified by similar publications—the OEO, the BAR, and the FSR—to asses show the IMF stacks up against other international and national bodies. We first compare the WEO with the OEO with regard to the identification of global macroeconomic risks, and then compare the GFSR with the BAR and the FSR for global financial risks.

The WEO

A desk review indicates that most of the risks identified in Chapter I of the WEO were downside risks related to external imbalances, sharp exchange rate corrections, adjustments in corporate and household balance sheets, oil market volatility, geopolitical uncertainties, and the like. The WEO and the OEO identified more or less the same vulnerabilities that could pose downside risks to the global economy. Occasionally, one institution was ahead of the other in identifying risks that subsequently became more widely recognized (e.g., the rise in long-term interest rates in the 2004 WEO or the failure of an investment pickup in the 2003 OEO). However, it is difficult to conclude that the overall performance of one was better than the other, even if the focus is on risks relevant to OECD countries.

The difference between the two institutions concerns nuance and emphasis. The OECD tends to analyze more deeply the causes of risks, and it understandably focuses more on OECD member countries. In contrast, the IMF tends to present risks in a more global context and to focus on international linkages. This is particularly true when external imbalances and cross-country growth differentials are addressed. The WEO covered not only developments and prospects in the G-7 countries but also in emerging Asia, and it considers the potential impact of different adjustment paths on emerging market economies.

More generally, the WEO has a wider geographical coverage that includes emerging market economies (although the coverage of these countries has diminished lately because of growing emphasis on external imbalances in industrial countries). For example, the WEO identifies risks related to fiscal positions in Eastern Europe and the Middle East, covers developments in Latin America and Turkey, and assesses the impact on Asia of a worldwide slump in the IT sector. The IEO assessment concludes that the WEO has been successful in identifying relevant global risks in a timely manner, at least relative to its main competing multilateral surveillance publication.

The GFSR

A desk review suggests that the GFSR identified virtually all of the global risks cited by the BAR and the FSR. Such risks include the follow-on effects from oil and commodity price shocks, global external imbalances, and what it called excessive yield compression (or investor complacency to risk). While it is difficult to determine which publication was first to spot any given risk, each document featured discussions of the proximate causes of the main risks, as well as a series of derivative risks associated with the response of different macroeconomic and financial variables to an initial shock.

The various documents differed in some important ways. Relative to the GFSR, the FSR provides fuller information on bank balance sheet fragilities and is more likely to include associated vulnerabilities in its discussion of risks, especially regarding the United Kingdom. Understandably, its focus is more closely linked to those vulnerabilities that affected financial sector firms operating within its jurisdiction, such as potential fragilities in U.K. household and corporate balance sheets. The BAR takes a markedly more global view than the FSR, while at the same time also focusing on risk exposures contained in bank balance sheets. Consistent with the role of the Bank for International Settlements (BIS) in bringing together G-10 central bankers, the BAR’s starting point usually involves shocks to the real economy, carrying over into interest rates and then to asset prices.

The GFSR also has a distinctively global view but it focuses more on capital markets. For example, it frequently refers to potential shocks deriving from sudden changes in investor sentiment, and it is the only publication that lists geopolitical risks in emerging markets.10 The GFSR’s weakness, relative to both the FSR and the BAR, has to do with its analysis of risks associated with bank balance sheets. This, however, is not a weakness of IMF multilateral surveillance more generally, as global banking sector risks are more fully covered in the Monetary and Financial Systems Department’s semiannual Financial Systems Trends, an internal publication initiated in 2005. Integrating this publication more closely would have allowed the GFSR to provide a more balanced coverage of global financial risks.11

Policy Advice and Forecasts

Multilateral surveillance provides value added when it goes beyond the simple aggregation of bilateral surveillance. In this section, we evaluate multilateral surveillance from this perspective. Specifically, we review below how policy advice compares between multilateral and bilateral surveillance and how the bilateral orientation of IMF surveillance affects the quality of forecasts.

Policy advice

IMF surveillance has a strong bilateral orientation, given the dominant role played by area departments in forming judgments on country-related matters. This is clearly indicated by Table 2.1, which compares the IMF’s policy advice for the United States, the euro area, Japan, and China given in the WEO and in Article IV consultation reports in 2004.12 In general, IMF staff reports for Article IV consultations provide richer details and better context for the policy advice being offered. The advice offered in the WEO, on the other hand, is shorter and crisper. Undoubtedly, these differences reflect the fact that a staff country report can devote much more space to discussing policies in a particular country, whereas the space that can be devoted to a particular country in the WEO is rather limited. Other than this, the substance of the policy advice appears to be virtually identical between the WEO and the staff reports.13

Table 2.1.Selected Policy Advice in Multilateral and Bilateral Surveillance, 2004
Country or Area September WEO1Article IV Consultation
United States2
  • “[F]iscal policy should aim to bring the federal government budget back to balance … by the end of the decade. This is also important for the orderly resolution of global current account imbalances … the better-than-projected outcome for FY2004 … should be used to strengthen the targeted outcomes in the coming two years … tax revenues may also need to rise … consideration could be given to broadening the tax base, for example by cutting tax exemptions, and introducing a national indirect tax.”
  • “The challenge for monetary policy is to return interest rates to neutral … the Federal Reserve’s approach of ‘measured tightening’ appears generally appropriate, but… much will depend on the nature of the incoming data.”
  • “Bringing the budget back to balance … within a reasonable period would leave the U.S. fiscal position much better placed .... The recovery provides a valuable opportunity for embarking upon the fiscal effort … emphasis could be laid on reforms to broaden and simplify the tax base, for example by cutting tax expenditures … or by introducing a national indirect tax. Although stronger growth abroad should play a key role in fostering the resolution of global current account imbalances, deter-mined efforts to strengthen the U.S. fiscal position would help boost national saving and ensure that the adjustment is orderly.”
  • “[T]he time has come to start removing stimulus. Although there appears scope for a measured tightening, recent statements byofficials have helped market participants recognize that the withdrawal of monetary stimulus will not be unduly delayed.”
Euro Area3
  • “[T]he long-term fiscal situation in many countries remains difficult … those countries with the weakest underlying positions should seek to reduce underlying deficits by at least ½ percent a year.... Reductions in tax burdens in many countries are also highly desirable, but … are a lower priority until a significant downpayment on fiscal adjustment is in place.”
  • “The widespread endorsement of the Lisbon reform agenda has not been matched by equally widespread implementation .... Progress has been greatest in centrally led reform, notably the Financial Services Action Plan and the Single Market; nationally sponsored reforms have lagged, particularly in labor markets … greater prioritization of the Lisbon Agenda, focused on the key issue of raising labor utilization, could be helpful. Renewed central initiatives for product market deregulation… might also add to incentives for national labor market reforms.”
  • “[C]ountries should demonstrate a continued commitment to medium-term fiscal discipline … countries with weak budgetary positions should undertake measured but high-quality fiscal consolidation that improves the cyclically-adjusted deficit by at least ½ percent of GDP a year.”
  • “There has been measurable progress in, and toward, deregulating and integrating product and financial markets. While much remains to be done, the basic directions and stepsto be taken have been agreed and instigated by area-wide institutions . . . . The greatest concern is over reform in areas where national competencies loom large . . . . The Lisbon reform agenda should be prioritized and focus on boosting work incentives . . . . Stepping up deregulation of goods and services markets … could provide additional momentum for labor market reform … peer pressure on governments could be strengthened, including by ‘naming and shaming’ cases of lagging structural reforms.”
Japan4
  • “[T]he current monetary stance… should be maintained until inflation is firmly positive. If financial markets became concerned that the policy of quantitative easing might end too early, the Bank of Japan could increase the current account target to signal its resolve to maintain the framework until deflation is decisively subdued . . . . as the onset of inflation draws nearer, enhancements to the Bank of Japan’s communication strategy could help to focus inflation expectations, including by setting a suitably positive medium-term inflation objective and by publishing more of the Bank’s views on monetary policy and the inflation outlook.”
  • “[A]chieving savings in FY2004 relative to the budget would be desirable … options for consolidation include further cuts in capital spending, broadening the personal income tax base, and—in the medium term—raising the consumption tax . . . . stepped-up structural reforms could also improve Japan’s fiscal position by increasing potential output growth.”
  • “The current monetary policy stance and strategy … should be maintained until inflation is firmly positive … further increases in the current account target could be considered, if needed, as a way of further signaling to markets the commitment to maintain the quantitative easing framework as long as necessary.... As the onset of inflation draws nearer, enhancements to the BoJ’s communications strategy could help to stabilize inflation expectations. This could include quantifying the BoJ’s inflation objective … [and] publish[ing] more details of its views on monetary policy and the inflation outlook.”
  • “[A]chieving savings in FY2004 relative to the budget would be desirable … options for consolidation could include further social security reforms … further cutting capital spending and broadening the personal income tax base, and (in the medium term) raising the consumption tax .... Front-loaded regulatory reforms … would help to unlock Japan’s economic potential.”
China5
  • “[F]urther monetary tightening is likely to be needed, which would be aided …by greater exchange rate flexibility.”
  • “Fiscal policy also has a key role to play in cooling down the economy, including through saving revenue overperformance and reducing public investment at both central and local government levels.”
  • “[A] further tightening of monetary policy would appear to be needed . . . . [This] would ensure that growth in monetary and credit aggregates comes down ... in line with the central bank’s targets . . . . Increased flexibility of the exchange rate would also improve the effectiveness of monetary policy in containing domestic demand and price pressures.”
  • “Fiscal policy should play a more supportive role in achieving a soft landing of the economy. The authorities are urged to save the expected revenue overperformance .... reduce public investment .... and lower the deficit below the level targeted in the 2004 budget.... the authorities [should] aim to steadily reduce the fiscal deficit by ¼ to ½ percent of GDP per year over the medium term.... A number of measures are needed to address growing concerns over fiscal risks at the local government level.”

The WEO was discussed by the Executive Board on September 3, 2004.

The Article IV consultation with the United States was concluded on July 23, 2004.

The Executive Board discussion of the staff report on “Euro Area Policies” was held on July 26, 2004.

The Article IV consultation with Japan was concluded on July 28, 2004.

The Article IV consultation with China was concluded on July 28, 2004.

The WEO was discussed by the Executive Board on September 3, 2004.

The Article IV consultation with the United States was concluded on July 23, 2004.

The Executive Board discussion of the staff report on “Euro Area Policies” was held on July 26, 2004.

The Article IV consultation with Japan was concluded on July 28, 2004.

The Article IV consultation with China was concluded on July 28, 2004.

This means that, in terms of the content of policy advice, multilateral surveillance differs little from bilateral surveillance. To be sure, the institution’s policy advice must be consistent regardless of the channel of communication. Rather, the problem is that the language of multilateral advice is no more based on explicit consideration of economic linkages and policy spillovers than that of bilateral advice—and it offers no more insight into the possibilities for policy coordination or collective action. It appears that the national and regional sections of the WEO’s Chapter I are meant to reinforce the policy message of bilateral surveillance, not by setting it in an explicitly global context, but simply by repeating the same message.

This is not to suggest that the bottom-up procedure is necessarily a weakness of IMF multilateral surveillance. To the contrary, it can be considered to be its very foundation. According to a number of people interviewed by the IEO, it is the backing of the IMF’s familiarity with individual countries that gives credibility to what the IMF says about the global economy. The critical thing is that this strong country knowledge is appropriately complemented by multilateral perspectives, if the IMF is to play its role as the machinery for international monetary cooperation. For this, there must be an effective counterweight to the inherently bilateral orientation of IMF surveillance.

From the perspective of discussing economic linkages and policy spillovers, the real value added of the WEO seems to lie in the first several pages of Chapter I, which precede the national and regional discussions. These first few pages discuss and analyze extensively such issues as (1) how external imbalances are distributed across countries or regions; (2) how global commodity market developments are related to growth performance in the world economy, and their likely global impact; (3) financial market and exchange rate developments (with cross references to the GFSR); and (4) how to manage potential risks and vulnerabilities in the world economy, including the path to an orderly adjustment of external imbalances. Although this section is rich in analysis, in most cases it does not offer clear and pointed policy advice.

The lack of policy content in this uniquely multilateral section may reflect some understandable factors. Economics is still unable to offer an unambiguous picture of exactly how different policies affect different variables and how different economies interact with each other; there is no clear consensus on how to define what is sustainable and what is not. All in all, the problem of uncertainty is a fact of life in economic policymaking, but it is compounded many times when dealing with global economic linkages and policy spillovers. To be more useful to policymakers, the WEO could get around this problem by spelling out these uncertainties more explicitly—for example, by in dicating possible ways in which economic interactions may play out. It could also make greater use of scenario analysis to bring policy content to an area where concrete policy prescription is not possible. WEO authors have done this over the years, but usually not in the main text of Chapter I (see also Chapter 3, section on “Presenting the Message,” and Table 3.3).

Table 3.3.Use and Presentation of Scenario Analysis in the WEO, 2000-051
DateType of SimulationWhere Reported
May 2000The combined impact of stronger world growth, higher inflation expectations, and a decline in U.S. equity prices on growth and exchange rates in the United States, Europe, and Japan.Box
October 2000The impact on 200’-04 GDP growth in major countries and regions of (‘) a more rapid slow-down of the U.S. economy; and (2) more buoyant growth in the euro area and Japan.Appendix
May 2001The impact on 200’-04 GDP growth in major countries and regions of a more rapid reduction of global imbalances through rapid and more gradual slowdowns of the U.S. economy.Mai n text
October 200’The impact on 2002-06 GDP growth, inflation, and exchange rates in major countries and regions of positive and negative productivity shocks in industrial countries.Main text
December 200’Possible global growth effects of a greater-than-expected global shock to business and consumer confidence.Appendix
April 2002Impact on GDP growth, inflation, and exchange rates in major countries and regions of further weakness in Japan and reduced willingness of investors to extend exposure to countries with structural current account deficits.Appendix
September 2002Sensitivity of GDP growth, current accounts, and real exchange rates in major countries and regions to different assumptions on industrial country productivity.Chapter II
April 2003Long-run effects on GDP, consumption, and investment in the United States and the euro area of Europe’s competition-enhancing structural reforms.Chapter IV
September 2003Assessment of how various emerging market characteristics (e.g., debt load, trade openness, trade exposure to the United States and the euro area, exchange rate pass-through, and monetary regime) affect the volatility of real GDP in response to G-3 real exchange rate volatility.Chapter II
April 2004(1) The impact on GDP, real consumption, and investment in the United States and other industrial countries of alternative paths of the U.S. fiscal deficit.Chapter II
(2) Trade-related impact on welfare, output, exports, imports, and terms of trade in different regions and country groups of faster-than-baseline Chinese integration into the global economy.2Chapter II
September 2004Estimated impact on GDP growth and current accounts in emerging Asia of a decline in China’s import growth.Box
April 2005The implications of different portfolio preferences in the euro area, Japan, and emerging Asia for financial flows into the United States and the sustainability of the U.S. current account deficit.Chapter II
September 2005Alternative adjustment paths of global imbalances, given different assumptions on foreign demand for U.S. assets.Appendix

Unless otherwise noted, simulation is based on MULTIMOD or GEM.

Based on the GTAP model, described in detail in Appendix 2.1 in the April 2004 report.

Unless otherwise noted, simulation is based on MULTIMOD or GEM.

Based on the GTAP model, described in detail in Appendix 2.1 in the April 2004 report.

WEO forecasts

The optimistic bias of WEO forecasts has been documented by a number of econometric studies. The early work of Artis (1997) found that, between 1973 and 1994, the WEO’s one-year-ahead forecasts for industrial countries and current-year forecasts for developing countries had an optimistic bias. More recent studies have reaffirmed the tendency of WEO forecasts to have optimistic bias for developing countries (e.g., Timmermann, 2006).14 These results are replicated by the IEO’s own calculations of the forecast errors of the WEO’s recent current-year and year-ahead forecasts for different regions (Table 2.2).

Table 2.2.WEO Forecast Errors, 1991-20031
(GDP GrowthInflation
Meanz-statMeanz-stat
Current year
Industrial-0.26-2.530.172.71
Emerging Asia-1.07-0.88-1.07-1.61
Latin America0.543.71-6.02-2.09
Transition20.802.36-10.58-1.75
Middle East-0.07-0.062.502.07
Africa1.455.85-3.79-6.39
One year ahead
Industrial0.070.580.253.46
Emerging Asia0.00-0.01-1.38-1.65
Latin America1.086.34-1.95-0.22
Transition21.413.35-13.81-1.02
Middle East-0.16-0.273.382.33
Africa1.375.10-5.65-7.77

A positive number indicates that the WEO forecast overpredicted the variable concerned (i.e., a positive number for growth and a negative number for inflation represent optimistic bias). For inflation in Latin America and transition countries, the samples start in ‘992 and ‘996, respectively, in order to remove extreme values.

Also includes some nontransition EU accession countries.

A positive number indicates that the WEO forecast overpredicted the variable concerned (i.e., a positive number for growth and a negative number for inflation represent optimistic bias). For inflation in Latin America and transition countries, the samples start in ‘992 and ‘996, respectively, in order to remove extreme values.

Also includes some nontransition EU accession countries.

Table 2.2 indicates that, for the period 1991-2003, the accuracy of WEO forecasts differed from region to region. Current-year forecasts significantly under-predicted growth and overpredicted inflation for industrial countries; for one-year-ahead forecasts, the pessimistic bias remained for inflation but not for growth. On the other hand, the WEO’s current-year forecasts significantly overpredicted growth for Latin America, the countries in transition, and Africa, while they significantly underpredicted inflation for these countries; the optimistic bias for growth remained for one-year-ahead forecasts for these regions. The optimistic bias for African growth was both statistically significant and numerically relatively large during 1991-2003.

Part of the bias in WEO forecasts may be the inevitable result of their conditional character. Every issue of the WEO spells out the assumption that “established policies of national authorities will be maintained.” In this respect, the forecasts defer to the intentions of national authorities, as opposed to their likely behavior, suggesting a possible built-in optimistic bias. The optimistic bias recently observed, however, relates almost exclusively to those regions with relatively small economies, so that the bias does not materially distort the IMF’s view of the overall global economy.

In fact, in terms of root mean squared errors (RMSEs),15WEO forecasts on growth and inflation compared favorably to the corresponding Consensus forecasts for the G-7, Europe, and emerging Asia during 1991-2003 (Table 2.3).16 For the G-7 and Europe, in particular, the WEO forecast errors (both current-year and year-ahead) were extremely close to the Consensus forecast errors.17 The WEO forecast errors for Eastern Europe and Latin America appeared to be numerically larger than those of the corresponding Consensus forecasts. This most likely reflected the optimistic bias of the WEO forecasts observed in Table 2.2.

Table 2.3.WEO and Consensus Forecast Errors, 1991-20031
GDP GrowthInflation
WEOConsensusWEOConsensus
Current year
G-71.391.360.530.48
Europe1.351.390.940.94
Emerging Asia2.141.972.232.30
Eastern Europe3.993.3121.3617.55
Latin America2.912.4847.3443.01
One year ahead
G-71.541.490.770.77
Europe1.711.751.211.29
Emerging Asia3.092.873.554.29
Eastern Europe4.474.0161.6360.42
Latin America4.063.7584.2656.53

Root mean squared errors. The country groupings correspond to the Consensus regions. Data are not available for Africa.

Root mean squared errors. The country groupings correspond to the Consensus regions. Data are not available for Africa.

The IEO believes that part of the consistent bias observed for Africa (and, to a lesser extent, other regions) relates to how the forecasting exercise is conducted. While the Research Department coordinates the process, the growth forecasts provided by area departments for most countries are rarely adjusted. This allows the optimistic bias of country desks to come through.18 The lack of optimistic bias for the industrial countries supports this conjecture. The evaluation team has learned that RES interacts far more extensively with major country desks at various stages of preparation, so that global consistency is better incorporated into major country forecasts. Also, the greater availability of public information (including competing forecasts by Consensus Economics and other institutions) may place discipline on the area departments’ forecasts for these countries.

Forecasts are never perfect, but the fact that some are biased in one direction, year in and year out, may already have undermined the credibility of IMF surveillance. About twice as many of the national authorities surveyed replied that they agreed with the statement that “WEO forecasts are optimistic,” compared with those who disagreed.19 In addition, all of the forecasters at regional development banks interviewed by the evaluation team felt that WEO growth forecasts for their respective regions were almost always more optimistic than their own.20WEO forecasts, however, seem to enjoy considerable influence in the public and private sectors. A number of national officials and market participants explained to the IEO that, even when they disagreed with WEO forecasts, they still considered them as a benchmark—for example, to evaluate their own assessments. Less biased forecasts probably would make an even better benchmark.

Integration of Capital Market and Macroeconomic Analysis

The IMF has made considerable efforts to build up its capital market expertise, but capital market analysis has yet to be integrated in the IMF’s core macroeconomic work, in the view of outside experts.21 Part of the lack of sufficient integration between macroeconomic and capital market analysis is that integration is difficult—or may even be unnecessary—in some areas. Many of the issues and risks discussed in the GFSR’s Chapter II fall in this category; for example, the April 2005 GFSR noted:

Box 2.1.Macroeconomic and Capital Market Surveillance: Corporate Accounting Scandals of 2001-02

The series of accounting scandals in 2001-02 involving financial misrepresentations by major corporations in the United States and elsewhere represented a global financial shock with potential real economic consequences. Both the WEO and the GFSR covered this event.

WEO coverage. The April 2002 WEO stressed the emerging signs of global economic recovery, but dedicated only a single paragraph to the risk that financial markets might be optimistic about earnings expectations. This paragraph included a brief mention of weaknesses in the accounting and auditing framework revealed by the recent scandals, and referred the reader to the GFSR. Separately, the WEO contained an analytical chapter entitled “Three Essays on How Financial Markets Affect Real Activity.” Half a year later, the September WEO expressed concerns about the pace and stability of the recovery, following the weakening of financial markets and the deterioration of financing conditions for emerging markets. The WEO analyzed in some detail the impact on the short-term outlook of recent equity market declines and cited the risk that any new accounting scandals might set off further declines in equity markets. The GFSR’s call for strengthening corporate governance and transparency (see below)was cited several times. Yet, the WEO was upbeat by noting positive indicators for sustained recovery over the medium term, albeit at a slower pace than previously expected.

GFSR coverage. 1 The discussion in the March 2002 GFSR was largely descriptive of the recent scandals and the channels through which the markets had already reacted. The June GFSR, however, identified the quality of corporate profits in mature markets as the main source of uncertainty, with attendant risks to the balance sheets of insurance companies in the event of substantial equity market declines—declines that materialized just one week later. The September GFSR noted the apparent resilience of markets but also highlighted the risk that investor confidence might continue to erode to the point of withdrawing en masse from risk taking, with further equity price corrections cascading across markets to trigger liquidity events. A related risk identified in the GFSR was that the capital positions of financial institutions were being impaired to the point of causing systemic problems. Recommendations were longer-term in nature, including increased vigilance by supervisors and participation in the IMF’s initiatives on standards and codes. By December, the confidence shock triggered by the collapse of Enron was essentially over. The December GFSR noted that the risks to international financial market stability remained limited and manageable, thanks in part to U.S. monetary policy action and key regulatory measures to restore investor confidence in corporate accounting.2

Comparing the WEO and the GFSR. Although the WEO and the GFSR covered the same event, their approaches differed in two ways. First, the GFSR adopted a short time horizon in monitoring market sentiment and market positions, whereas the WEO applied analytical tools to the incoming data—drawing conclusions more germane to resolving starting point uncertainty of the real economy and its prospects over the medium term. Second, the September GFSR’s mostly intuitive discussion of systemic risks to financial systems did not feature in the WEO, while the data-intensive analytical work of the WEO on assessing bubbles, balance sheets, consumption, and investment more broadly did not feature in any GFSR. The WEO and the GFSR cited each other, however, thus giving the reader an alternative point of reference.

1 The GFSR was published quarterly in 2002.2 The Sarbanes-Oxley Act strengthening oversight of accounting was passed on July 30, and a key August 14 deadline for the executives of selected listed U.S. companies to certify their financial accounts passed uneventfully.

“At some point, markets may become impatient with the pace of change, and asset prices will start to play a more forceful role in bringing about the needed adjustments. In that event, U.S. government bond yields and credit spreads on corporate bonds would likely increase sharply … contributing to a deterioration of the external financing environment for emerging markets” (p. 21).

This possible event is cast in such a hypothetical way that it does not elicit a need for response. Likewise, the analytical chapters of both the GFSR and the WEO have also included many issues that do not allow for easy integration of the macroeconomic and capital market approaches—such as risk management practices, fiscal policy, and labor market institutions.

With respect to a number of issues, however, integration between the GFSR and the WEO would have been both feasible and desirable but did not take place. Examples from the analytical chapters include the following:

  • In September 2003, the WEO discussed reserve accumulation in Asia and the impact of exchange rate volatility on emerging markets, while the GFSR analyzed the volatility of private capital flows to emerging markets. Both addressed crisis vulnerabilities but they did not even cite each other. A year later, in September 2004, the GFSR featured a chapter on emerging markets as net capital exporters and discussed reserve accumulation in this context, but it made only a passing reference to the September 2003 WEO discussion of the macroeconomic aspects of the same topic.
  • The September 2004 WEO briefly discussed the impact of population aging on financial markets and public pension plans. This discussion, however, did not draw on the GFSR analysis that also appeared in the same month of the longerterm issues for the pension fund industry.
  • The September 2004 WEO discussed the global house price boom, analyzing why the structure of mortgage contracts differed across countries. Six months later, the April 2005 GFSR discussed mortgage contracts from a household risk management perspective but made only a footnoted reference to the September 2004 WEO.

These analyses were stand-alone pieces that did not necessarily require the support of an alternative analytical approach to be complete. Still, better integration of the macroeconomic and capital market approaches would have provided additional insights.

Similar examples can also be found in the review chapters (i.e., Chapter I of the WEO and Chapter II of the GFSR). These chapters in the two publications follow standard templates that cover financial market issues without overlapping each other, but where one stops and the other begins appears largely arbitrary. When the GFSR and the WEO deal with the same issues (including energy market developments recently), they do not seem to go much beyond making cross references to each other (Box 2.1). The overall tone could also be different. For example, in April 2005, the WEO struck a cautionary note on the outlook in light of diverging patterns of growth and growing external imbalances. The GFSR was more sanguine, however, seeing “no particular reason to believe that this benign scenario might come to an end anytime soon”. These examples demonstrate the so-called “silo”problem of the IMF’s internal organization.

1

In addition, there were 6 cross-country analyses on exchange rate issues, so that the total number of exchange-rate-related topics in the WEO was 12.

2

Following the renminbi exchange rate reform in July 2005, the September 2005 WEO discussed the economic impact of greater exchange rate flexibility in emerging Asia accompanied by a decline in the rate of foreign exchange reserve accumulation.

3

In the April 2003 issue only, developments and risks were reviewed separately for mature markets (Chapter II) and emerging markets (Chapter III), and a structural topic was covered in one chapter (Chapter IV).

4

The selection of topics for the GFSR is guided by an explicit objective stated in the preface that appears in every issue: “The Global Financial Stability Report (GFSR) assesses global financial market developments with a view to identifying potential systemic weaknesses. By calling attention to potential fault lines in the global financial system, the report seeks to play a role in preventing crises, thereby contributing to global financial stability and to sustained economic growth of the IMF’s member countries.”

5

The Western Hemisphere Department (WHD) also prepares aregional outlook, but it is available only through the department’s website and is generally used as material for presentations by senior staff. Consequently, its publication does not involve the IMF’s internal review process. The European Department (EUR) does not prepare a regional outlook of any kind, but organizes itsregional surveillance work along subgroups of countries, including the European Union (EU).

6

Within the IMF, B-level staff refers to senior staff (Directors, Deputy Directors, Senior Advisors, Advisors, and Division Chiefs), while Alevel staff refers to Economists and Senior Economists.

7

“Somewhat useful” combines “useful, but more resources are not needed” and “marginally useful” in the staff survey and corresponds to “useful, but value-added is small” in the authorities’ survey. A further breakdown (not reported in the figure) indicates that the share of national authorities who considered regional out-looks to be “highly helpful” exceeded 90 percent in MCD countries, about 80 percent in AFR countries, 60 percent in APD countries, more than 50 percent in EUR countries, and less than 50 percent in WHD countries.

8

The GFSR focuses more narrowly on a few issues, and its intersection with the G-7 or G-20 tends to be limited. The IEO evaluation found that the GFSR discussed only four emerging policy issues subsequently raised in G-7 communiqués.

9

The terrorism financing issue was discussed at many subsequent meetings and is thus considered to have become a perennial issue.

10

Emerging market considerations were not absent from the other publications, but the discussion usually related more to cyclical developments (e.g., an economic slowdown in China) than to geopolitical effects on investor risk appetite.

11

It is expected that the forthcoming merger of ICM and MFD will result in a better integration of capital market and financial sector surveillance.

12

We chose 2004 for illustrative purposes only. Almost identical conclusions can be drawn for any year between 2001 and 2005.

13

In contrast, the GFSR is not produced through a bottom-up process and seems to be less constrained by the views of area departments. In the April 2005 issue, for example, the GFSR noted: “a revaluation of the Chinese renminbi is seen as the key to a broadening of the adjustment process. A revaluation of the renminbi would probably create headroom for other Asian currencies to strengthen, and pressures on them to do so would intensify.” At this time, the area department (along with the WEO) was publicly arguing for China’s need to introduce greater exchange rate flexibility, and not for a revaluation of the Chinese currency.

14

Timmermann (2006) found that, between 1990 and 2003, the IMF tended to overpredict growth and underpredict inflation incurrent year forecasts for several regions, especially Africa, Central and Eastern Europe, and the Commonwealth of Independent States.

15

The RMSE is the most widely used measure of forecast accuracy and is given by: RMSE=(Σt(FtRt)2)/n,

where Ft and Rt are, respectively, the forecast and realized valuesof the variable in question, t is a time subscript, and n is the number of observations.

16

Consensus forecasts represent the means of representative private sector forecasts compiled by Consensus Economics, a private U.K.-based data provider.

17

Country groupings in Table 2.3 correspond to the Consensus regions and do not exactly match the WEO groupings used in Table 2.2. Insufficient coverage of Africa in Consensus forecasts precludes a similar comparison for that region.

18

For example, U.S. General Accounting Office (2003) found that, between 1990 and 2001, WEO forecasts for growth and inflation were optimistically biased for 57 countries under IMF-supported programs (see also Timmermann, 2006). Ghosh and others (2005), however, cite evidence that optimistic bias is no greater in program countries than in nonprogram countries.

19

About 40 percent of the private sector experts surveyed agreed with the statement that WEO forecasts were optimistic, while another 40 percent stated that they neither agreed or disagreed.

20

A formal test would show, however, that these impressions are not supported by actual data. The IMF’s growth forecasts have in fact been less optimistic than those of some other multilateral institutions for relevant regions.

21

As recently as November 2005, the McDonough Report cited the need to “foster effective cross-fertilization between area and functional departments and ensure that available financial expertise is fully and efficiently brought to bear in the Fund’s bilateral and multilateral work.” See Report of the Review Group on the Organization of Financial Sector and Capital Markets Work at the Fund, November 2005.

    Other Resources Citing This Publication