- International Monetary Fund. Independent Evaluation Office
- Published Date:
- April 2016
Independent Evaluation Office
of the International Monetary Fund
Behind the Scenes with Data at the IMF An IEO Evaluation
© 2016 International Monetary Fund
Joint Bank-Fund Library
Names: Wagner, Nancy L. (Nancy Louise) | International Monetary Fund. Independent Evaluation Office.
Title: Behind the scenes with data at the IMF : an IEO evaluation / this report was prepared by an IEO team led by Nancy Wagner.
Description: Washington, DC : Independent Evaluation Office, International Monetary Fund, 2016. | Includes bibliographical references
Identifiers: ISBN 978-1-49838-598-5 (paper)
Subjects: LCSH: Economics, Mathematical. | Finance—Statistics. | Economic indicators. | International Monetary Fund.
Classification: LCC HB137.B44 2016
Publication orders may be placed online, by fax, or through the mail:
International Monetary Fund, Publication Services
P.O. Box 92780, Washington, DC 20090, U.S.A.
Tel. (202) 623-7430 Fax: (202) 623-7201
- Executive Summary
- 1 Introduction
- 2 Evaluation Framework
- 3 Evolution of Data Activities at the IMF: Progress Through Crises
- 4 The Current State of Play
- 5 Main Findings and Analysis
- 6 Recommendations
- 1. A, B, or C? Grading a Country’s Data Adequacy for Surveillance
- 2. Argentina and the Breach of Obligations
- 3. Greece: Policy-Based Evidence-Making and the Perils of Statistics
- 4. Faulty Data and Faulty Analysis: Past Examples
- 5. You Don’t See What You’re Not Looking For
- 6. Does Lack of Data Still Prevent the Use of Balance Sheet Analysis?
- 7. The Common Surveillance Databases and the Quest for Better Data Sharing at the Fund
- 8. Data Management Practices in Comparable Institutions
- 9. China: Subscribing to the SDDS
- 10. Pitfalls in Building a Data Governance Framework
- 1. Staff Perceptions of Data Quality and Availability for Operations
- 2. Staff Perceptions of Data Quality and Availability for Bilateral Surveillance
- 3. Did Your Country’s Last Article IV Report Include a SIA?
- 4. Perception of Statistics Technical Assistance
- 5. Data Flows at the IMF
- 6. New Data Governance Structures at the IMF
- 7. Staff Perceptions of DMX
- 8. Staff Perceptions of Centralized Provision of Data Services
- 9. Staff Perceptions of Data Management Practices Across Departments
- 10. Use of IMF Specialized Databases
- 11. Survey Responses: “Data Quality is Monitored and Endorsed by the IMF”
- 12. Familiarity with the GDDS and SDDS
- 13. Survey Results: “Subscription to the SDDS/GDDS improved my country’s . . .”
- 14. Data Issues and the IMF’s Mandate
- 1. Key IMF Databases and Data Initiatives
- 2. Summary of Background Papers and Document
- 3. A Brief History of Data and Statistics at the Fund
- 4. Key Data-Related Findings in Selected IEO Evaluations and IMF Policy Reviews
- 5. Do Staff Follow the Operational Guidelines on Data Provision for Fund Surveillance?
- 6. Comparability of Data Across Countries
- 7. Past Work on Data Management Issues at the IMF
- Statement by the Managing Director and the acting chair’s summing up
- Statement by the Managing Director
- The Acting Chair’s Summing Up
- Background Work
- The following Background Papers and Background Document are available on the IEO website at www.ieo-imf.org.
- BP/16/01. The Rules of the Game: Data-Related Mandate, Obligations, and Practices at the IMF
- BP/16/02. Progress Through Crises: The Evolution of the IMF’s Statistical Arsenal
- BP/16/03. Old Acquaintances: Past Views on Data Problems in the IMF
- BP/16/04. Inadequate Statistics and Faulty Analysis
- BP/16/05. On the Effect of IMF Data Standards Initiatives: Do They Affect Foreign Direct Investment, Exchange Rate Volatility, and Sovereign Borrowing Costs?
- BP/16/06. Data and Statistics at the IMF: Quality Assurances for Low-Income Countries
- BD/16/01. How Well Is the IMF Doing on Data? Evidence from Surveys
The following conventions are used in this publication:
- An en dash (–) between years or months (for example, 2015–16 or January–June) indicates the years or months covered, including the beginning and ending years or months; a slash or virgule (/) between years or months (for example, 2015/16) indicates a fiscal or financial year, as does the abbreviation FY (for example, FY2016).
- “Billion” means a thousand million; “trillion” means a thousand billion.
As used in this publication, the term “country” does not in all cases refer to a territorial entity that is a state as understood by international law and practice. As used here, the term also covers some territorial entities that are not states but for which statistical data are maintained on a separate and independent basis.
Some of the documents cited and referenced in this report were not available to the public at the time of publication of this report. Under the current policy on public access to the IMF’s archives, some of these documents will become available 3 or 5 years after their issuance. They may be referenced as EBS/YY/NN and SM/YY/NN, where EBS and SM indicate the series and YY indicates the year of issue. Certain other types of documents may become available 20 years after their issuance. For further information, see www.imf.org/external/np/arc/eng/archive.htm.
Data issues have long been a source of concern for the IMF. Since the 1980s, IMF staff has presented more than 150 papers on data issues for the Executive Board’s consideration. However, many of the problems raised in the earlier papers remain largely unresolved. Moreover, each of those earlier papers tended to look at just a small slice of the picture regarding data—that is, a piecemeal approach. This report tries to look at data in a holistic manner.
In particular, we try to look at data from the perspective of the Fund’s role as a global macroeconomic risk manager, that is, its role in (i) crisis prevention/mitigation (surveillance) and (ii) crisis response (lending). Data are essential in discharging these responsibilities, leading to a seemingly simple, yet complex question: “Are data adequate for surveillance and lending?”
Indeed, problems with data or data practices—missing data, misleading data, or ignoring available data—have at times hampered the Fund’s ability to respond effectively to imminent challenges. As a result, for a time after each crisis, data issues are front and center, resulting in major changes in the Fund’s statistical arsenal. But this attention to data tends to wane after a while, as data become, once again, an afterthought.
The roots of these problems are diverse, ranging from external (e.g., member country capacity constraints) to internal (e.g., lack of appropriate staff incentives, entrenched work practices). While most of these have been recognized for decades, they are now cast in a different light due to the proliferation of data sources, technological advances, and a surge in demand for multilateral and financial surveillance and cross-country analyses. This presents greater challenges for the Fund, but also greater opportunities for change.
This evaluation found that noteworthy progress has been made—particularly with external data provision and internal data management—but important obstacles to reform have yet to be tackled. The report thus advocates, first and foremost, that the IMF should design and implement a long-term overarching data strategy, one that goes well beyond data management and recognizes data as a strategic institutional asset. Our other recommendations are important elements of such a strategy, but their implementation could begin in parallel: define and prioritize the IMF’s data needs; reconsider the role and mandate of the IMF’s Statistics Department; reexamine staff incentives; and make clear the IMF’s responsibility regarding the quality of the data it disseminates.
I am encouraged by the broad agreement, expressed by the Managing Director and the Executive Board, with the findings and recommendations of this report. It is our hope that this report will help catalyze the efforts to address remaining data problems and thereby better support the Fund in delivering on its evolving and more challenging role in today’s increasingly interconnected global economy.
Moises J. Schwartz
Independent Evaluation Office
Behind the Scenes with Data at the IMF: An IEO Evaluation
This report was prepared by an IEO team led by Nancy Wagner. The IEO team included Miguel de Las Casas, Chris Monasterski, Roxana Pedraglio, and Thomas Reichmann. The evaluation was informed by background studies prepared by Carlos de Resende, Morten Jerven, Franz Loyola, Tam Nguyen, and members of the evaluation team. The evaluation benefited from discussions with participants—William Alexander, John Boorman, Eduard Brau, Carol Carson, Philip Cross, Donal Donovan, John Hicklin, Russell Kincaid, Anne Krueger, Jin Liqun, Meg Lundsager, David Robinson, Marko Škreb, Hector Torres, Edwin Truman, and Onno Wijnholds—at two workshops that took place in Washington, D.C., and at a workshop in Berlin organized jointly with the Federal Ministry for Economic Cooperation and Development (BMZ) and the German Institute for Development Evaluation (DEval). It also benefited from comments by IMF staff. However, the final judgments are the responsibility of the IEO alone. Arun Bhatnagar, Annette Canizares, and Amy Gamulo provided administrative assistance. Rachel Weaving, Roxana Pedraglio, and Esha Ray provided editorial and production management assistance. The report was approved by Moises Schwartz.
African Department (IMF)
Asia and Pacific Department (IMF)
Bank for International Settlements
balance sheet analysis
Coordinated Direct Investment Survey
credit default swap
Consultative Group on Exchange Rate Issues
Currency Composition of Official Foreign Exchange Reserves
consumer price index
Coordinated Portfolio Investment Survey
Common Surveillance Databases
Data Gaps Initiative (G20)
Data Management for Excel
Data Quality Assessment Framework
Dissemination Standards Bulletin Board
External Balance Assessment
European Central Bank
Economic Outlook Suite
Economic Data Governance Group
Economic Data Management Initiative
Economic Data Steering Committee
Economic Data Team
Economic Data Warehouse
Enhanced General Data Dissemination System
Economist Intelligence Unit
Hellenic Statistical Authority
emerging market economy
European Department (IMF)
Early Warning Exercise
Fiscal Affairs Department (IMF)
Finance Department (IMF)
Financial Stability Assessment
Financial Sector Assessment Program
Financial Stability Board
Financial Soundness Indicator
Financial Sector Stability Assessment
Fiscal Transparency Evaluation
A grouping composed of major advanced economies and systemically important emerging market and developing countries
Globally Systemically Important Financial Institutions
General Data Dissemination System
gross domestic product
Global Financial Stability Report
Human Resources Department (IMF)
Inter-Agency Group on Economic and Financial Statistics
international financial institution
International Financial Statistics
International Investment Position
Integrated Monetary Databases
Information Technology Department (IMF)
National Statistics and Census Institute
Middle East and Central Asia Department (IMF)
Monetary and Capital Markets Department (IMF)
nonbank financial institutions
Organisation for Economic Co-operation and Development
Office of Internal Audit and Inspection (IMF)
Office of the Managing Director (IMF)
Poverty Reduction and Growth Facility
Regional Economic Outlook
Research Department (IMF)
Report on the Observance of Standards and Codes
Special Data Dissemination Standard
Statistical Data and Metadata Exchange
Statistical Issues Appendix
System of National Accounts
Strategy, Policy, and Review Department (IMF)
standardized report forms
Statistics Department (IMF)
Technology and General Services Department (IMF)
Technical Memorandum of Understanding
Triennial Surveillance Review
use of Fund resources
World Economic Outlook
Western Hemisphere Department (IMF)