Chapter

VI Measurement Issues

Author(s):
Paul Hilbers, Alfredo Leone, Mahinder Gill, and Owen Evens
Published Date:
April 2000
Share
  • ShareShare
Show Summary Details

This section reviews statistical issues pertaining to the two major types of statistical information used in macroprudential analysis: macroeconomic indicators pertaining to the financial sector (financial macrostatistics) and aggregated microprudential data. Financial macrostatistics—such as monetary statistics, financial accounts of the System of National Accounts (SNA), and sectoral balance sheets—are frameworks for organizing data into comprehensive overviews of the condition and transactions of the financial sector and its key components, and thus can provide indicators of the activity and operation of the financial system. Aggregated microprudential data are summations of (mostly) supervisory information on the condition of individual banks that may provide indications of the overall condition of the financial sector.

We examine key issues affecting the statistical accuracy, usefulness, and international comparability of MPIs, and consider how the IMF could integrate work on MPIs into its statistical programs and support national authorities in the compilation of timely and reliable statistics needed to assess the condition of the financial system. Appendix I reviews the statistical frameworks for compilation of macroprudential data that are in place at the IMF, other international organizations, and selected central banks and supervisory agencies, and reviews the suitability of these frameworks for compilation of macroprudential data.72

The importance of reliable statistics in the assessment of the condition of the financial sector is well established. Unfortunately, in a significant number of problem cases, available statistics have not been of sufficient timeliness and/or quality to provide early and clear warning of emerging difficulties. In this connection, the importance and quality of monetary, balance of payments, and financial system data, as well as the need for comprehensiveness in the collection, methodological soundness of the compilation, accuracy of compilation, and timely and informative public disclosure have often been emphasized. Moreover, comparability of MPIs across countries contributes strongly to their usefulness, a point emphasized at the September 1999 consultative meeting by private sector users of MPIs. Such comparability can be achieved through adherence of MPIs to internationally agreed supervisory, accounting, and statistical standards that provide clear rules for both the compilation and interpretation of MPIs.

Financial macrostatistics and aggregated microprudential data, which are both used in macroprudential analysis, interrelate in numerous ways because both are derived from individual banks’ balance sheets and other detailed financial information. The two types of data could be brought into closer correspondence by applying standard statistical concepts (such as definitions of residency, sectors, and financial instruments) when compiling aggregate microprudential data, and by enhancing financial macrostatistics with additional detail needed for macroprudeniial analysis (such as information on nonperforming loans).

Statistical Frameworks for MPIs

Financial Macrostatistics

Nearly all countries compile financial sector macroeconomic statistics, primarily in the form of monetary statistics. However, monetary statistics generally do not provide the specific types of data used for macroprudential analysis or may lack needed detail. Other financial statistics frameworks, such as flow of funds accounts or sectoral balance sheets,73 can provide the detailed financial information for the financial sector and other sectors of the economy that can be used For macroprudential analysis. Among the numerous MPIs that can be constructed directly from monetary statistics or other financial macrostatisticai frameworks are: central bank credit to banks, the ratio of deposits to M2, the ratio of loans to capital, the ratio of loans to total deposits, lending to nonresidents, the ratio of foreign currency loans to total loans, the ratio of foreign currency liabilities to total capital, and the distribution of credit by sector.

International standards exist for the construction of these macrostatistics frameworks, which contribute to their comparability across countries. An important attribute of these frameworks is that they present specific sectors within the context of the overall economy and can be used to analyze the dynamics of the financial sector and the transmission of financial stress across sectors. Also, these frameworks are flexible and can be enhanced with additional detail needed for macroprudential analysts. These frameworks are highly developed in only a few countries.

The IMF is moving to promote compilation of financial sector macroeconomic statistics harmonized with international standards through the forthcoming Monetary and Financial Statistics Manual. Financial statistics compiled in accordance with the manual can be further augmented to provide more macroprudential information, such as on impairment of claims, credit concentration, maturity of liabilities, subordinated debt, capital adequacy, connected lending, and relations with foreign affiliates.74 Work is currently ongoing also at the ECB to augment the monetary statistics program with macroprudential information (see Appendix I).

Aggregations of Microprudential Data

The second major type of information used for macroprudential analysis consists of summations of information used by supervisors to assess the condition of individual banks. In addition to the use of these data in specific MPIs, a recent report by the Bank of England called for national supervisory authorities to design a template with minimum requirements for key indicators of bank quality for disclosure of aggregated microprudential data to the public: We recommend that national supervisory agencies lake upon themselves the responsibility for the collection, compilation and dissemination of data on banks to meet the needs of users. These data would be at least at the peer group and aggregate level; both on solo and consolidated basis; and include key indicators of capital, asset quality, earnings and liquidity, such as capital adequacy ratios, non-performing loans as a percentage of total assets, return on assets and equity, and a breakdown of assets and liabilities by maturity. Data should be published on a quarterly frequency. The above list is only a suggested bare minimum and not a comprehensive list of indicators. A common disclosure template in the form of a minimum requirement could be agreed on by the Basel Committee on Banking Supervision and could be so designed to meet the needs of macroprudential surveillance. This would require implementation of greater disclosure requirements than those currently applicable in many countries, and possibly even legislative changes to augment the authority of supervisors to ask for and to publish these data. We recommend that countries take up this task with the priority it deserves.75

Some microprudential information can be meaningfully aggregated to provide a useful depiction of the condition of the financial sector. Some other microprudential information, however, may reflect specific information needs of supervisors on the condition of individual batiks that might prove difficult to aggregate or unsuitable for aggregation. For example, VaR analysis is only valid for the analysis of specific portfolios. Other potential MPIs are affected in a similar way. Also, simple aggregation of prudential information of individual banks can disguise important structural information, and it is often necessary to supplement the aggregate data with information on dispersion, peer-group analysis, and the interrelationships between systemically large banks.

It is instructive to review how the most commonly used indicator, the risk-based capital ratio, could be aggregated into a statistic to describe the condition of the banking sector. The ratios for individual banks cannot be directly aggregated—data on the numerator (capital) and the denominator (risk-adjusted assets) must be collected from each bank and separately aggregated. The supervisory definition of capital used as the numerator is unique so that data cannot be extracted directly from either accounting records or statistical sources, and there are analytical needs to compile separate information on the three tiers of capital recognized by supervisors. Likewise, data on risk-weighted assets used in the denominator are also based on supervisory concepts not used in accounting or statistical work, and thus are not comparable across countries because they are affected by national accounting practices for valuation of assets, accrual of income, and recognition of impairment. The aggregate ratio is calculated by simple division of the aggregate numerator by the aggregate denominator, A low ratio is a clear sign of vulnerability, and a declining trend may signal increased risk exposure and possible capital adequacy problems. A relatively high ratio, however, does not guarantee that there are not serious difficulties in financial institutions that account for a significant share of the system’s assets.

There have been numerous calls for compilation and dissemination of information on the aggregate risk-based capital ratio but, as described above, a number of practical and conceptual issues, and decisions about ancillary information, need to be considered in creating a statistical measure of the ratio.

Statistical Issues Affecting MPIs and International Comparability

Table 4 summarizes some of the major statistical issues affecting MPIs.76 This table cross-classifies selected MPIs by major types of issues that could impede their construction, affect their usefulness for analysis or disclosure, or affect international comparability. The focus is on issues related to compilation of MPIs constructed from aggregated individual bank prudential data, which—in contrast to financial macrostatistics, for which there are recognized international standards-—are often affected by a range of statistical problems that might impair their comparability across countries and reliability as indicators. Even where ample individual bank prudential data exist, there might be practical difficulties or conceptual problems in compiling them into statistical aggregates. The most important statistical issues are discussed in the following subsections.

Table 4.Statistical Issues Affecting MPIs
No Prudential

Standards
No Statistical

Standards
Diverse

Accounting

Standards
Consolidation

Issues
Poor Data on

Asset Quality
Bank-Specific

Information
Derivatives

Issues
Capital adequacy indicators
Aggregate capital adequacy ratios
Distribution of the capital adequacy ratiosn.a.n.a.
Asset quality indicators
Lending institution
Sectoral credit concentration
Ratio of foreign currency loans to total loans
Ratio of nonperforming loans and provisions to
total loans
Loans to unprofitable public sector entities
Provisions for nonperforming loansn.a.
Risk profile of assets
Ratio of connected lending to total lendingn.a.
Ratio of loans to capital (leverage ratio)
Delays in payments
Borrowing institution
Debt-equity ratiosn.a.
Corporate profitabilityn.a.
Other indicators of corporate conditionsn.a.n.a.n.a.n.a.n.a.n.a.n.a.
Household indebtednessn.a.n.a.
Management indicators
Ratio of expenses to total revenue
Earnings per employee
Number of newly licensed institutionsn.a.n.a.n.a.n.a.n.a.n.a.n.a.
Profitability indicators
Ratio of net profits to assets
Ratio of net profits to equity
Ratio of net interest income to income/assets
Ratio of operating expenditure to income/assets
Narrow customer basen.a.n.a.n.a.n.a.n.a.
Interest rate spreadsn.a.n.a.n.a.n.a.
Liquidity indicators
Central bank credit to financial institutionsn.a.
Deposits relative to monetary aggregatesn.a.n.a.n.a.n.a.n.a.n.a.
Segmentation of interbank ratesn.a.n.a.n.a.n.a.
Ratio of loans to noninterbank deposits
Ratio of liquid assets to total assets
(liquidity ratios)
Maturity structure of assets and liabilitiesn.a.
Secondary market liquidityn.a.n.a.n.a.n.a.n.a.n.a.
Sensitivity to market risk indicators
Ratio of net foreign exchange exposure to
capital
Average interest repricing periods, assets and
liabilitiesn.a.
Average duration for assets and liabilitiesn.a.
Ratio of equity exposure to capitaln.a.
Ratio of commodity price exposure to capitaln.a.n.a.
Market-based indicators
Stock market pricesn.a.n.a.n.a.n.a.n.a.n.a.n.a.
Excess yieldsn.a.
Credit ratings
Sovereign yield spreadsn.a.n.a.n.a.n.a.n.a.
Notes: Types of statistical issues identified:No prudential standards. Indicates that international supervisory or regulatory standards do not exist for these MPIs or for key components of the MPIs, For example, the absence of uniform supervisory standards for provisions or accruals of income on impaired loans could result in wide variations in the meaning of MPIs employing information on the value of loans or bank profitability. Therefore, data reported by national authorities may employ different concepts or compilation methods and might be incomparable across countries.No statistical standards. Indicates that international statistical standards have not been promulgated for these MPIs or for key components of the MPIs, or that statistical equivalents to supervisory concepts have not yet been developed. For example, in the first case, there are no statistical standards on accruals of income on impaired loans. In the second case, little work has been done to date to develop standard statistical measures to capture supervisory information, even for straightforward measures such as sectoral concentration of lending.Diverse: accounting standards. Indicates that an MPI is unlikely to be comparable across countries because of diverse accounting practices in different countries.Consolidation issues. Indicates that the consolidation used by a bank can affect the meaning of the MPI. Most important, supervisory information collected from a multinational bank using a worldwide consolidation that encompasses activity throughout the world might have little relevance for the analysis of the condition of the financial sector in specific countries in which the bank operates. Consolidation issues may also arise because of variations between countries in the units consolidated within the reports (holding companies foreign trade subsidiaries).Poor data on asset quality. Refers to poor or missing information on the quality of assets, such as impairment of claims, misreporting of the effective value of claims, excessive volatility of assets, country risk, or risks from overconcentration of investments.Bank-specific information. Refers to prudential information specific to individual banks that cannot plausibly be aggregated to provide information on the overall financial sector. Qualitative in-formation. such as information on the skills and background of management, which is information typically sought by supervisors, cannot be aggregated- Moreover, information related to specific portfolios held by banks (VaR, net foreign exchange exposure) often cannot be aggregated, or must be used in conjunction with dispersion indices or measures of concentration.Derivatives issues. Refers to issues related to the recognition, valuation, or accounting treatment of derivatives and off-balance sheet instruments.
Notes: Types of statistical issues identified:No prudential standards. Indicates that international supervisory or regulatory standards do not exist for these MPIs or for key components of the MPIs, For example, the absence of uniform supervisory standards for provisions or accruals of income on impaired loans could result in wide variations in the meaning of MPIs employing information on the value of loans or bank profitability. Therefore, data reported by national authorities may employ different concepts or compilation methods and might be incomparable across countries.No statistical standards. Indicates that international statistical standards have not been promulgated for these MPIs or for key components of the MPIs, or that statistical equivalents to supervisory concepts have not yet been developed. For example, in the first case, there are no statistical standards on accruals of income on impaired loans. In the second case, little work has been done to date to develop standard statistical measures to capture supervisory information, even for straightforward measures such as sectoral concentration of lending.Diverse: accounting standards. Indicates that an MPI is unlikely to be comparable across countries because of diverse accounting practices in different countries.Consolidation issues. Indicates that the consolidation used by a bank can affect the meaning of the MPI. Most important, supervisory information collected from a multinational bank using a worldwide consolidation that encompasses activity throughout the world might have little relevance for the analysis of the condition of the financial sector in specific countries in which the bank operates. Consolidation issues may also arise because of variations between countries in the units consolidated within the reports (holding companies foreign trade subsidiaries).Poor data on asset quality. Refers to poor or missing information on the quality of assets, such as impairment of claims, misreporting of the effective value of claims, excessive volatility of assets, country risk, or risks from overconcentration of investments.Bank-specific information. Refers to prudential information specific to individual banks that cannot plausibly be aggregated to provide information on the overall financial sector. Qualitative in-formation. such as information on the skills and background of management, which is information typically sought by supervisors, cannot be aggregated- Moreover, information related to specific portfolios held by banks (VaR, net foreign exchange exposure) often cannot be aggregated, or must be used in conjunction with dispersion indices or measures of concentration.Derivatives issues. Refers to issues related to the recognition, valuation, or accounting treatment of derivatives and off-balance sheet instruments.

Absence or Diversity of Standards

The usefulness of MPIs for surveillance and public disclosure is hindered by incomparability across countries because of a lack of international standards, highly diverse national standards, failure of standards to keep up with rapid innovation in financial markets, or failure to adhere to applicable prudential or accounting standards. In the cases of supervisory and accounting standards, there may be no applicable international standards, or highly diverse national standards may exist. Also, existing accounting standards in many countries often apply historical valuations to claims and liabilities, which can disguise changes in corporations’ financial conditions. Little or no work has been done to date to develop statistical formulas and definitions for most of the proposed MPIs.

Poor Data on Asset Quality

Poor information on asset quality and on the holders of weak credits impairs the analysis of risks lacing the financial sector by reducing the usefulness of balance sheet data for making assessments of the conditions of financial institutions. These data limitations often can hide the buildup of systemic financial sector problems. Specific data limitations include lack of complete or realistic information on the full recoverable value of loans and securities, country risk, foreign exchange risk, exposures by counterparties, and the nettability of claims.77

Use of National Versus Global Consolidations

Much supervisory data is collected using a global consolidation that incorporates the worldwide activity of a bank into a single financial statement, which guarantees that all of its relevant activity is captured. Such data, however, might relate only loosely to financial conditions within any specific country in which a multinational firm operates, and much of the reported data may refer to activity or financial positions outside national authorities’ jurisdictions and policy control. In contrast, standard macroeconomic statistics use a national consolidation, and therefore exclude affiliated units in other countries.78 National financial statistics can be related to the other national macroeconomic statistics, such as GDP or national interest rates, and cover national financial activity that will be under the influence of national policy officials.

The use of the two different consolidations can have important implications for the construction of MPIs, For example, a global risk-based capital ratio is relevant for the supervision of a bank operating in multiple countries, but it is not possible to aggregate meaningfully global ratios for all banks operating in a country. This implies that there is a need to collect separate data for institutions’ domestic activity and their global activity. Such a separation is straightforward for some assets and liabilities, such as loans and deposits, but for other items there may be difficulties such as uncertainty over the allocation to individual national branches of capital items registered at the level of the global corporation.79

The scope of MPIs in different countries can also differ significantly depending on the precise collection of units drawn within the consolidated reports. This scope, in turn, depends on factors such as national legal definitions, the scope of activities permitted by banks, and rules on consolidation of subsidiaries and branches. Moreover, a related statistical coverage problem is that rapid change in financial markets can result in growth of new financial industries that might not be captured within existing supervisory or statistical reporting systems. A particular concern is that supervisory or statistical systems may fail to encompass all financial activities that might involve significant systemic risks (e.g., hedge funds and other mutual funds, consumer finance companies, trust funds, securities clearing systems).

Derivatives and Off-Balance Sheet Positions

Financial derivatives and off-balance sheet positions present special problems in evaluating the condition of financial institutions, because of the lack of reporting of positions, high volatility, and potentially large positions. Such concerns have led the accounting profession to move toward explicit recognition of virtually all derivatives on balance sheets using a market value or equivalent measure of value (fair value). International statistical standards for recognition and valuation of derivatives have also been developed, largely based on work at the IMF. These standards are now just beginning to be implemented, mostly in the context of the Economic and Monetary Union (EMU) monetary statistics and the international reserves template. The Basel Committee on Banking Supervision, of BIS, and IOSCO have also proposed new standards for the recognition, valuation, and disclosure of information on derivatives.80 Increased recognition of most derivatives on balance sheets at fair value, which is in line with most new regulatory proposals, will affect many of the proposed MPIs.

Options for Further Development of MPIs

A precondition for further work on aggregation of prudential information for individual hanks is ascertaining through surveys or other means the feasibility (given national legal and supervisory practices and statistical operations) of collecting data for the various types of MPIs that have been proposed. Because of the diversity in national supervisory practices and philosophies of supervision, the types of prudential data collected by national central banks and national supervisory offices are not well known. The IMF is therefore in the process of carrying out a survey of national authorities and users of MPIs to ascertain what types of MPIs they need, whether prudential statistics are compiled systematically for individual banks or are available as aggregates, the types of data covered, gaps in coverage, and the accounting, legal, and institutional standards that affect compilation of the data. National practices and regulations related to public disclosure are also being assessed. An important aspect of the survey is to gather information and ascertain the feasibility of constructing a core set of indicators or whether different sets of MPIs are required for different types of economies—such as financial centers, other industrial economies, emerging market economies, and developing economies.

The survey and technical reviews are aimed at gaining a clear understanding of what is involved in compiling or disseminating MPIs. For example, it might be found that a significant number of MPIs are inherently microeconomic in nature and cannot be meaningfully aggregated. Moreover, new MPIs might be proposed and the priorities in the formulation of international standards might change.81

Another important element in developing MPIs is to consider them in the context of the rapid changes in perspectives and standards of supervisors, accountants, and the public. Many initiatives are under way to develop standards that might bring about greater coherence and enhance the quality of MPIs,82 Important changes in standards are now taking place and others are forthcoming in a process that may take considerable time to approach completion. As standards are developed, national practices will gradually come into line, which should enhance reporting within each country and improve the international comparability of data. Moreover, to the extent that standard-setting organizations come to agreement among themselves (including on the adoption of applicable international statistical standards), the results will be greater coherence in compiled data, better understanding by the public, improved statistical support for the development of policy, and reductions in respondents’ and compilers’ costs of compiling data. Substantial differences across countries will continue for some time though, which requires an approach that works in parallel, both for greater future harmonization of data, but that also proceeds now on the basis of available, un-harmonized data.

In summary, additional information gathering and technical research is needed before we come to a decision point on the statistical strategies to follow in developing MPIs. Depending on options selected for developing MPIs, major resource and prioritization issues as well as organizational or legal issues could confront international organizations and national entities. Some types of MPIs may prove difficult and costly to compile, or may require new data collection systems that do not fit easily into existing statistical arrangements. Conversely, much of the work of upgrading statistical systems to encompass MPIs dovetails with the ongoing work at the IMF and elsewhere to enhance statistical, accounting, auditing, and supervisory systems to keep pace with globalization and rapid changes in financial markets,83 and thus might be viewed as incremental initiatives to work already under way.

Following is a list of some of the statistical options available for compiling MPIs, The specific strategy for following these options will depend greatly on the willingness, technical strengths, and resources of the various international and national entities that might be involved.

  • (1) Monetary statistics could be augmented with specific types of data used for macroprudential analysis. The additional data sought would consist mostly of balance sheet information, but might also include information on financial institutions’ income, expenses, and profitability. Under this option, the IMF would augment its existing system for compiling monetary statistics and use it as a basis for compiling MPIs across a range of countries.84
  • (2) A new monthly or quarterly compilation of financial sector prudential data could be instituted, covering all MPIs (should a decision be made not to use option 1), or covering only those MPIs that are not readily included will in a monetary statistics framework. The lead role in such work could be taken by the IMF or other international organizations.
  • (3) National entities could be encouraged to compile and disseminate unharmonized national data on the condition of individual banks or aggregations of microprudential data.
  • (4) National entities could enhance their programs to compile financial macrostatistics, especially sectoral balance sheets and flow of funds accounts, to support macroprudeniial analysis. These accounts are tools to assess the financial strength or vulnerabilities of the major sectors of an economy and the potential for transmission of financial stress between sectors.
  • (5) Modalities for monitoring or contributing to ongoing work to develop international standards could be explored. One possibility would be to convene an interdisciplinary working group that would follow proposals for accounting, auditing, supervisory, and statistical standards as well as changes in disclosure requirements for financial institutions, and that would support harmonization with international statistical standards.
  • (6) A handbook or manual on statistical compilation of MPIs could be prepared to provide guidance to compilers and to assist users in analyzing MPIs.

    Other Resources Citing This Publication