A Code of Practice for Supervisory Stress Tests
BY JO PAISLEY AND MARK CAREY
President and Former Co-President of the GARP Risk Institute
December 2018
Menu
The Basel Committee on Banking Supervision’s Stress Testing Principles set the standards to which banks and supervisors should aspire. As the BCBS notes, stress testing is ‘now a critical element of risk management for banks and a core tool for banking supervisors and macroprudential authorities.’ Certainly, stress testing has immense value for both banks and supervisors.
For banks, it provides critical insights into their vulnerabilities, and underpins robust capital, liquidity and business planning. What’s more, it provides boards with insight on their firms’ ability to withstand severe scenarios and the range of management actions that might need to be taken to build resilience.
For supervisors, stress tests provide a rich source of quantitative and qualitative insights, such as assessing firms’ vulnerability to macro-economic scenarios and the quality of their capital and liquidity planning processes.
Supervisory stress testing came to the fore in the financial crisis, where tests based on common scenarios across a range of banks – so-called ‘concurrent’ stress tests – helped to calm market turbulence, providing much-needed, forward-looking information on the health of banks’ balance sheets. Indeed, these tests have become tools to set regulatory capital requirements for individual banks, and can be used not only to assess system-wide resilience but also the efficacy of banks’ recovery and resolution plans.
In recent years, as documented in the Basel Committee’s paper, Supervisory and bank stress testing: range of practices, the tests have both intensified and proliferated. Yet, there is a danger that the uncoordinated proliferation of supervisory stress tests makes the Basel Principles themselves less achievable, leading to many undesirable consequences. Today, for example, it is difficult for either supervisors or analysts to compare the results of these stress tests, making it harder to share results meaningfully and leading to potential misinterpretation.
Moreover, the current resource-intensive system also (1) requires duplication of effort across jurisdictions for supervisors; (2) leads to higher costs and a lower quality of outputs for both supervisors and banks; (3) risks making stress testing within the banks a compliance exercise; (4) will likely ‘crowd out’ stress testing that is used in firms’ internal risk management; and (5) makes it harder for international banks to achieve economies of scale in the production of stress test outputs.
Greater harmonization across the tests would aid the exchange of information between supervisors, as the tests would be more directly comparable; give banks the ability to produce the information more efficiently, improving the quality of the outputs and encouraging greater investment in strategic IT infrastructure; and help investors and analysts who need to interpret the results of different stress tests. (The last benefit would be particularly helpful in cases where there are stress tests on different parts of an international bank.)
Although Basel Principle 9 encourages communication, coordination and harmonization of stress tests across jurisdictions, from a practical point of view there is, as yet, no international agreement on how to achieve this. This Code of Practice provides recommendations for plugging this gap. It sets out a framework for thinking about how stress tests are designed and executed and offers an initial set of global guidelines for supervisors to consider.
It will take work to analyze different approaches and to work out the most effective candidates (e.g., data definitions or output templates) for harmonization. But the work is necessary if we are serious about achieving the Basel Principles in full, across all banks in all jurisdictions.
The GARP Risk Institute authored this paper, with input from practitioners, to start a dialogue.1 GRI would welcome the opportunity to work with regulators and banks to help improve the design and execution of stress testing across the globe. We welcome feedback and ideas from practitioners and supervisors.
Our Code incorporates the Basel Principles within a framework for harmonization. Each section provides relevant commentary. Cross reference is made to the relevant Basel Principles, where appropriate.
It is neither plausible nor necessary for every aspect of supervisory stress testing to be harmonized. But there are elements where harmonization could prove both useful and feasible, for both supervisors and banks. Achieving this requires a practical ‘decomposition’ of stress testing into an organizing framework (see Figure 1, below).
Figure 1: Anatomy of a Stress Testing Framework
The three pillars in this organizing framework are: Goals, Global Guidelines and Process Components.
The level of harmonization achievable will be limited by the needs of individual regulators to satisfy the goals of individual stress testing exercises. Falling in line with previously established international practices (see Stress Testing: Where Next), our Code provides indications of how the process should be administered, so that the needs of supervisors are balanced against those of the participating banks.
Supervisory stress testing exercises today have many different goals, with some exercises having multiple objectives. The most fundamental distinction is micro and macroprudential supervision, focusing, respectively, on the financial health of the individual institutions or the financial system (in a country or region). Many exercises, particularly the large concurrent exercises, serve both purposes.
According to the Basel Range of Practices paper, microprudential supervisory stress test results are primarily used by supervisory authorities for reviewing and validating the Internal Capital Adequacy Assessment Process of banks and their liquidity adequacy assessments. Supervisors can use the results to set capital requirements in a wide variety of ways – e.g., by setting capital add-ons or assessing the quality of a bank’s capital planning processes.
Macroprudential stress tests focus on the resilience of banking systems. When assessing the interaction between firms, supervisors will tend to use ‘top-down’ modelling approaches to capture any feedback loops, amplification mechanisms and spillovers.
Stress tests can also be used to calibrate macroprudential measures and supervisory policy changes. Moreover, they might be employed for risk identification purposes, such as providing early warning indicators of potential weaknesses of the financial system.
Setting out the goals clearly is consistent with the first Basel Principle, which states that stress testing frameworks should have clearly articulated and formally-adopted objectives. This principle also notes that the authorities should communicate the intended objectives/use of stress testing results to participating banks in advance of the exercise. Beyond this, it would be helpful for authorities to publish the high-level objectives of their stress testing frameworks, seeking to answer the following questions:
Basel Principle 8, which states that authorities should regularly review whether stress tests conform to their stated objectives and governance arrangements, should also be considered.
To encourage greater coordination and, ideally, harmonization between the authorities, all supervisory stress tests – irrespective of their goals, precise design or jurisdiction – should follow some basic internationally agreed global guidelines.
Supervisory colleges should be used as a means of sharing stress testing information across jurisdictions. This includes information about methodologies, insight from the stress tests and any other information that helps host countries better understand the quality of the stress testing.
Supervisors may decide to also disclose details of their approach to stress testing, as well as the results, to the public. The results might be at an individual institution level or at an aggregated level. The nature of any public disclosure will be a matter for individual regulators to decide, given the goals of their specific regimes.
Host supervision of branches poses a particular problem, but should be handled by normal home-host arrangements – e.g., through sharing information and supervisory colleges. Home supervisors should be able to share appropriate group-wide stress testing results with host supervisors of branches, subject to legal provisions and the need to respect confidentiality.
Ideally, supervisors should announce – either publicly or privately – a schedule for supervisory or system-wide stress tests. Supervisors should routinely exchange information on stress test plans and results through supervisory colleges. When circumstances require plans to change, revisions to the timetable should be announced, together with a clear explanation.
Supervisors should try to coordinate their timetables as much as possible. For example, this might mean that supervisors coordinate amongst themselves and try to run tests at different times to avoid the ‘peak load’ problem at the banks. Alternatively, they may use common templates and methodologies across jurisdictions. Irrespective of which approach is taken, supervisors should consider the appropriate balance between internal and supervisory stress tests.
Both banks and regulators need to plan for the execution of stress tests, given that these exercises are resource intensive and take time to execute. Sufficient notice should be given for a major enterprise-wide stress test.
This does not necessarily mean that the scenario needs to be shared with the banks at the time notice is given. But banks should be aware of the requirements (for example, template structure/documentation requests) to allow them to plan resourcing. The notice period required will depend on the scale of the exercise, as well as the granularity and complexity of the exercise.
Supervisors should require robust internal stress testing regimes; indeed, Basel Principle 3 states that stress testing should be used as a risk management tool and to inform business decisions. To ensure that supervisory-initiated exercises do not crowd out internally-driven exercises, supervisors should consider the banks’ need to allocate resources to such internal work when calibrating their own demands.
Banks and supervisors should carefully balance the workload involved and the value gained from the stress testing exercises, so that they do not “over-engineer” the tests. Supervisors should also be attentive to the demands posed by multiple, closely related stress tests such as ICAAPs, concurrent and even internal stress tests. Where possible, they should aim to exploit insights from different exercises for the benefit of the others.
Supervisors should also consider the adequacy of their own resources to administer the exercises, as set out in Basel Principle 5. For example, they should evaluate the quantum and quality of resources needed to oversee stress testing exercises. This includes interacting with the banks that participate in exercises – e.g., having the appropriate processes and infrastructure to address clarifying questions from banks, to check data quality/discrepancies and to provide feedback to the banks on the results of the exercises.
The resources available, especially at the authorities’ level, should inform the frequency and complexity of the supervisory stress tests to avoid compromising the quality of the supervisory exercise and imposing unnecessary burdens on participants.
The importance of portfolios and risks varies across banks, and the resource requirements of stress tests for each portfolio and risk should vary with importance. Requirements for use of the most sophisticated models and methodologies, and for very granular decompositions to support greater precision, should be reserved for portfolios and risks material enough to have a non-negligible impact on overall results. Model risk management intensity should also be similarly proportionate. Please see Model Risk Management at the Crossroads: Meeting New Demands with Limited Resources.
For other portfolios, simplified methods and data should be permitted. Freeing up bank and supervisory resources to focus on material matters would improve the overall quality of results.
Many ways to accomplish such a division can be imagined, but one method would involve an initial separation of portfolios and risks into material and immaterial categories, with the burden on the bank to convincingly demonstrate immateriality. Supervisors interested in a portfolio or risk for reasons unrelated to the goals of the stress test would remain free to conduct focused reviews, but outside the perimeter of the stress test.
Basel Principle 4 states that the scenarios should be comprehensive, relevant and sufficiently severe. But, at present, there is no international standard for scenario choice for use in capital (or liquidity) stress testing by supervisors. Since the choice of scenario has a determining effect on the level of capital that banks are required to hold, this allows the possibility of divergences in capitalization levels from one regulator to another.
Individual supervisors typically choose a severely adverse scenario for their home country or region. For global banks, the likelihood that all countries and regions will experience a similarly bad scenario is much less than the likelihood that such a scenario will occur in the home country or region. Choosing an equally severe scenario in each region effectively makes it extremely unlikely that the global combination of such scenarios would occur, and therefore holds global banks to a stricter standard than less-diversified local or regional banks.
This section of the Code examines the primary components in a supervisory stress test process, highlighting features that are especially appealing candidates for greater harmonization. The five primary components are (1) key assumptions and methodology; (2) input templates; (3) scenarios; (4) output templates; and (5) disclosures.
Driving appropriate harmonization across these components for different supervisory tests would help improve the quality, comparability and efficiency of supervisory stress testing. Judging what is ‘appropriate’ will take time and effort, but the overall aim to strive for something that is neither the ‘lowest common denominator’ nor the ‘gold standard.’
Supervisors should set out clearly any methodological guidance that they require the banks to follow. These requirements ought to be consistent with the stated goals of the exercise, as envisaged in Basel Principle 7, which states that models and methodologies should be fit for purpose.
Today, across the existing stress tests, assumptions differ about the time horizon for the scenario and how the balance sheet/income statement evolves. For example, the EBA stress test assumes a ‘static’ balance sheet over a three-year horizon. In contrast, in the Bank of England stress test, banks must follow stipulated lending paths over a five-year horizon. The US stress test, on the other hand, is based on a nine-quarter horizon and assumes that the credit supply does not fall. (We should note, though, that aspects of US approach are currently under review.)
At the very least, it would be good if there was better understanding of the importance of these underlying assumptions in driving outcomes, so that the results could be better understood by regulators, investors and the public. Developing a series of ‘guides’ that set out the pros and cons of different methodological assumptions would help supervisors when making design decisions. For example, which balance sheet assumption is most suitable for micro or macroprudential stress testing? How should adjustments be made to stress tests run under the assumption of a ‘static’ balance sheet where there are discontinued activities? What is the best way to estimate ‘constant currency’ stressed losses?
Undertaking this type of work would help inform the design of supervisory stress tests and would likely lead to greater harmonization (or at least transparency) of assumptions across supervisors. The work would also aid comparability of results, supporting the sharing of results across supervisors, as set out in Basel Principle 9.
Research into these types of methodological issues across jurisdictions would be helpful for both the authorities and participating banks.
This covers the data required to be collected and/or submitted for the purposes of running the supervisory stress tests. Several factors affect the comparability of approaches across supervisors, as well as the amount of processing and (manual) adjustments to be made to data used by banks for internal purposes. The section below looks at the main dimensions, including the structure and granularity of reporting; definitions; and treatment of accounting standards.
Structure and granularity of reporting
The level of detail for information on exposures/portfolios can vary in different exercises, ranging from counterparty-level data to more aggregated sub-portfolios (e.g., mortgages by region/country). Basel Principle 6 encourages authorities to leverage, to the extent possible, data that are already provided by banks to authorities, such as through banks’ regular supervisory reporting.
Where possible, regulators should ensure coherence between their data requests for stress testing and either other regulatory data collections or firms’ internal management information. The level of granularity required should be driven by the goals of the stress test and the materiality of the portfolio or risk being examined.
Definitions
This includes the definitions of key metrics to be used for the stress test, such as capital, risk-weighted assets, risk and product parameters.
Treatment of accounting standards
Global banks operate under different accounting standards (mostly US GAAP or IFRS) at a consolidated level. In addition, regulatory capital reporting in host countries often must follow local GAAP. Supervisors and banks should be mindful that the use of different accounting standards – not only across banks but in different parts of a banking group – makes calculations more complex and results harder to compare and interpret.
The two most relevant dimensions of scenarios are the variables chosen and the time horizon.
Scenarios and variables
The scenario in a supervisory stress test can be viewed as an articulation of the supervisor’s risk appetite and its perception of the key risks in its jurisdiction. But while it is likely that supervisors will have different views on the appropriate scenario to set, there may be scope to harmonize the way that economic scenarios are articulated – e.g., the templates that set out the variables, the specific definitions for GDP growth and unemployment rates.
Time horizon
The time horizon for a scenario will depend on the risks that are being explored, but most macroeconomic supervisory scenarios are between two and five years. This is consistent with the ICAAP rules covering internal stress testing requirements. Some stress tests also set out market or ‘traded’ risk scenarios, which differ according to the amount of detail provided and the assumptions about holding periods.
There should be a relationship between the time horizon for a stress test and the granularity of the projections. Generally, the longer the time horizon, the less granular the projections should be, given the increasing degree of uncertainty associated with projections over longer time horizons. There is likely to be an opportunity to standardize some of these time horizon aspects across supervisory stress tests.
Banks are often required to report the results of a stress test to the supervisor in a pre-defined results template. Additional data (e.g., on qualitative aspects of the stress test) may be required. In the US, banks also supply raw data to supervisors that the authorities use in their own calculations, which are then contrasted with banks’ internal results.
The templates differ across jurisdictions, particularly with respect to granularity and the various stress testing impact indicators requested – e.g., capital and liquidity metrics, non-performing loans ratios). The more that input templates and methodologies are harmonized across supervisors, the more likely that aspects of the output templates could also be harmonized.
This process addresses the question of whether supervisory stress testing results are disclosed to the public, and if so, in what form. It also covers how the results are used and how management actions are considered. Designs of disclosures should take account of Basel Principle 9, which states that stress testing practices and findings should be communicated within and across jurisdictions.
Public/private disclosure
Generally, the decision on whether to publicize the results of the stress tests will depend on the objectives of the stress tests and the benefit to credibility/improved market confidence that transparency brings. The major concurrent stress testing exercises run by the Fed (DFAST/CCAR), the Bank of England and the EBA are currently disclosed.
Use of results
The stated goals of the stress test will have a direct bearing on the appropriate nature and format of disclosures, including a decision on whether results should be made public.
Treatment of management actions
Banks can use ‘management actions,’ such as changing dividends or cutting expenditures, to offset some of the impact of the stress test.
Supervisors need to define rules around which management actions they will permit in the stress test. This might reflect their judgement about which actions could be plausibly executed in the scenario that they are testing the banks against. Some regulators split these actions into ‘business-as-usual’ actions (e.g., tightening loan covenants) and ‘strategic’ actions, which are more significant and would require board approval.
The treatment of management actions can make quite a difference to the results of a stress test. But when the treatment differs, it can make the stress test results difficult to compare. Supervisors should therefore take care to make disclosures as transparent as possible with respect to the treatment of management actions. There is likely to be an opportunity to harmonize the approaches to management actions.
Disclosure templates
Given the range of differences across supervisory stress tests, there is room for disclosures to be misinterpreted if they are not clear, comparable and as self-explanatory as possible. Harmonization of disclosures across jurisdictions is likely to improve understanding among investors, the media and wider public.
Authorities need to be cognizant of the potential implications of inappropriate disclosure of stress test exercises on banks. Any disclosure of stress test results should consider ways to ensure that market participants understand data that is disclosed, including limitations and assumptions on which it is based. The timing of disclosures by supervisors also needs careful consideration, bearing in mind banks’ own disclosure requirements (e.g., around dividends, capital levels, etc.) to ensure that stress testing disclosures do not add to market uncertainty.
Broadening disclosures by authorities to other critical pieces of information of the supervisory stress test exercise (e.g., the objectives, methodologies, scenarios and actions) would improve our understanding of the results in the market, thus helping to mitigate any unintended consequences on banks.
Stress testing has certainly come a long way since the financial crisis. Today, many regulators, including the Bank of England, the Federal Reserve and the EBA, are reviewing their approach to concurrent stress testing in the light of lessons learned over recent years. It is therefore a good opportunity to step back and reconsider the approach.
The Basel Stress Testing Principles, recently updated, remain as pertinent now as they were in 2009. But to achieve these principles, we need a framework that encourages greater coordination and harmonization across supervisors.
The approach detailed in this Code is not about weakening standards. Rather, it is about being proportionate and coherent, organizing stress testing in a way that adds meaningfully to both supervision and risk management.
The next step is for supervisors and practitioners to examine the scope for coordination and harmonization of the matters discussed herein. The GARP Risk Institute welcomes feedback on any aspects of the Code and stands ready to work with supervisors to implement it.
About the Authors
Jo Paisley, Co-President, GARP Risk Institute, served as the Global Head of Stress Testing at HSBC from 2015-17, and as a stress testing advisor at two other UK banks. As the Director of the Supervisory Risk Specialists Division at the Prudential Regulation Authority, she was also intimately involved in the design and execution of the UK’s first concurrent stress test in 2014.
Mark Carey, Former Co-President, GARP Risk Institute, worked at the Federal Reserve Board for over 25 years, during which time he sat on a senior committee that helped govern the CCAR. An expert on credit risk, he also validated one of the CCAR’s wholesale credit risk models.
1 Some of the concepts herein were inspired by an unpublished anonymous work entitled ‘Stress Testing Harmonization’ that was circulated in 2013.
•Bylaws •Code of Conduct •Privacy Notice •Terms of Use © 2024 Global Association of Risk Professionals
