The Rise of Conjectural Risk Management

Scenario analysis has taken center stage in the 15 years since the global financial crisis (GFC). It’s now not only a core component of forecasting for expected credit losses but is also at the heart of regulatory and internal bank stress tests across the globe.

Indeed, I would argue that the most consequential change in the practice of risk management over the past decade-and-a-half has been the rise of scenario-based stress testing. But just how effective is conjectural scenario analysis as a risk management tool?

Bank failures remain a recurring problem, after all. So, are risk managers truly better prepared now to project financial disasters – particularly with respect to extremely rare and unprecedented events?

Tony Hughes

We’ll attempt to answer these important questions in a minute, but let’s first consider how we’ve arrived at this stage.

The GFC and Its Aftermath

Recently, I was reflecting on the GFC, trying to pinpoint the ways risk management in banks has changed over subsequent years. The cohort at the center of those events, now drifting toward retirement, received – and rejected – regular warnings about problems in the housing market.

"House price declines are always limited to local markets,” was one popular response to the warnings. Some bankers also cited the geographic diversity of their portfolios, while emphasizing that “mortgage crises never trigger national or global recessions."

We know how that all worked out.

Efforts to shore up confidence in the banking system actually began as the GFC was still raging, and then morphed into the regular capital adequacy tests now applied by regulators around the world.

The technique was considered so effective that international accounting bodies adopted it for the calculation of expected losses under IFRS 9 and CECL. Regulators have also used the approach to assess climate risk and, in 2023, new scenarios were proposed to assess liquidity in the wake of the failures of Silicon Valley Bank and several of its peers.

Before the GFC, in response to rumors and fears, the typical banker would reject the concerns of risk managers, unless they were backed by historical precedent. Now, with scenario analysis, they have an avenue through which they can be seen to be taking action.

Whether engaging in such an activity would have made a difference to a pre-GFC subprime mortgage executive, though, is a difficult question to answer.

At the time, regulators would have had a hard time justifying any action to curtail the industry’s behavior, especially because the Bush Administration was – admirably if not prudently – trying to boost the rate of home ownership. Moreover, the banks themselves would not have acted on the results of such a stress test, because they would have lost market share to competitors and missed out on what seemed likely at the time to be a big payday.

This may have been different if the fears were tangible. If the politicians, regulators and bankers had access to clear historical precedents, and thus built their stress scenarios using solid empirical data, the justification to act would have been much greater and the crisis may have been averted.

Of course, history is also replete with decision-makers failing to heed the lessons learned by their forebearers.

The Inefficacy of “Forward-Looking” Scenarios

Banks are now encouraged to use conjectural scenarios for idiosyncratic stress testing exercises. Undoubtedly, in the future, there will be a new headline or development that can plausibly be viewed as a threat to banks’ business – and scenario analysis will be used to explore its implications.

The prevailing premise is that forward-looking scenario analysis can spark boardroom discussion, even if it doesn’t rely on empirical data. In short, the belief is that having risk managers quantify the effects of a threat – however shakily – will make an organization more resilient. But if the exercises make no difference to the bank's strategy or the way it is managed, is this really the case?

Obviously, if the stress test prompts the bank to increase its capital holdings, it will boost resilience. Likewise, analyzing data from past stressful events will enable related risks to be subsequently better managed. A hypothetical bank dipping its toe in subprime mortgages in 2024, for example, would presumably make a much better list of things than they did the first time around.

But these are not examples of conjectural risk management – the type of scenario analysis that is so popular today.

To imagine how this could work in practice, suppose that a risk manager identifies a novel threat to an erstwhile profitable part of a bank's portfolio. The top brass do not know it yet, but the threat, while seemingly remote, could one day sink the entire business.

In such a situation, what could the risk manager say or do to convince the bosses that the portfolio is dangerous and that they should change course? Would a scenario analysis help, bearing in mind that, by necessity, it cannot be based on empirical modeling and instead must be speculative or theoretical in nature?

In this simple example, the bosses are informed of the threat, have discussed it, and been given access to an accurate stress scenario. Given the tenets of conjectural risk management, let’s also assume that the bank’s procedures, to date, have been impeccable – i.e., they have taken all the appropriate steps to fortify the business against the threat.

Addressing the concern in real time would therefore involve shutting down or redirecting a profitable business based on little more than a hunch. Even if the threat has disaster potential, a hunch should not be enough to convince business heads to meaningfully reshape the bank's profit-oriented strategy.

There have been situations, of course, where organizations, including banks, have responded to conjectural threats and successfully made their operations more robust. But these are normally related to important functions that can be altered without disrupting the core mission of the business. Fortifying IT systems against cyber threats and training staff to cope with operational disruptions – like pandemics – are obvious examples.

Parting Thoughts

In the early years after the GFC, the development of scenario analysis was not yet in its conjectural phase. The focus of the industry was on improving data gathering processes and building modeling capability to meet the new expectations of regulators. The practice of scenario modeling was focused on replicating the contours of the GFC, which was obviously grounded in observable history.

However, as the years rolled by, scenario analysis was asked to do more and more heavy lifting. People became less concerned with managing the last recession and wanted to focus instead on potential causes of the next crisis. Scenario analysis was already being used, so regulators and risk managers repurposed it, even though they were moving further away from any solid empirical underpinnings.

In 2024, large sections of the financial services industry have considerable faith in scenarios. Ask them why and they tell you that they help spark discussion in the boardroom and that they make managers think about threats that otherwise wouldn’t be recognized.

But it’s difficult to imagine that these perceived “benefits” bolster the resiliency of financial institutions in any material way. Just consider the bank failures we’ve seen in the 15 years since the GFC.

There simply must be a better way to project financial disasters, to forecast expected losses and to improve the overall practice of risk management.

Tony Hughes is an expert risk modeler. He has more than 20 years of experience as a senior risk professional in North America, Europe and Australia, specializing in model risk management, model build/validation and quantitative climate risk solutions. He writes regularly on climate-related risk management issues at UnpackingClimateRisk.com.