Risk Management and Radical Uncertainty

Having lived through a financial crisis in the late 2000s and a global pandemic in the early 2020s, and with the threat of global warming and cybercrime ever present, risk managers might be excused for feeling overwhelmed.

Over the next decade, we can all imagine a multitude of events – both disastrous and uplifting – that may confront us. There is also a longer list of unimaginable potential happenings that we can’t begin to envision.

In this context, the concept of radical uncertainty, introduced in a 2020 book by John Kay and Mervyn King, has garnered a lot of attention in the industry. This was piqued following the pandemic – perhaps the quintessential example of such uncertainty – and the difficulties created by inflation and rising rates in the post-pandemic economy. 

The core critique of Kay and King is that trying to understand the world through the lens of probability distributions is fundamentally hampered by the existence of radical uncertainty – events to which it is impossible to assign probabilities. In Kay and King’s estimation, these situations abound; under such circumstances, they elaborate, our statistical model building exercises are doomed to fail.

Here I want to provide a defense of traditional model building while conceding that the criticism of the probability-based method is generally sound. I’d also like to highlight a few situations where I have seen the concept of radical uncertainty used and abused in the industry.

 The Continuing Importance of Historical Data

So, how should we think about radical uncertainty in the context of credit modeling? Our profession is dominated by the type of probabilistic behavioral models skewered by Kay and King. There is no question that the borrowers we model face a constant barrage of radically uncertain possibilities.

Let’s consider a specific uncertain event – a globally significant cyberattack – and its potential impact on a particular company’s probability of default (PD). We can obviously theorize that, should the attack occur, the company’s chance of survival might be affected. But we don’t know how, by how much or even whether the attack will be advantageous or detrimental to the company in question.

What we do know – or at least what we can statistically determine – is how the company, and others of its ilk, have performed across the wide range of observed historical conditions. In other words, if history is any guide, the model can provide us with a reasonable prediction of the company’s PD during a period of generic stress.

There’s a famous quip: history never repeats, but it often rhymes.

My point here is that a well-built model that accurately summarizes the lessons of history will often provide useful, accurate predictions for managers making tough, coalface decisions. Even the best model, though, is fundamentally misspecified, because the future is inherently uncertain.

In situations where history does not provide clues, we should expect our models to fail. A key lesson of radical uncertainty is that risk managers need to have a strong sense of fatalism; no matter how much we prepare, the future will still surprise us.

On a positive note, once something has happened (like a pandemic, for example), we can incorporate what we learn in our models, making future uncertainty far less radical.

Radical Uncertainty and Scenarios: A Match Made in Heaven?

One of Kay and King’s recommendations is to use more narrative-based methods in the face of radical uncertainty. Humans are social animals; we often use relatable stories to help us navigate situations that are dangerous and unprecedented. In the context of banking, one can readily imagine a water cooler discussion of possible responses to all sorts of crazy future events.

Bank stress testing efforts have also adopted a narrative-based methodology, via the scenarios that have become ubiquitous in the industry since the GFC. It would seem, on its face, that narrative scenario analysis is perfectly suited to a consideration of radically uncertain events, since it is already used by banks.

Tony Hughes

Is this perception accurate? Well, yes and no.

In recent months, I’ve seen a couple of cases where people have stated that the situation is radically uncertain and then proceeded to produce seemingly precise, statistical portfolio projections under the stated scenario. I’ve also seen one forecaster criticize another forecaster’s forecast on the basis of radical uncertainty.

It goes without saying that precise model predictions are either available (because the world is amenable to statistical exploration) or they are not because the world is radically uncertain. You can’t have it both ways.

Risk modelers should never project a false impression of precision. If they know or suspect that something is unmodelable, they should clearly state this to the end user.

I don’t mind the notion of informal scenarios where managers imagine directional responses to unusual events with the aid of narratives. When scenario projections are presented as precise, though, the level of precision must be demonstrated. This is extremely difficult to do in practice.

Recent Radical Uncertainty: What Worked and What Didn’t?

So, the world is radically uncertain. Time for a beer?

Truthfully, it’s often difficult to work out a coherent game plan with these types of concepts. But we’ve just lived through the pandemic – a perfect example of an unpredictable future event.

Now, looking back, can we identify elements of risk management that performed well during COVID-19 and those that performed poorly? What we’re trying to do is identify strategies that may be suitable in dealing with future radically-uncertain events.

In my view, the most successful risk management activity was the work-from-home drills and related IT arrangements that many companies undertook in the years prior to the pandemic. I remember thinking that these exercises were disruptive in real time but, with 20-20 hindsight, they now look to be exceptionally prescient.

A key observation is that these preparations would have been useful in any scenario requiring a sudden decentralization of the workforce. They worked very well during COVID, but they were not specific to it in any meaningful sense. 

The least successful tool, in my opinion, was the unnecessary increase in loan loss reserves prompted by the introduction of CECL and IFRS 9 financial accounting rules. As soon as it was known that a pandemic would occur, the rules required banks to predict the pandemic’s effect on their loan portfolios. They then wrongly projected that loan losses would build quickly, creating a significant amount of unnecessary volatility in bank financial statements.

The concept of radical uncertainty undermines the foundations on which CECL and IFRS 9 are built, because the notion that risk analysts can provide useful forward-looking information during periods of crisis is fundamentally misplaced. Indeed, methodologies like CECL and IFRS 9 cannot work as intended when the reasonable and supportable forecast horizon falls to zero.

Parting Thoughts

We are risk managers, not uncertainty managers. In the traditional parlance, this suggests that the risk manager should stick to situations where probabilities can be objectively assigned. The requirements of running complex businesses, however, means that we must consider uncertainties – even radical ones – within our broader remit.

But we should be realistic about the amount of control we exert over radically uncertain outcomes. Our aim should be to identify strategies that can cover a range of possibilities, rather than to focus on specific circumstances that are inherently unmodelable.

Finally, we should realize that history often provides a valuable guide for decision-making. Traditional models are still useful, even if they are incapable of dealing with radical uncertainty.

Tony Hughes is an expert risk modeler. He has more than 20 years of experience as a senior risk professional in North America, Europe and Australia, specializing in model risk management, model build/validation and quantitative climate risk solutions.