Amid the pandemic, inflation, the oil crisis, and the ongoing geopolitical risk of the past two years, there has been a lot of fair criticism of both backward-looking and forward-looking approaches to quantifying risk over the longer-term horizon. The former was too reliant on historical data and the latter was unjustly driven by narrow-minded scenarios – and neither could properly account for the unprecedented market conditions.
Indeed, when it comes to strategic risk, the standard risk measures appropriate for intraday risk assessment (e.g., value-at-risk) have failed while the new measures (stress-test scenarios developed after the global financial crisis) have proven inadequate.
The reason that traditional statistical models used for scenario design fail in unprecedented market and economic conditions is two-fold: their hypotheses are biased by modelers’ experience, and the selected drivers no longer steer the outcomes. Consequently, their forecasts and tail estimates can miss by orders of magnitude.
Backward-looking models lack relevant information on current and prospective economic conditions, and can’t forecast unprecedented results. Forward-looking models, on the other hand, typically come in two forms: (1) parametric or semi-parametric distributions with statically-calibrated parameters; or (2) narrowly-focused forecasting and stress-testing scenarios. The former one fails when the correlations between risk drivers change (just as they did in recent crises), while the latter approach often does not include relevant scenarios.
Neither of these approaches answer the most critical question that firms must address today: What are the future outcomes for which we need to be prepared?
Capturing the Tails
Since it is not possible to accurately predict even the nearby future, risk managers must focus on the breadth of the future outcomes rather than trying to guess (precisely) a few specific ones. This means capturing firm-specific (realistic) tails of risk for key performance indicators (KPIs).
To capture such tails and to identify what capital and liquidity ratios or net income represent a potential danger to the firm (e.g., occurring with more than 1% probability), full-range, automatically-generated scenarios are needed.
The critical feature of this approach is that it can prepare firms for any relevant future. If you employ multiple full-range scenarios that do not target precision but cover tail risk (including all realized outcomes), it will not matter if the realized future isn’t exactly forecasted by any scenario.
Comparing and contrasting the GFC and the COVID-19 pandemic drives home this point. Before the GFC, banks had very low capital buffers (that did not properly account for tail risk), and therefore suffered major default losses as the crisis unfolded.
In contrast, when we fast-forward to the pandemic, we realize that banks were much better capitalized to weather the storm thanks to stress-test scenarios that were carefully crafted. These stress scenarios didn’t anticipate the pandemic, but still provided sufficient capital buffers that covered tail risks.
A Better Forecasting Solution
Full-range scenario analysis explores capital and liquidity needs under a wide variety of historical and unprecedented combinations of market and macro shocks. By projecting balance-sheet and income-statement segments, this approach gives a sufficiently high-level view of the full distribution of firms’ KPIs, enabling management to identify the future outcomes for which they need to be prepared.
Moreover, it addresses all the problems that are usually associated with the forward-looking scenarios. How do you know, though, that all of your bases are covered? By backtesting using the full-range, multi-scenario approach. It is true that all models use historical data, but if that information is analyzed in a special way, it is possible to generate historically-unprecedented outcomes.
Historical data must be separated into two major categories: (1) stable periods with fairly static distribution parameters; and (2) shocks that change correlations, volatilities and other behavioral patterns. The first category can be calibrated with standard diffusion modeling techniques, while the second can be represented by Poisson processes with probabilities and severities of the shocks estimated by explainable machine-learning (ML) methods. (Black boxes should never be used!)
When such shocks are applied to the scenarios, they interrupt stable behavior and implicitly change volatilities of scenario variables – as well as the correlations between them.
This approach of using historical data to generate forward-looking synthetic data also answers two other important questions: (1) How can a new data point enhance scenario projection; and (2) What should we do with pandemic data that skews all previous predictions?
If a new data point emerges amid stable market conditions, it will not significantly change the parameters of the scenario-generation algorithm. But if it’s a game-changing data point, it can be used to calibrate shock impacts – and might produce different tail risk results.
During the pandemic, for example, despite record-setting unemployment, we didn’t see a major increase in credit losses, partly because of a new shock (one of those game-changing data points) that interfered: government support. This doesn’t mean, of course, that these two shocks will always come in a pair in the future. Indeed, a responsible risk manager can’t rely on such happenings.
In the wake of a crisis-driven aftershock, firms must be ready for radically-different scenarios, including one in which there is a quick recovery and another that forecasts a much slower bounce-back with much more severe consequences.
The new data points that have emerged amid the pandemic have provided fresh parameters – not only for improving scenario generation but also for calibrating the potential impacts of future crises.
When projecting the behavior patterns of balance-sheet and income-statement segments – like loan volumes, deposit runoffs or fees from various business activities – one cannot rely on a limited set of risk drivers.
But one can get past these limitations by using explainable ML methods (like regression with regularization) that can: (1) fit the outliers without overfitting; (2) perform exhaustive cross validation to better model an unprecedented future; and (3) select a few key variables, out of an unlimited set of potential candidates, that explain behavioral patterns.
In short, if the new data points continue the historical trend, the selected set of the variables will likely remain unchanged. On the other hand, if a considered forward-looking scenario is drastically different from historical trends, the variables driving KPIs calculation will no longer work. In the latter case, the model will select new drivers that explain the new dynamics.
These “driver updates” have to be performed at least quarterly or upon arrival of a new shock, as a supplement to traditional (annual) stress tests. A firm that employs quarterly forecasting updates (or amid each new shock) can adjust its selection of the key risk factors, which should help with improving the quality of forward-looking scenarios and with discovering hidden risk concentrations.
Scenario-based risk modeling that is both transparent and intuitive can help banks proactively discover the future scenarios for which they need to be prepared. Moreover, it empowers banks to develop contingency plans that help them act before it’s too late.
The future is uncertain. Even thousands of generated scenarios might not contain the one that is actually unfolding. However, estimating the full distribution of outcomes will help you face the uncertainty with much more confidence.
Alla Gil is co-founder and CEO of Straterix, which provides unique scenario tools for strategic planning and risk management. Prior to forming Straterix, Gil was the global head of Strategic Advisory at Goldman Sachs, Citigroup, and Nomura, where she advised financial institutions and corporations on stress testing, economic capital, ALM, long-term risk projections and optimal capital allocation.