Traditional vs. Fixed Analysis: Weighing the Pros and Cons of An Alternative Approach to Scenario Design

Imagine a world where risk modelers turn conventional wisdom upside down and use fixed economic scenarios with dynamic probabilities to calculate expected credit losses. This unconventional approach challenges the traditional method of estimating credit risk, which typically involves considering multiple economic scenarios based on fixed percentiles.

Given the large impact that scenario design can have on risk estimates, let’s now consider the strengths and disadvantages of these two approaches.

Cristian deRitis

Scenario design is a crucial aspect of risk analysis, shaping numerical outcomes and facilitating – or inhibiting – effective communication with stakeholders. While complex simulations involving thousands of potential economic paths may offer more accurate risk assessments, they often require extensive computational time, calibration and maintenance.

In fast-paced environments, analysts often don’t have the luxury of time or resources. Therefore, selecting the optimal set of scenarios to use becomes essential, considering factors such as accuracy, interpretability, reasonability and ease of communication.

Why Use Multiple Scenarios?

The economy is an ever-changing landscape, influenced by a multitude of factors such as weather, technology and consumer behavior. Predicting the future with certainty is impossible, but that doesn't mean we can't make more informed choices.

By appreciating upside and downside risks and quantifying their probability distributions, we can better manage uncertainty and make more accurate forecasts of losses, revenues and other financial metrics compared to relying on a single scenario.

Fixed Probabilities: A Typical Approach

While considering a multitude of economic scenarios or simulations may be ideal, it can be overly complex and time-consuming for day-to-day risk management tasks. Instead, a more common analysis involves estimating outcomes under a baseline scenario (50th percentile), supplemented by one upside (10th percentile) and one downside (90th percentile) scenario. Although the scenario percentiles are fixed, the shape and severity of the individual scenarios will change with each update to capture the current state of the business cycle.

Figure 1: Traditional Multi-Scenario Analysis with Set Probabilities

One advantage of this approach is that it allows us to consider the same relative risks consistently over time. For example, when setting a loss allowance or reserve for a portfolio, it may be beneficial for auditors and shareholders to review estimates under the same set of relative risks each quarter. This consistency facilitates comparison across reporting periods to assess the reasonability of the estimates.

However, a drawback of this approach is that it requires analysts to consider a new set of scenarios each quarter. As the economy evolves, the severity of the downside scenario may change, affecting the interpretation of any calculations based on these scenarios. For example, the 90^th percentile downside scenario may have the unemployment rate peaking at 8% in one quarter but peaking at 10% a few quarters out, as the economy worsens or other shocks hit the economy.

Fixed Scenarios: An Alternative Approach

Alternatively, we could develop a set of scenarios where the severity and definition of each scenario are held constant, but the probabilities are allowed to fluctuate. For instance, we could create a family of scenarios with varying peak unemployment rates (such as 4%, 6%, 8%, 10%, and 15%), and we could engineer proportional declines in household consumption and business investment to hit each of these desired targets. The transition from current conditions to each peak would follow a predetermined path over a few quarters.

Figure 2: Consistent Scenarios with Adjustable Probabilities

We would need to attach probabilities to each of these scenarios to make them useful for forecasting. These could be determined subjectively through internal or external surveys of key stakeholders or experts.

Alternatively, an econometric model could assess the likelihood of each fixed scenario based on the historical relationship between current conditions and future economic paths.

The advantage of this approach is that the individual scenarios remain constant – e.g., the “8% unemployment” scenario will always look the same. For certain applications, this construct may prove easier to explain and manage with stakeholders.

However, there are two significant drawbacks to this approach. One is that analysts must carefully consider – and potentially adjust – scenario probabilities each time they run through an analysis. Even with a formal model of scenario likelihood, the rarity and uniqueness of recessions necessitate scrutiny of the probability estimates based on the most current information.

The second drawback is that this algorithmic approach will not produce a narrative for the evolution of each scenario.  

When setting a 90th percentile scenario under the traditional (fixed percentile) approach, economists will typically develop a unique narrative describing the factors driving the transition from the current state to the downside scenario. This narrative can aid risk managers and auditors in assessing scenario reasonability and in identifying ways to mitigate potential risks.

Without an underlying narrative, the fixed scenario approach may fail to capture nuances surrounding the timing and impact of specific shocks to the economy. While a narrative could be developed for each of the probability assumptions, it would not influence the dynamics or severity of each of the fixed scenarios. This may be a significant drawback for portfolios and applications that are particularly sensitive to the timing and nature of economic shocks.

Parting Thoughts

Scenario design plays a crucial role in risk analysis, impacting both numerical outcomes and stakeholder communication. While complex simulations may improve the accuracy of risk estimates, they may be time-consuming and increase the risk of operational error.

The fixed-scenario approach is unconventional but offers a straightforward method for defining paths with varying scenario probabilities, which some risk managers and organizations may find attractive. However, careful attention must be given to the tradeoffs involved in adopting this approach over the more orthodox fixed-percentile approach. Chief among these is the importance of scenario narratives for expository purposes.

As demands for scenario analysis increase and new scenario design tools emerge, risk organizations will need to determine the best approaches for each of their applications. Given the outsized impact that scenario selection and weighting can have on loss reserves and other risk estimates, focusing careful attention on scenario design is time well spent.

Cristian deRitis is the Deputy Chief Economist at Moody's Analytics. As the head of model research and development, he specializes in the analysis of current and future economic conditions, consumer credit markets and housing. Before joining Moody's Analytics, he worked for Fannie Mae. In addition to his published research, Cristian is named on two U.S. patents for credit modeling techniques. Cristian is also a co-host on the popular Inside Economics Podcast. He can be reached at cristian.deritis@moodys.com.