The Federal Reserve is now being sued, by the American Bankers Association and other lobby groups, about the opaqueness and fairness of its stress testing framework. Though the central bank has heard complaints about CCAR, its annual capital adequacy stress test, in the past, it’s possible the lawsuit could yield significant stress testing changes.
This is obviously important news for credit risk practitioners. Put simply, the bankers are arguing that CCAR was “adopted in secret” and that the central bank’s models “[produce] vacillating and unexplained requirements and restrictions on bank capital.” This echoes a recurring gripe of the banks – that the Fed’s stress tests are arbitrary, and therefore (possibly) capricious.
Currently, the central bank’s stress testing regime is unscientific, relying heavily on scenario analysis. We’ll have to wait to see how the lawsuit plays out, but it does raise questions about the CCAR revisions that it could possibly fuel.
Tony Hughes
Will it, for example, lead to more stress testing transparency and less volatility? Perhaps more importantly, will the current scenarios-heavy approach yield to more scientific models with clear validation criteria? The latter outcome would be a major change, because it would call for the Fed to defer to banks in instances where their stress testing models prove superior.
By its very nature, of course, scenario analysis really is arbitrary. When faced with competing predictions of portfolio performance given a certain scenario, there is no way to determine which is the most reliable, either in prospect or in hindsight.
Indeed, after 15 years of stress tests, we still cannot say that the Fed’s models are better than those of banks. This is because we have no way of making an objective evaluation, for example, by comparing past scenario predictions to actual observed outcomes.
Let’s now consider the potential changes (good and bad) the banking groups’ lawsuit could yield, including any unexpected consequences.
Transparency, Volatility and Objectivity
Interestingly, the Fed responded to the suit proactively, declaring the day before it was filed that they would act to improve the transparency of the models and take steps to reduce the volatility of the capital buffer they impose upon the banks. These adjustments, however, will have no effect on the core problem of arbitrariness.
It will be fascinating to see how the courts grapple with this reality. Banks have previously had no practical way of objecting to the efficacy of CCAR. However, by rushing to make the changes it announced, the Fed has conceded that the banks are correct when they complain of having no way of effectively challenging the current methodology.
If the court decides that this situation is intolerable, what is the remedy? Is transparency enough? The banks may well view this as sufficient compensation. You can imagine them rationalizing the situation as follows: “As long as we know how to optimize our businesses in the face of arbitrary capital allocations, it doesn’t really matter that they are arbitrary.”
Of course, humble taxpayers and citizens will likely be harmed by this solution. Shining more light on the stress testing process will unquestionably enhance the ability of the banks to game the system. If they know the precise capital consequences of every loan they originate, they can use proprietary models to find the most profitable exposures for any given quantity of capital, boosting profits while also increasing the amount of aggregate risk in the financial system.
What happens if the courts also view this as intolerable? What are we left with? The Fed would be sent back to the drawing board to design capital rules that are both transparent and impossible for the banks to undermine.
Such a methodology would have to be based on cold, hard, science. It would require clear validation criteria based on predictive performance that would allow models to be objectively ranked. In cases where the banks’ internal models could be shown to outperform the regulator’s formulations, the Fed would have to defer to the banks.
Lacking Scientific Credibility
By placing unscientific methods at the heart of its stress test methodologies, the Fed has now exposed itself to the ultimate model risk.
The real headscratcher for me is why financial regulators have not diversified their analytical toolkits within the stress testing arena. Since the global financial crisis, scenario analysis has been viewed as a cure-all – the answer to every important question posed by central banks when assessing banking risk.
This led to a situation where the world’s financial regulators have placed all their analytical eggs in one basket. And now the banks are challenging the freshness of those eggs in an actual court of law.
The scenario-led approach was used successfully to quell market angst during the dark days of the global financial crisis. But when the financial system is not blowing up, when routine calculations need to be performed accurately and consistently, the scenario method is too scientifically loosey-goosey to be of any practical benefit.
Parting Thoughts
I’m no lawyer (though I have watched a lot of Law & Order), but I don’t think it will be very difficult for clever corporate lawyers to highlight the weaknesses within the current stress testing regime. Every time I imagine it, the witnesses for the defense are always floundering, unable to justify the Fed’s faith in the unproven technique it has staked its reputation on.
The regulators’ goal for 2025 should be to reform the process by addressing CCAR's glaring weaknesses. If this means they must develop new methods with greater scientific merit, so be it.
Tony Hughes is an expert risk modeler. He has more than 20 years of experience as a senior risk professional in North America, Europe and Australia, specializing in model risk management, model build/validation and quantitative climate risk solutions. He writes regularly on climate-related risk management issues at UnpackingClimateRisk.com.
