The use of generative AI in banking is now pretty well established. It’s currently used for a variety of tasks, and its strengths and weaknesses are becoming more obvious.
If the technology was an employee, given its highly unusual set of skills, its probationary period would have ended and its future would be considered bright by most senior managers.
Tony Hughes
But the new employee has not yet shown that it is completely trustworthy. We know that it is prone to hallucinations and that it can suffer from a variety of intellectual shortcomings.
What’s more, it is often insubordinate, refusing to follow simple black-and-white directions. I suspect that it has no defense against arguments from authority and that it is susceptible to the bandwagon fallacy – i.e., that the most popular view is the correct one to espouse.
While generative AI tools can do many things that humans find tedious or impossible, questions remain about the proper way to deploy the technology in the context of bank risk management. So, how can and should it be used?
I was reminded of generative AI’s potential quite recently while working on a credit risk stress testing project with a major European bank. They had identified a highly eclectic portfolio of corporate loans and were trying to determine whether the macro risks faced by the exposures had been adequately assessed by the in-house team. When I say the loans were, eclectic I’m not kidding around – things like chemical plants producing substances you’ve never heard of and companies producing intermediate widgets for other widgets.
The bank asked for a high-level critique of its risk assessment process. It was near the end of the project, and they needed my report in 24 hours.
With full disclosure to the client, I used ChatGPT to help me quickly gather intelligence about the industries involved, about which I had no prior knowledge or experience. My strategy was to identify the most obvious risks to the companies and to determine whether a competent risk analyst could have reasonably been expected to spot the factors identified. Subsequently, I compared this to the manner in which the exposures were actually assessed by the internal risk team.
To their credit, the client asked exactly the right questions: Is it reasonable and appropriate to use generative AI in this manner? Can it be trusted in this specific context?
From my perspective, the use of the technology enabled me to meet an extremely tough deadline with at least a reasonable response. Had I not used ChatGPT, I would have spent many hours researching the industries in question, just to get me to the starting line in assessing the bank’s procedures. The bot also found a few crucial references that I probably would have missed if I only had Google at my disposal.
I’m pretty sure the in-house credit teams did not use generative AI the way I did. Had they, I suspect that their analysis would have been based on a far richer dataset and that it would have thus been more accurate.
For one of the industries in question, for example, ChatGPT identified data indicating that Chinese producers were scaling back production due to a local supply glut and waning domestic demand. Given these conditions, it would be extremely difficult for a European competitor to hit its ambitious growth projections. The credit rating for the exposure was highly contingent on these forecasts.
Now, obviously, the references and data sources provided by ChatGPT needed to be corroborated and verified. To the extent that it was possible in a single day, I checked a selection of the bot’s references and data, and felt them to be sound. Given a bit more time, this could have been done exhaustively. I told the client that my report should be viewed as “tentative” until this additional work could be completed.
Of course, a report produced without the use of generative AI should also be subject to this kind of validation. I’m certain that many traditional risk analysts, like their artificial colleagues, are prone to the odd hallucination and fall for a variety of intellectual fallacies on a regular basis. Generative AI might be susceptible to a range of biases but, alas, so are you and me.
To my mind, the proof of the pudding must be in the eating. The use of artificial intelligence in, say, an artistic context may obscure or even pollute its core aim – which is, in part, to celebrate human excellence for its own sake.
But in financial risk management, all that really matters is making accurate credit prognostications in an efficient manner. If this is artificially enhanced, frankly, so what?
I suspect that the Brave New World of reliable, automatic credit risk assessment is still a long way off. But the current range of available tools has enhanced the ability of humans to assess risk quickly and accurately.
Our new electronic colleague should be welcome at the office, even though it isn’t ready to work without intense supervision.
Tony Hughes is an expert risk modeler. He has more than 20 years of experience as a senior risk professional in North America, Europe and Australia, specializing in model risk management, model build/validation and quantitative climate risk solutions. He writes regularly on climate-related risk management issues at UnpackingClimateRisk.com.