
Gathering data and modeling is – by a considerable margin – the best way to optimize portfolio construction and management. But because modeling talent is spread thin and wide, it would be impractical for even a large bank to have every modeler it needs or wants on its payroll.
This is one reason why large and small analytics companies have a chance to provide their services to major financial corporations – and to earn tidy profits as a result.
I’d like to now share lessons I've learned from the process of selling modeling services, primarily to banks. (Note: I’ve always been an outside vendor or consultant and am certain that an experienced banker would have a very different perspective on these interactions.)
Sellers and Buyers: Breaking Down the Landscape
There are elements of the interaction that newbie vendors and consultants should be aware of when they set out of their journey. I also think there are some questions bankers should ask of analytics vendors, particularly given that we are living through an age where conjecture dominates many aspects of risk management.
Tony Hughes
There are only a handful of situations where the relationship between an analytics vendor and their customers is likely to be static and long lasting. However, if the outsider happens to own a critical dataset that would be difficult or impossible for a different third party to replicate (or for a bank to mimic internally), the customer will generally be reliant on the models produced by that vendor.
The most notable examples of this are companies like credit bureaus and major ratings agencies. The information provided by these firms is simply indispensable for making a wide range of credit decisions.
There are, of course, companies that set out to develop or collect new datasets they view as potentially useful to banks – but it is labor intensive work and, as such, a tough road to hoe. That said, if you put in the effort and wind up with unique data that provides useful information to large corporations, there will be a deep moat around your business and success will be all but assured.
Those who are using public or non-exclusive data, or who are proposing to apply a proprietary modeling technique using the bank’s own data, face an entirely different challenge. This description applies to most of the generative AI and physical climate risk products that are currently flooding the market.
“In-House” Arguments and Regulatory Challenges
In these cases – assuming the models are truly novel and of significant benefit to the bank – the vendor faces an eternal running race against the bank’s internal analysts. Right from the first meeting, the bank’s managers will tell you, either accurately or inaccurately, that the in-house team is perfectly capable of developing their own version of your models. So, you’ll be asked, why do they need you to hold their hand through the build and implementation processes?
Even if your intellectual argument is powerful, you will still need to have a good answer to this question. Sometimes you get lucky (e.g., when the in-house team has too much on their plate), but generally you need a good description of how your involvement will save the bank time and/or lead to a substantially better final product.
Once you’re in the door, the race does not stop. Forging a long-term relationship with your clients will require a high level of customer service which, for third-party modelers, means collaborating closely with in-house teams. Regulators increasingly require bank staff to have a tight working knowledge of all the inner machinations of their models (whether they are developed by external vendors or built in house), with very strict documentation requirements.
Finding Your Place in the Market
Fencing off your intellectual property from these interactions will either be exceptionally difficult or impossible. If your method is highly technical, requiring skills lacked by the in-house team, you’ll generally be invited to stick around as long as the benefits offered by your models continue to stack up.
The only other situations where you will be invited to stay is if your output is exceptionally labor intensive to produce or if you’re making continual improvements to the technique as time passes.
The other important service provided by external vendors is market intelligence. It will never be legal or ethical to share the secrets of one bank with its competitors, but if your model is used by a wide variety of banks, you will be able to identify common use cases and suggest these to other clients.
Often, one client will help you develop your product in some way that then makes it more attractive to a broader audience. Sometimes products used across the industry become so entrenched that a common language evolves around the product that will be understood by all industry professionals.
If you ever reach this point, you'll know you've been successful.
AI and Scenario Analysis: Tough Sells
When it comes to selling, risk analytics that are based on conjecture potentially pose a significant problem. I think that a lot of AI products fall squarely in this category, simply because the reasoning behind the output is impossible to explain to a third party.
Things like scenario projections share the same fundamental characteristic: we can peruse the output, but judging its quality requires faith that the human/robot responsible for it really is very intelligent. Gut feelings are conjectural, whether they stem from a motherboard or, ultimately, from an actual mother.
Questions I would pose to a purveyor of these products would attempt to elucidate these abilities. For example: (1) Why is your product more trustworthy than the one we looked at last week? (2) Do you have an established track record of making good projections? and (3) Can you demonstrate this, and how do you know that your projections can truly be trusted?
I wonder, moreover, about the staying power of these conjecture-driven products. Can an AI build trust and loyalty? Similarly, if scenario-based projections can never be tested against actual, real-life data, is there a mechanism capable of dispelling the customer’s lingering doubts?
Parting Thoughts
The financial services industry has a heavy intellectual burden. Banks, insurers and investors need to understand risk, both for their own profit-making aspirations and for regulatory compliance. Companies are invariably looking for an edge, whether this involves reducing the cost of doing business or by finding more profitable ways to service new and existing clients.
This has opened the door for risk analytics vendors and consultants. The proof for sellers is, and will always be, in the pudding. From my experience, banks are ruthless about canceling products that no longer provide a demonstrable value.
Unless your product is continually improving, the writing is always on the wall.
Tony Hughes is an expert risk modeler. He has more than 20 years of experience as a senior risk professional in North America, Europe and Australia, specializing in model risk management, model build/validation and quantitative climate risk solutions. He writes regularly on climate-related risk management issues at UnpackingClimateRisk.com.
Topics: Model Risk