From the inception of financial engineering, it took the industry decades to integrate value-at-risk (VaR), Greeks, and copulas into its standard business practice.
Artificial intelligence is currently in such an introductory phase. Acknowledging this parallel can be challenging, as it subverts the conventional idea of risk management.
Shake a sandbox repeatedly, and certain light objects rise to the top. One can watch them rise, but the phenomenon itself never explains why those objects surfaced while others did not.
Much of contemporary machine learning operates like this: Useful outputs come to the surface, and the causal mechanisms that produced them stay ambiguous.
Finance does not have the option of that ambiguity. Decisions flow along long lifecycles, produce direct monetary consequences, and rest above all on credit, the oldest known form of capital. When regulators come back long after the fact and demand a rationale, a model that cannot explain its outputs can become a liability.
The major regulatory frameworks emerging today – including the European Union’s AI Act and South Korea’s AI Basic Act, particularly Section 35 – classify AI systems by their domain of application. If a model filters credit card applications or handles retail banking recommendations, it faces strict compliance paths because of the domain it touches, regardless of whether the model uses a basic decision tree or an advanced multi-task deep learning network.
Regulators are not auditing neural weights; they are auditing systemic domain outcomes.
This domain-first perspective reflects a generally agreed view within financial history. When quantitative finance expanded decades ago, risk management did not establish separate rules for every mathematical iteration. Instead, frameworks settled around asset portfolios, credit lines, and institutional resilience. The rulebooks evaluated what the tools did to the capital, not just how elegant the equations looked on paper.
This history explains why standards such as the Federal Reserve model risk management guidance, known as SR 11-7 and recently revised as SR 26-2, remain remarkably functional even when applied to deep learning. It does not care about the underlying code parameters. It specifies that a model is a system that processes data to produce quantitative estimates. While this principle holds, its implementation must evolve.
Seonkyu Jeong: Inherent explainability.
The challenge with modern deep learning is that safety and compliance cannot be added as an afterthought. Post-hoc guardrails, including output classifiers, prompt filters, alignment layers, and Shapley Additive Explanations (SHAP) explainers, are necessary. But they often bolt safety and interpretation onto a model that was trained primarily for performance. As highlighted in Model Risk Management and the AI Effect, these methods can create a fragile operational baseline.
True model risk management requires embedding these controls directly into the model architecture. This means moving toward inherent explainability. By incorporating structural constraints, causal graphs, or bounded induction biases directly into the network design, developers can ensure compliance principles are integrated from the beginning. Safety then becomes a property of the model’s structure, rather than a filter running on top of it.
Shifting risk management into the model architecture also resolves an operational bottleneck.
In most institutions, risk management teams are significantly smaller than the core engineering and development teams. A small risk team cannot review every model output, prompt variation, or inference log in real time. Expecting them to act as a manual, post-hoc quality-assurance gate creates an unmanageable administrative burden while failing to capture system-level risks before they manifest.
Instead, human judgment must be concentrated upstream, integrated into the model production pipeline and structural design phase. For a risk officer, evaluating a model should not mean decoding complex neural weights or manually inspecting trillions of inferences.
Rather, their role must focus on auditing the operational pipeline architecture – verifying data lineage controls, data drift measurement protocols, and invariant guardrails at designated checkpoint substrates before deployment. This co-design of the Machine Learning Operations (MLOps) pipeline transforms the risk officer’s function from isolated, reactive reviewer to strategic advisor at the design stage. The risk parameters defined at that stage are then enforced by the pipeline itself.
When financial engineering first introduced value-at-risk and factor models, the industry adopted a new technical vocabulary to describe, quantify, and ultimately limit risk. True integration, however, did not occur when these terms merely entered the industry's lexicon; that occurred when they were hardcoded into daily management mandates, trading limits, and capital allocation frameworks. The mathematical concepts were made concrete through institutional blueprints.
Today, deep learning systems have brought forth a new vocabulary: explainable AI, causal inference, and model observability, synthesized under the umbrella of responsible AI. Yet, these concepts currently risk hovering above the practical field as abstract, defensive checkboxes.
For these terms to achieve true institutional integration – similar to the tools of quantitative finance decades ago – they must not wander as post-hoc commentary. Instead, the concepts of responsible AI must converge onto concrete architectural specifications. They must be translated directly into the network's structural design templates, mathematical constraints, and input substrates.
The structural decisions being made now will determine whether AI matures into a core engine of financial trust or remains an unmanageable regulatory liability. For practitioners navigating the transition from traditional risk domains into AI modeling infrastructure, the path forward is clear. True integration demands that our risk vocabulary ceases to be an afterthought and becomes, fundamentally, the architecture itself.
AI is not the first new tool finance has met. We already know what it takes for a new tool to find its meaning in our work.
Seonkyu Jeong is an AI practitioner and FRM who over a 13-year career in financial risk and quantitative modeling has transitioned into developing core AI infrastructure and recommendation modeling systems at Korea Post Financial. His research on architecture-integrated model risk management, covering inherent explainability and audit-substrate design, is published on Zenodo.