Culture & Governance Risk | Insights, Resources & Best Practices

Beyond the Doomer-Utopian Binary: Building a Three-Level AI Risk Governance Structure for Eight Billion People

Written by Brenda Boultwood | October 2, 2026

Artificial intelligence has reached a political and operational tipping point. Will we call a time out? Or will commercial and competitive pressures endure? The time has passed for AI governance to be left to abstract academic exercises or the market’s invisible hand.

Across the United States, public frustration has erupted against the physical footprint of AI infrastructure, with 61% of adults opposing local data center construction as wholesale electricity prices spike by up to 267% in heavy-development corridors. An NBC News poll revealed that 70% of Americans are more worried than excited about AI.

At the same time, AI industry fractures have spilled into public view. This is highlighted by high-profile resignations of frontier lab researchers warning that firms are racing toward self-improving superintelligence without adequate safety controls. And national security agencies have issued an urgent advisory regarding industrial-scale model distillation by foreign competitors.

For Chief Risk Officers (CROs) and policymakers across the government and private industry, this convergence of local economic stress, geopolitical friction, and technical volatility demands a sophisticated analytical lens. As Max Weber observed during the rise of industrial capitalism, major technological changes represent secular shifts that standardize human activity.

Brenda Boultwood

This can be exemplified today in algorithmic recommendation engines that silently curate attention, steer desire, and make certain socieconomic paths appear natural while obscuring others. Furthermore, as W.E.B. Du Bois emphasized, technological evolution cannot be analyzed in isolation from elite power, resource extraction, and global inequality.

AI risk governance must not be designed merely to protect corporate balance sheets or the interests of a handful of billionaires. It must serve the stability and welfare of eight billion people.

This requires abandoning reductive debates between some AI “singularity” utopians and a doomer-existential “rationalism.” Instead, risk leaders must construct a comprehensive, business governance structure across three distinct levels: federal, industry, and firms (see Figure 1 below).

An AI Risk Governance Structure

Level 1: Federal Governance and Mandated Capacity. At the macro level, private voluntary commitments are insufficient to manage systemic threats to public safety, critical infrastructure, and macroeconomic stability. Federal governance must possess genuine intellectual, technological, and statutory capacity to oversee frontier AI development and business usage. Legal scholar Cass Sunstein has proposed creating a federal AI Regulatory Commission. This would require an active and informed role of the federal government in a truly independent and dedicated oversight body to provide the statutory authority necessary to set binding operational rules and enforce compliance.

While frontier labs advocate for mandatory national safety requirements or embedded independent evaluators, true federal governance requires that oversight and enforcement functions remain strictly in public hands rather than be delegated to corporate partners. Key federal objectives must include:

  • Systemic Risk Monitoring. Establish mandatory reporting thresholds for compute usage, advanced capability evaluations, and severe security incidents.
  • Public Infrastructure and Resource Protection. Regulate data center energy consumption, water usage, and ratepayer impacts to protect local communities from severe cost spikes.
  • Geopolitical and Intellectual Property Defense. Enforce strict cybersecurity controls and monitoring API traffic to prevent illicit model distillation and unauthorized extraction of national technological assets by foreign state-backed actors.

Level 2: Self-Regulatory Organizations (SROs). Government regulation alone cannot keep pace with the hyper-rapid evolution of frontier models. To bridge the gap between broad statutory mandates and technical execution, the frontier labs and firms using AI models require industry-level SROs, modeled after the Financial Industry Regulatory Authority (FINRA) in financial services.

An AI SRO, funded by industry participants but governed by an independent board of technical experts, ethicists, and public representatives, would establish standardized operational protocols across competing firms. An SRO could establish strategy and objectives to include:

  • Standardized Safety Benchmarks. Developing consensus protocols for capability evaluations, red-teaming, and model alignment prior to commercial deployment.
  • Cross-Lab Threat Sharing. Operating an incident clearinghouse to share real-world failure modes, prompt-injection vulnerabilities, and distillation attacks in real time.
  • Pacing Protocols. Establishing voluntary and enforceable mechanisms to "deliberately pace" or coordinate pauses to frontier model releases when safety evaluations flag potential autonomous capabilities or uncontrolled self-improvement.

Level 3: Firm-Level Governance and the Three Lines Model. Within firms using AI models – whether frontier AI labs such as Anthropic and OpenAI, or financial institutions or healthcare companies deploying third-party AI models – internal governance relies on operationalizing the Three Lines of Defense (3LoD) model.

  • First Line (Business Management): Product development teams and AI researchers directly identify, assess, and mitigate risks during model training, fine-tuning, and deployment using automated safety filters and curated datasets based on an established AI risk management framework.
  • Second Line (Risk and Compliance): Dedicated risk management, legal, and compliance teams provide independent oversight, draft internal policies, monitor Key Risk and Performance Indicators (KRIs, KPIs), and enforce responsible scaling protocols.
  • Third Line (Internal Audit): Independent internal auditors provide objective assurance directly to the Board of Directors' Audit Committee. Effective large language model (LLM) auditing requires a three-layered approach combining process-oriented governance audits of organizational structures, performance-oriented model audits of pre-trained systems, and impact-oriented application audits of downstream deployments. Internal audit serves as the board’s “eyes and ears,” ensuring executive teams do not bypass safety controls under commercial pressure.

Figure 1: Three-Level AI Risk Governance Framework

 

Firms Across Industries Must Also Adapt

Without federal and industry governance, a firm using closed or open-source AI models might respond merely by banning suspicious AI activity, setting ad hoc rules, or doing nothing. Applying the three-level framework reveals how complex business decisions require coordinated firm governance across all three levels.

  • Firm Level: The third line (Internal Audit) reviews API security logs and red-teams anti-distillation defenses, while the first and second lines implement dynamic output modification to defeat automated AI content-scraping of its website, identify hacks into its proprietary networks, and wargame new approaches to thwart determined AI agent infiltration.
  • Industry Level: The AI SRO aggregates threat intelligence across labs and identifies multi-platform scraping campaigns that target multiple developers simultaneously. Like FINRA, the AI SRO would also perform critical roles designed to promote compliance and ensure public trust (see Figure 2).
  • Federal Level: Government agencies leverage export and import controls, including potential bans, and national security enforcement to penalize foreign corporate entities engaging in illicit intellectual property siphoning. Federal diplomatic clarity will be critical.

Figure 2: Six Key Roles of an AI Sector SRO

 

Three Paradoxes for a Chief Risk Officer

As CROs across sectors integrate this three-level governance framework into enterprise risk management, they must navigate three structural paradoxes. Each applies equally to the firms developing AI models and using the AI models.

The Speed vs. Safeguard Dilemma: Commercial imperatives push firms to deploy models rapidly to capture market share and match foreign competitors. Yet, rushing deployment without rigorous second-line verification and third-line auditing increases the probability of catastrophic safety failures or severe regulatory penalties.

The Independence vs. Technical Intimacy Paradox: Effective third-line internal audit requires deep technical understanding of foundation internal and open-source AI model architecture. However, as auditors become embedded in technical development teams to understand complex capabilities, they risk losing the organizational independence necessary to challenge executive decision-making objectively. When open AI model architecture cannot be understood, they will need to make unpopular recommendations about allowed usage.

The Local Community vs. Global Hegemony Conflict: A national AI development moonshot demands rapid expansion of data centers and compute clusters to maintain technological dominance over global rivals. Yet, this global mandate directly conflicts with local community concerns, driving severe ratepayer backlash, environmental disruption, and political pushback. CROs must manage enterprise risk in an environment where federal mandates and local voter opposition are in direct opposition.

Parting Thoughts

Managing AI risk is no longer a choice between passive acceptance of technological determinism and futile calls for outright technological halts. By structuring governance across federal, industry self-regulation, and robust firm internal audit procedures, risk executives can build frameworks that withstand both commercial pressure and geopolitical instability.

While instilling AI risk governance is a critical step, I look forward to a future discussion of an AI Risk Management Framework, detailing common taxonomy considerations, approaches to risk assessment, specific measurement tools, risk tolerance metrics, and separate third-line considerations for internal auditors.

 
 

Brenda Boultwood is the Distinguished Visiting Professor, Admiral Crowe Chair, in the Economics Department at the United States Naval Academy. The views expressed in this article are her own and should not be attributed to the United States Naval Academy or the U.S. Department of Defense.

She is the former Director of the Office of Risk Management at the International Monetary Fund. She has previously served as a board member at both the Committee of Chief Risk Officers (CCRO) and GARP, and is also the former senior vice president and chief risk officer at Constellation Energy. She held a variety of business, risk management, and compliance roles at JPMorgan Chase and Bank One.