Strategic Visions in AI Governance

Mapping pathways to victory

What future do we want for advanced AI? There are many possible ‘strategic visions’ for how to navigate the transition to superintelligence. This isn’t just an academic question: your preferred vision determines whether you think the US should increase its lead over China, whether governments should take control of AI development, and whether international cooperation is necessary or counterproductive.

Advanced AI poses several severe risks, including takeover by misaligned systems, wars between great powers competing for AI dominance, and extreme concentration of power. How we try to address these depends on fundamental uncertainties - will AGI come soon? Can international cooperation work? Is AI alignment profoundly difficult?

We identify nine distinct strategic visions for navigating these challenges, from the Silicon Valley status quo to a global and government-led project. Each makes different tradeoffs, has different empirical assumptions, and foregrounds different goals. This post summarizes the diverging visions, their policy implications, and the technical and institutional variables underlying which are most feasible. Table 1 shows our overall subjective assessment of each strategic vision.

Note that the specific people we assign to different strategic visions may not solely or wholly endorse that vision, and their views may have changed since writing the source we refer to.

Table 1: Overall subjective ratings of the nine strategic visions. Colors represent our assessment of how well each vision addresses each risk and how feasible each vision is to implement: red is high-risk/low-feasibility, green is low-risk/high-feasibility, and yellow is in-between.

Nine strategic visions

We organize the landscape of strategic visions along two key axes:

• Private vs government control: Who decides when and how to develop advanced AI?

• Distributed vs centralized power: Is control over advanced AI spread across many actors or concentrated in one project?

Figure 1: The nine strategic visions mapped to the private/government and distributed/centralized axes.

Cluster 1: Private company-led development

Competing private projects (status quo)

This is the world we live in today. Multiple companies—OpenAI, Anthropic, Google, Meta, and others—compete to build the best models under relatively light regulation. The core logic: AI is a normal technology where market competition drives both innovation and safety, and standard corporate antitrust and taxation tools are sufficient to avoid extreme power concentration.

Advocated for by: Silicon Valley techno-optimists.

Problems: This vision fundamentally assumes alignment is relatively easy. If it’s not, competitive dynamics create a race to the bottom on safety - each company faces pressure to ship products quickly rather than invest heavily in alignment work. The benefits of being safe are shared broadly while the costs are borne privately, a classic collective action problem.

Feasibility: This seems like the default trajectory. The main question is whether it gets disrupted by a crisis, government intervention, or other factors.

Single private project

In this vision, one company achieves AGI and automates AI R&D before anyone else, triggering recursive improvement in intelligence that gives them an unassailable lead. Governments are too slow to intervene. This single company then accumulates extreme global influence. With a large lead over competitors, this company can afford to invest heavily in alignment and control techniques to ensure safety.

Advocated for by: No one, publicly.

Problems: Under this vision, a small group of unelected, unrepresentative individuals would make civilization-shaping decisions. This sort of extreme power concentration is illegitimate and unlikely to produce good outcomes for the rest of society.

Feasibility: Currently unlikely. Multiple well-resourced companies and countries are near the frontier. For one to pull far ahead would require something like a major algorithmic breakthrough combined with tight information security.

Global private project

Market forces and rising costs drive consolidation into a single multinational company that develops frontier AI. Financing, talent, compute, and energy come from many countries. Governments are involved - investing, approving mergers, possibly even banning rival projects - but the entity remains a private company with standard corporate governance (shareholders, board of directors, etc.).

This is like the previous vision but with broader stakeholders, reducing the concentration of power in a few executives’ hands.

Advocated for by: Nick Bostrom

Problems: While power is less concentrated than in the previous vision, centralization means there is still a single point of failure. And despite multinational governance, investors may be shortsighted, prioritizing rapid monetization over careful risk management. Additionally, democratic oversight is poor, since people without shares in this company will not (directly) be represented.

Feasibility: Requires some international cooperation but less than formal treaties.

Cluster 2: US government-controlled development

US leadership with domestic regulation

Private companies continue developing AI in the US, but the government plays a more active role in ensuring safe development. This could include requiring safety cases before training runs, mandatory third-party evaluations, and incident reporting requirements. Power remains distributed between different AI companies and government agencies. Internationally, the US uses export controls and diplomacy to maintain its lead over China.

Advocated for by: Dario Amodei, Helen Toner, Cullen O’Keefe

Problems: The federal government currently lacks the technical talent and know-how to perform hands-on inspection and regulation of the AI industry. Without a crisis, oversight may be very weak. And continued racing between private developers could incentivize merely perfunctory compliance with safety requirements.

Feasibility: Requires significant political will and initiative but is less radical than full nationalization. The US government is already committed to parts of this vision, like slowing Chinese AI development through export controls.

US centralized government project

The US government decides AI is too strategically important to leave to private companies and centralizes development under government control. This could look like a government-led consortium (such as the Apollo model, with NASA coordinating multiple contractors) or a single prime contractor (such as Lockheed Martin with the F-35 fighter jet). The project develops a large lead over competitors and uses this lead to invest in AI alignment and control research, and deploy models carefully without needing to worry about falling market share.

Advocated for by: Leopold Aschenbrenner, U.S.-China Economic and Security Review Commission

Problems: This is the most likely vision to trigger an international arms race, or even a hot war. China would likely respond by launching its own centralized project, transforming commercial competition into a direct military-technological showdown. Both sides may then race recklessly, with less room for safety work and collaboration. Additionally, the massive increase in executive branch power from controlling advanced AI could threaten Constitutional democracy by making it harder for Congress and courts to maintain checks and balances.

Feasibility: Requires major expansion of federal authority. This would be more likely after a major AI incident or if development costs become too large for private actors. Our recent expert forecasting study gave a median probability of 34% for government control of advanced AI.

US+allies government project

Similar to the previous vision but internationalized: the US leads a coalition of allies (EU countries, UK, Canada, Australia, Japan, South Korea, Taiwan) in a joint intergovernmental project. These countries collectively dominate semiconductor manufacturing, research talent, and financing, making it easier to outpace rivals than if the US acted alone. Benefits from the previous vision still apply, such as a large lead enabling strong safety investment, but joint decision-making among allies reduces concentration of power concerns.

Advocated for by: Forethought, (possibly) Haydn Belfield

Problems: There is still a significant risk of triggering an aggressive Chinese response to this Western coalition. This could include launching a rival project or sabotaging the Western project.

Feasibility: Requires substantial international coordination. The US may be reluctant to include allies rather than going ahead by itself.

Cluster 3: International approaches

International competition and deterrence

In this vision, proposed by Hendrycks, Schmidt, and Wang, rival powers each pursue AI development, but mutual deterrence prevents reckless racing. Each side fears the other achieving superintelligence and a decisive strategic advantage, so they threaten to sabotage projects that race irresponsibly toward ASI. This is somewhat akin to nuclear deterrence, where the threat of mutual destruction creates stability. Under these dynamics, the US and China might coordinate to slow AI progress until ASI can be developed safely. The future is then shared between rival powers rather than monopolized by one.

Advocated for by: Dan Hendrycks

Problems: This vision allows non-democratic states like China to maintain significant control over the future, which is arguably problematic. Deterrence equilibria can also be unstable: if countries have different understandings of what actions are unacceptable, miscalculation could trigger conflict. A great power war could also easily become a nuclear war.

Feasibility: As I discussed previously, this depends on adversaries expecting significant losses from rivals’ AI development, being motivated to make threats, and countries being influenced by those threats. Each premise is somewhat more likely than not, but the conjunction is very uncertain. This also requires a radical divergence from the status quo of international relations.

Global centralized government project

Major powers agree to consolidate frontier AI development into a single project jointly controlled by the US, China, and possibly others. Rather than competing, they jointly develop advanced AI with shared control and a strong safety focus. The joint project has a large lead over any other AI developers, so it can afford to invest heavily in alignment and control.

Advocated for by: Various civil society organizations

Problems: Although this vision has some democratic legitimacy, it still has a single point of failure. It requires governance structures robust enough to prevent regulatory capture or an unwanted slide toward world government formed around the joint AI project.

Feasibility: This may be the most ambitious vision, since it requires close cooperation between rivals on strategically critical technology. It is especially unlikely if ASI arrives soon, since it’s far outside what’s politically achievable today.

Global coordinated regulator

An international body—sometimes called an “IAEA for AI”—has oversight of all frontier AI projects. Multiple private developers or governments could still pursue AI, but the regulator blocks further capabilities research if developers haven’t demonstrated adequate alignment and control. The regulator needs verification and enforcement powers to ensure compliance.

A more radical version would enforce a ban on AI development above some capability threshold until enough progress is made on solving alignment.

Advocated for by: 2023-era OpenAI, MIRI

Problems: The global regulator could come under immense pressure and be subject to political capture: countries may pressure the regulator to quickly approve their models while delaying rivals’. Moreover, verification is difficult: how do you assess AI capabilities and alignment without invasive access to countries’ data centers? And enforcement against non-compliant states is challenging without mechanisms that may themselves be difficult to establish.

Feasibility: This is even more politically difficult than comprehensive US domestic regulation. But it requires less trust than full consolidation since development stays distributed.

Transitions between visions

The landscape isn’t static: Figure 2 shows plausible transition pathways between different visions.

Figure 2: Possible transition pathways between strategic visions.

Some key transitions:

Status quo → domestic regulation: Growing awareness of AI risks and political will to act, without a crisis forcing more drastic measures

Domestic regulation → government project: A crisis or security incident triggers nationalization, or development costs become prohibitive for private actors

US project → US+allies: Allied governments demand inclusion rather than being left out of civilization-defining decisions

Any unilateral approach → MAIM dynamics: Adversaries fear being left behind and threaten or take action to prevent rivals from achieving a decisive advantage

MAIM → global cooperation: Countries formalize an initially tense standoff into binding international agreements

Understanding these pathways matters because actions taken to advance one vision might inadvertently trigger a transition to another—potentially less desirable—one. For instance, a US government project intended to ensure safety might be the very thing that triggers an international arms race.

Policy implications

Different strategic visions recommend strikingly different policies. Consider two examples where visions diverge: the level of transparency into AI companies, and whether to accelerate US AI progress.

Transparency into private AI developers

Should we require AI companies to disclose their capabilities, safety testing results, and internal deployment practices?

In favor: Government oversight visions (domestic regulation, government projects) strongly favor transparency. The government needs visibility into what companies are doing to design and enforce sensible guardrails. International visions also benefit from transparency reducing information asymmetries that could lead to miscalculation.

Against: Private-led visions are more mixed. If you think governments lack the competence to interpret disclosures well, transparency requirements may slow developers without reducing risks. However, some voluntary transparency might forestall heavier-handed regulation.

Accelerating US AI progress

Should the US maximize the speed at which it develops advanced AI?

In favor: Status quo visions favor fast progress before the geopolitical landscape changes in ways that disrupt private-led development.

In between: Government-led visions need progress fast enough that China doesn’t catch up, but slow enough that the government has time to step in and organize oversight or take control if needed.

Against: International cooperation visions may need time to negotiate agreements and build institutions. Slowing capabilities progress buys that time, even if it means a smaller US lead.

These aren’t the only policy debates - the full report analyzes six in detail - but they illustrate how your strategic vision shapes your policy priorities. Some policies, like improving developer cybersecurity, are good across most visions. Others are deeply contentious.

The key uncertainties (cruxes)

What you believe about the following questions largely determines how desirable and feasible each vision is:

Technical questions

Will AGI come soon? The sooner AGI arrives, the less time there is for major changes in domestic governance or geopolitical arrangements. Short timelines favor the status quo or modest domestic regulation. Long timelines make government-led or international approaches more feasible since there’s time to build new institutions.

Will ASI follow quickly after AGI? Fast takeoff scenarios make it more likely that a single project achieves a large lead and possibly a decisive strategic advantage. This favors centralized visions. Slow takeoff gives laggards time to catch up, making multipolar outcomes more likely and giving governments time to respond and implement oversight.

Can one actor build a large lead? This depends on whether compute advantages persist, whether algorithmic innovations diffuse quickly, and how effective cybersecurity is at preventing model theft. A large lead makes centralized visions more likely and attractive. If no actor can maintain a significant lead, international approaches become more necessary, since the U.S. would not be able to monopolize powerful AI capabilities indefinitely.

Will ASI confer a decisive strategic advantage (DSA)? If the first actor with ASI can achieve unrivaled global dominance (potentially including undermining nuclear deterrence), this makes unipolar outcomes more likely and MAIM dynamics more intense. If ASI doesn’t enable a DSA, multipolar cooperation becomes more stable. Additionally, what key actors believe about the likelihood of a DSA matters as much as the underlying reality.

How hard is alignment and control? If it’s profoundly difficult, this favors centralized development to eliminate racing and allow massive safety investment. If it’s tractable with modest resources, pluralistic distributed approaches look more attractive.

Institutional questions

Will the US government prioritize AI? A greater government focus makes domestic regulation and government-led projects more likely. If the government remains relatively disengaged, private-led development continues by default.

Is international cooperation feasible? Pessimism here favors US-centric visions. Optimism makes ‘global regulator’ and ‘joint project’ visions more viable. This depends significantly on advances in verification to make cooperation between parties that don’t trust each other more viable.

Are governments more trustworthy than companies? If you think democratic governments have better incentives and legitimacy for making high-stakes decisions about AI, you’ll favor government-led visions despite concerns about technical competence. If you think governments are vulnerable to capture or lack expertise, private-led approaches may be better.

Lessons and implications

The transition to superintelligence is among the most consequential challenges humanity faces. Several lessons emerge from mapping the strategic landscape:

No vision dominates. There is no “one vision to rule them all.” Each makes different tradeoffs between the three risks. Each assumes different things about timelines, takeoff speeds, institutional competence, and international relations. Rather than assuming one pathway is inevitable or obviously best, we should prepare for multiple scenarios.

Favor robust strategies. Some interventions are valuable across nearly all visions:

• Improve AI developer cybersecurity to at least RAND security level 4.

• Develop verification technologies for potential future agreements, even if those agreements seem far off.

• Make progress on alignment and control research.

• Build government expertise on AI risks so policymakers can respond effectively if needed.

Resolve key uncertainties. We should prioritize research on the cruxes that most affect which visions are desirable and feasible:

• Which is more concerning: power concentration from centralized AI development, or corner-cutting on safety from race dynamics in distributed AI development?

• Will it be possible to build and maintain a large lead in AI, and therefore broader strategic power, or will technology diffusion equalize capabilities?

• What geopolitical scenarios are likely to be stable and desirable?

Our goal should be to understand the landscape well enough to pursue robust strategies, gather information to resolve uncertainties, prepare for multiple futures, and avoid stumbling blindly into catastrophe. The stakes are too high for anything less.

This post summarizes findings from “Strategic Visions in AI Governance,” a report by Oscar Delaney, Maria Kostylew, Oliver Guest, and Peter Wildeford for the Institute for AI Policy and Strategy.