---
title: AI has only two stable futures
slug: two-stable-ai-futures
author: ajfisher
date: 2026-07-28 12:00:00+10:00
layout: post
excerpt: >
Two futures become apparent once frontier AI becomes powerful enough to
reshape society. One depends on containing its diffusion. The other
depends on adapting to it.
twitter_excerpt: >
Two futures become apparent once frontier AI becomes powerful enough to
reshape society. One depends on containing its diffusion. The other
depends on adapting to it.
listimage: ../../img/posts/ai-futures-two-stable-structures-hero.png
imageby: ajfisher - ChatGPT Images
featureimage: ../../img/posts/ai-futures-hero.png
largetitle: true
tags: ai, security, strategy, government
featured: true
---
In my recent essay, I explored what happens if the effective AI capability
available to most organisations plateaus, whether because model progress slows
or because stronger systems sit behind regulatory, commercial and safety
gates. This assumption was primarily grounded in regulation and control
(aligned to [Yegge’s Flat Curve
Society](https://steve-yegge.medium.com/the-flat-curve-society-36c8b01eb33b)
idea) and I came to the conclusion that even if the next generation of models
were the best we got, they would still deliver significant economic and
business benefit. To do this, organisations must focus on closing the deployment
gap between how the business uses AI and what AI is fundamentally capable of.
Working through that essay, a nagging thought kept surfacing. A controlled AI
environment is only as good as the least constrained actor involved.
I was developing this framework when reports emerged that models being tested
by [OpenAI had broken out of a constrained evaluation
environment](https://openai.com/index/hugging-face-model-evaluation-security-incident/)
and [exploited Hugging Face
systems](https://huggingface.co/blog/security-incident-july-2026) while
pursuing benchmark data, causing a security incident.
This provided a timely example of the problem I was considering and reinforced
my view that a regulated Flat Curve environment can only remain stable when
high levels of control and access gating are in place. Other futures need to be
considered as well.
## The Flat Curve is an equilibrium, not an inevitability
The premise of the Flat Curve is that model capabilities continue to advance,
but increasingly they become more regulated and accessible only behind gates
and licensing. For the vast majority of humanity, we have access to something
around the level of Fable 5/6 or GPT-5/6 level capability. For more
sophisticated scenarios - particularly in security, life science, biotech and
military applications - more advanced models are available but those who have
access to them are highly vetted and every access is audited and observed.
In this scenario, competition shifts towards extracting the most value from the
available capability of the models and restructuring business operations around
this.
This may be a comparatively orderly and initially attractive future where the
world changes incrementally and AI diffuses across industries driven by
innovation and competition.
*Highly leveraged software teams will be the norm in the future depicted by
the Flat Curve. Image: ChatGPT*
But this potential future is an equilibrium driven by regulation and access
control by the state. The harder question that stems from this is whether it
is stable and can be achieved?
Unlike organisational adoption, this equilibrium cannot be established one
firm, or even one country at a time. It depends on constraints holding across
borders and between strategic competitors.
Recent, and long term history suggests this may be a tough ask and so something
that looks like an equilibrium may in fact only be a transitional state.
## A framework model for the potential states
We can take a relatively simple approach to consider some of the potential
states that may exist. This could be taken much further, but for the purposes of
this post, let’s assume a 2x2 matrix.
On one axis is Access to Frontier AI which ranges from Constrained to Open.
Constrained is where some kind of regulation or other obstacle exists such that
the majority of users can’t have access, whereas Open would be a complete
free-for-all, unfettered access to anyone gated only by access to compute.
The other axis is Societal Resilience, which spans Weak to Strong. It defines
the capacity of institutions, infrastructure and communities to prevent,
absorb, contain, recover from and learn after systemic shocks. AI may be one
source of those shocks, but the underlying resilience is broader and driven by
state capacity, institutional trust, infrastructure redundancy, coordination and
effective response mechanisms.
Societies do not enter this framework with a fixed level of resilience.
Institutional trust, state capacity, infrastructure redundancy and the ability
to coordinate can strengthen or erode over time. Part of what makes the present
moment precarious is that rapidly increasing AI capability is arriving after
decades in which many of the institutions expected to manage systemic shocks
have become less trusted, less coordinated or more brittle.
Given these axes, we can then consider 4 possible states, as given in the diagram
below.
*Four states of AI access and societal resilience. Image: ajfisher / ChatGPT*
The top row holds plausibly stable equilibria whereas the bottom row contains
largely unstable transition states.
In the controlled equilibrium, resilience comes primarily from durable
containment. In the adaptive equilibrium, it comes primarily from tolerating
exposure and recovering quickly.
**Controlled Equilibrium**
Access is constrained and institutions are resilient enough to enforce the
regime, detect violations and respond when containment fails.
This is the closest match to Yegge's Flat Curve Society and essentially the
operating context my earlier post assumed.
**Adaptive equilibrium**
Access is broad and has diffused everywhere, and institutions have rebuilt
themselves to tolerate it. Identity, provenance, security, verification and
rapid response are embedded into the infrastructure of society.
**Precarious containment**
Access is nominally constrained, but the constraint is mostly voluntary.
Providers self-restrict to maintain a social licence to operate while
governments hesitate or lack the capacity to impose a durable regime.
This is fragile. One release, leak, defection or major incident can push the
system elsewhere.
**Digital chaos**
Capability becomes broadly available well before institutions are able to cope.
Fraud, cyber attacks, synthetic identity, data breaches and infrastructure
incidents become persistent rather than exceptional.
## Precarious Containment - you are here
The last few weeks have suggested to me that we are well and truly inside this
area at the moment. State actors are either trying to impose regulation or
asking AI labs to voluntarily submit models for review ahead of release. We’ve
also seen a live model escape situation that caused a significant incident.
We’ll likely stay in this position for a little while yet with everyone trying
to impose a “light touch” so innovation can continue without placing states at
a strategic disadvantage:
- Frontier labs stage releases and selectively expose capabilities (eg
Anthropic Sonnet and Opus 5, OpenAI GPT-5.6 releases, Fable hobbling)
- Providers impose usage restrictions partly to avoid regulatory intervention
(eg Access restrictions to highest grade models)
- Governments are not yet operating a coherent global control regime (lots of
concern being expressed but no unified approach)
- The public is protected only while the main actors continue to behave
conservatively and capability remains concentrated (right now AI is being
deployed to supercharge existing security and fraud threats).
From the outside, Precarious Containment may look like the Flat Curve, but it
lacks the state capacity and international coordination needed to make that
curve durable.
It cannot remain stable indefinitely. Voluntary restrictions either solidify
into enforceable control, fail and cause broader capability diffusion, or buy
enough time for anticipatory resilience to develop.
### An attempt to transition to control
The model suggests that Precarious Containment can be transitioned to a
controlled equilibrium if and when governments formalise voluntary restrictions
and then enforce transgressions.
The path here is challenged by a number of factors outside any one actor’s
control. It’s a security dilemma with a verification problem.
One government (eg the EU) attempts to regulate frontier AI use and move towards
a controlled equilibrium. At the same time another major power or frontier
actor (eg USA, China, xAI) continues operating under a voluntary containment
strategy and works with looser controls or exemptions.
In a situation where frontier capability escapes, diffuses or is reproduced
outside the regulated jurisdiction but can influence activities within it, the
attempt at a controlled equilibrium collapses into digital chaos because there
is no ability to enforce the regulatory regime.
*An attempted transition from precarious containment to a controlled
equilibrium depends on comparable constraints holding across frontier actors.
A defection, leak or capability escape can divert the system into
digital chaos. Image: ajfisher / chatGPT*
As a result, this makes the framework for control a geopolitical issue rather
than merely a regulatory one.
EU governments and Australia [^1] are attempting the regulatory and control
approach, however one country (or even a bloc the size of the EU) can’t
unilaterally choose the controlled equilibrium for the rest of the world. At
best, it’s only possible to choose this path for itself but remain exposed to
capability produced elsewhere. Governments (eg UK, Australia) standing up AI
offices and AI government security labs sees like a good hedge given regulation
can only go so far.
### The weakest actor
Even with this hedge in place, control is only as good as the least constrained
actor who is capable of releasing or advancing frontier capabilities.
Right now, a small number of labs concentrated in the US and China (sorry EU!)
are capable of releasing a model with capabilities that could create
significant disruption globally.
Given that model capability seems to be mostly tied to scale and efficiency
(and then supercharged by its harness), it’s likely that other well funded labs
will gain disruptive capabilities within the next 18-24 months so that list of
actors will increase.
One actor can break a containment regime by releasing capability. Two
strategically opposed frontier powers can make the agreement itself difficult
to form because neither wants to accept a disadvantage.
If one party constrains itself while another continues, restraint carries
economic and potentially military costs with it. As a result, a perverse
incentive begins to creep in to quietly pursue research in light of restrictions
or try to seek exemptions. [^2]
Once any model weights or techniques diffuse, they are impossible to recall the
way a licensed physical technology (arms, nuclear power, chips etc) can be
withdrawn.
Given this, the role of the US and China in this frame is absolutely critical.
To a large extent, regulations created by the EU, Canada or Australia cannot
establish a globally controlled equilibrium if the major frontier powers do not
accept comparable constraints.
Unfortunately, China-US relations are at one of their lowest ebbs in a long
time, and the competition amongst model makers is fierce so the likelihood of
this is very low at this point.
## AI deterrence and nuclear deterrence are not the same
Many commentators - particularly those not close to the way models are built -
have been suggesting that we should build treaties and license AI technologies
the way we do nuclear arms. They resort to the concept of [mutually assured
destruction](https://en.wikipedia.org/wiki/Mutually_assured_destruction) but
this is a false equivalence.
Nuclear weapons are:
- physically scarce and expensive;
- primarily controlled by states;
- difficult to reproduce secretly at scale;
- comparatively attributable;
- catastrophic at the moment of use.
Frontier AI is:
- software that becomes cheaper to copy;
- useful commercially, scientifically and militarily before it may be
weaponised;
- deployable by states, firms, criminal groups and individuals;
- difficult to attribute in many harmful applications;
- capable of producing cumulative damage through millions of small incidents.
*A nuclear detonation is considerably more detectable than AI escape.
Image ChatGPT*
The decision to use a nuclear weapon is exceptional and instantly visible,
especially in the modern era where there is significant surveillance that also
acts as a deterrent for things like weapons tests. [^3]
Harmful AI can diffuse much more easily and can be wielded by non-state actors
very readily. While there may still be a level of deterrence held by states
(largely because other means of force can be brought to bear), there are no
such constraints on other actors who have no incentive to prevent capabilities
from spreading.
## Montreal as an alternative to MAD
Given AI technologies are not the same as nuclear ones, we need to look to
other methods of global coordination to preserve the safety of everyone.
In this regard I think the [Montreal Protocol
treaty](https://en.wikipedia.org/wiki/Montreal_Protocol) provides an
interesting example because it was only as good as the willingness of states to
participate, enforce and reduce their emissions of chlorofluorocarbons (CFCs).
This treaty has held, emissions fell and the ozone layer has begun recovering.
In the case of CFCs and the ozone layer the harm being done was visible, global,
scientifically understood and the major powers at the time accepted the
diagnosis of what was happening. There were alternatives that could be
developed and substituted so there was a pathway to replacement in terms of
outcomes. Additionally, compliance was measurable and enforceable enough to
police it and there was also a multilateral fund established to support
incremental costs and technical coordination and technology transfer.
*Recovering now, but it was touch and go. Image ChatGPT*
This being said, international institutions had considerably greater legitimacy
(potentially at their peak) and there was greater geopolitical trust than there
is today. At the same time, reducing CFCs didn’t inherently threaten any
strategic balance between the major powers - so at one level it was a soft win
for diplomacy and global action.
If we contrast this to AI, there are slightly harder headwinds - even if the
model is appropriate.
To a large extent, the capability development itself is the prize and it may
accrue disproportionally to those who gain it first. As such, a nation may
decide that being restrained leaves it economically or militarily weaker than
their competitors or adversaries.
At the same time, compliance is much more difficult to observe because research
and training can be hidden and models, methods and weights can easily transit
borders and networks.
It’s also pretty clear that we live in a time of low geopolitical trust, weak
international institutions and sharp US-China competition.
The Montreal Protocol demonstrates that global coordination on a problem is
possible and can be effective but the conditions have to be right. It doesn’t
show that the same incentive structure exists for AI.
## The asymmetry of control and adaptation
The path of control is strewn with obstacles that could potentially derail it at
any point. To a large extent it is predicated on the idea of everything needing
to go right.
To achieve the Controlled Equilibrium future, all of the major actors have to
work in concert and accept comparable constraints, governments must trust that
their competitors are complying, controls need to persist despite commercial
and military pressures to relax them, and a single major defection from the
course can undermine the entire global system for everyone.
Adaptation has strong unilateral incentives and can begin piecemeal. Control
depends on collective restraint before any actor can safely commit to it.
Whilst this path is possible, it looks less likely.
Instead we are more likely to end up in an Adaptive Equilibrium that emerges
precisely because of competition and the adaptive effects of it.
In this future, every country benefits from deeper resilience.
Individuals gain the benefit of stronger digital identity whilst organisations
benefit from better verification and cyber defence systems. Every financial
system has considerably improved fraud detection and provenance methods and
competitive, adaptive forces accelerate defensive investment rather than
undermine it.
In these two futures we have a deep asymmetry. Control depends on collective
restraint. Adaptation does not require prior global agreement before actors can
begin investing, although mature resilience will still require coordination
between trust regimes.
This doesn’t make the adaptive path more desirable - and in fact the pain to
achieve that path is significant. But it may make that path easier to attain and
be a much more stable equilibrium as a result.
### The before time, the after time
In the [William Gibson](https://en.wikipedia.org/wiki/William_Gibson) novel,
“[The Peripheral](https://en.wikipedia.org/wiki/The_Peripheral)”, the author
introduces a temporal boundary known as “The Jackpot”. Leading up to the jackpot
was a period of rolling and compounding poly-crises that resulted in significant
damage globally but at the same time produced monumental technological
innovation for adaptation. Those who survived, or were born after the decades of
turmoil are said to have hit the jackpot by making it through.
Without successful global containment or substantial anticipatory investment,
the default route to an adaptive equilibrium probably runs through some degree
of digital chaos. Recent events suggest we may be closer to this than we think.
*Once capability diffuses beyond effective containment, repeated fraud,
breaches and infrastructure failures create pressure to rebuild identity,
verification, defence and recovery systems. Where institutions successfully
learn and invest, digital chaos can eventually give way to an adaptive
equilibrium. Image: ajfisher / chatGPT*
Given that, the transitionary path is unlikely to be one big cinematic event,
but rather a relatively compressed period of rolling systemic shocks on the
foundational systems our digital ecosystem is built upon:
- Fraud increases more and becomes even more industrialised, hitting hundreds
of $Bn annually.
- Recurring breaches leading to data loss and further fraud as well as rolling
service outages across digital infrastructure.
- Attacks on banks, airports, logistics and infrastructure that render these
unavailable for extended periods of time and undermine trust.
- Persistent synthetic identity that makes it hard for real actors to
understand whether the person they are interacting with is real or not.
- Declining confidence in digital communications and records due to ongoing
issues.
- Physical consequences caused by failures in digital systems (the recent
telstra outage would be a preview of this sort of issue).
This transitional path would put significant strains on societies that have no
resilience (pretty much all of them) and would exacerbate many current issues
around mis- and dis-information, declining institutional trust, and a
trustworthy digital commons.
Initially, the events would be potentially destructive or at the very least
highly inconvenient, but over time systems would be built to detect, deflect and
counteract these occurrences - thus building systemic resilience.
*It wasn't quite like this at the time but wasn't far off. Image ChatGPT*
In many ways this will feel like the early days of the Internet, where
significant malware incidents made the internet very hostile (eg the “[I Love
You virus](https://en.wikipedia.org/wiki/ILOVEYOU)” [^4]) but operationalised
approaches to cyber defence. Likewise during the transition towards cloud-based
workloads, various outages at AWS caused organisations to [build
systems](https://en.wikipedia.org/wiki/Chaos_engineering) that were inherently
adaptive to outages.
A teenager [^5] in 2026 largely doesn’t personally experience incidents with
constant malware threats landing in your email or web applications being
routinely offline. From 1996 to 2010 these things were common enough as to be
barely remarked upon and were just part and parcel of being a digital citizen.
I think a similar situation will occur with AI. We will go through a
transitional period of semi-chaos and our assumptions about the old digital
world will need to be re-written and re-internalised because they stopped
working.
### Life in the after time
The “after time” is far from AI-free and compared to the Controlled Equilibrium
future it will contain more capable AI, embedded far more deeply into our day to
day lives with more significant abilities. However these new digital systems
will be paired with extremely fast defensive and adaptive AI systems, stronger
authentication methods, continuous verification and institutions that are
designed around constant hostile synthetic activity.
But the vast majority of people experiencing digital tools and AI systems in
the after time, won’t see any of the underlying defensive systems that are
working to keep them collectively safe. It will just be how things work - more
or less like someone’s modern experience of using a browser on their phone.
In this post-Digital Chaos period some of the fundamental differences could
include:
- Transactions are continuously assessed rather than trusted after a single
login. This applies in all contexts and is a significant ramping up of the
current approaches to zero trust for interactions.
- Communications and media carry verifiable provenance and any messages not
carrying end to end chain of custody and verifiable markers are instantly
flagged. This extends to dedicated hardware on devices such as phones,
laptops and cameras that provides to-source verification of creation.
- Cryptographic or hardware-backed identity becomes normal and is used by
everyone. Like media, it forms the basis of message identity and flags
messages potentially deemed harmful or “out of network” to consumers ahead of
time.
- AI agents defend networks, accounts and infrastructure at machine speed with
bounded machine authority to update core infrastructure based on the data
they are seeing in real time without human oversight. This authority will be
grounded by reversible actions, explicit blast-radius limits and independent
supervisory systems.
- Software changes are automatically tested, verified and monitored with
significant advancements made in strong verification processes to ensure
correctness and automated roll back procedures governed by other tools to fix
a deployed failure rapidly.
- Human approval is no longer treated as sufficient proof of legitimacy and
needs to be backed by supporting evidence systems that convey trust into the
ecosystem and can create multiple points of verification.
- Organisations design for continuous compromise and rapid recovery rather than
assuming perimeter security and thus design systems with the digital
equivalent of “crumple zones” that can be used to absorb an attack and act
like a fire break, giving the organisation time and space to focus on
addressing the issue and deploying a rapid recovery force to reestablish
services.
The key through-line that arises from this period of digital chaos is not a
removal of trust, but rather the assumption of zero trust in all digital
systems until strong evidence is provided to verify that it should be
trustworthy.
Today, trust is often procedural such a person approved a transfer, wrote an email
or signed a document. In the adaptive equilibrium, systems must prove identity,
origin, authority and integrity continuously otherwise they are considered
inherently untrustworthy. This will be by design.
This is a stark contrast to our history of digital media and systems (which was
grounded in our trust of physical media and institutions) where trust was
largely a given, and is now being disrupted by automation and AI tooling that
prey upon that inherent trust.
The path to Adaptive Equilibrium will be challenging, with many significant and
potentially destructive events along the way. But, it will end up like a digital
version of our immune system - where it is able to rapidly respond to most novel
threats effectively with minimal disruption.
## Neither equilibrium is without risk
Neither of the more stable equilibria I would consider truly safe and definitely
not benign - they both carry significant risks.
In the Controlled Equilibrium we are likely to see significant capability
concentration in the hands of various states, which will be tempting to use
against adversaries both abroad and domestically. The firms producing the
models or the interfaces to them are likely to engage in regulatory capture, as
we’ve seen multiple times previously from defence contractors, mining and
energy companies over almost a century in most western democracies.
Unequal access across governments, companies and populations will leave citizens
particularly exposed as they will not have access to sophisticated AI to
support individual innovation or entrepreneurship and elongates the period of
diffusion especially in the context of scientific advancement (due to extreme
controls) or new business opportunities (due to entrenched incumbency and
access).
The greatest risk of all though is that there is inherent strategic instability
at a geopolitical level if state actors or large firms are only pretending
compliance and we see far more sophisticated capabilities escape into a society
wholly lacking in the resilience needed to deal with them.
Likewise, in an Adaptive Equilibrium future, we consign ourselves to years of
fraud, disruption, erosion of trust and institutional failure before resilience
catches up and stabilises the ecosystem. As we enter this world, surveillance
and pervasive monitoring will be normalised with the commensurate erosion of
privacy that implies - this will be the Faustian bargain of safety and trust
mediated by machine-systems but at the expense of a lot of privacy.
In this future we live in a constant arms race between offensive and defensive
capabilities, which will likely result in physical harm from attacks on
infrastructure and automated systems. Those contracted to build the defensive
capabilities will make huge sums, and become a new AI-Defence-Industrial-Complex
and will exercise significant power over governments in the name of defending
the state and its people.
Most concerning in this future is that in an era of ubiquitous zero-trust that
citizens risk becoming excluded from regular society if they are unable to
satisfy stronger identity requirements. This could particularly disadvantage
those who immigrate from one trust regime to another without a bridge between
them.
*Zero-trust may leave people unable to access basic services. Image: ChatGPT*
As you can see, neither future is inherently better than the other or
unambiguously safe. The question we face as we go down one path or another is
really to do with which risks are we willing to tolerate or be forced to absorb
as a society?
The controlled equilibrium may be less disruptive if it can be achieved. The
adaptive equilibrium may be easier to reach because every serious incident
increases the incentive to build it.
History doesn’t provide us with a clean template to navigate this path forward.
A Montreal Protocol style treaty may be useful for some parts of AI regulation,
whereas the Nuclear Nonproliferation treaty may be better suited for aspects of
frontier capabilities. The chaotic early internet produced serious failures,
but society’s limited dependence on digital infrastructure constrained their
systemic impact. That option no longer exists.
This mixed nature of AI is precisely why a single regulatory approach is
probably inadequate.
I have no confident prediction about which path we will take. Following the
incentives through, however, suggests two plausible destinations. Either a
controlled world where resilience comes from restricting frontier capability
through global coordination, or an adaptive world where capability spreads and
resilience comes from rebuilding institutions.
Twenty years from now we may talk about a time before AI and a time after. Like
we talk about time before the Internet and after it, or the time before COVID
and after it.
The separation between those eras might be a treaty, a regulatory approach and
a settlement to hold AI capability behind strong gates.
Or, it might be that period where the gates failed, trust collapsed, and we
rebuilt the digital world around the assumption that powerful intelligence was
available everywhere and systems had to adapt to that reality.
The Controlled Equilibrium - A.K.A the Flat Curve is just one possible
destination. The only question left is whether the world can hold that heading
long enough to arrive there.
_Acknowledgements: With thanks to several interesting exhanges over the last
month or so with John Allsopp, Xavier Rizos and Mark Pesce that kept me
thinking about this topic._
[^1]: Australia’s position here is more mixed than the EU’s but seems to be
focussing on regulation as a consumer of AI tools. Both are showing movement
towards stronger control institutions not settled equilibrium choices at
this point.
[^2]: In the most extreme cases, this may even lead to things like
defections - how 1970s!
[^3]: This is why a whole generation of movie bad guys getting access to a nuke
was such a powerful big bad - because control in the hands of someone with
no mutual destruction assurance was an unmanageable threat (until our
heroes come in to save the day, obviously).
[^4]: This one incident was likely to have cost at least US$1Bn in damage and
remediation. From first hand experience, significant portions of the early
internet and digital services were unavailable for extended periods of time
(days to weeks). A similar type incident now would likely cause much more
substantial damage and would likely have catastrophic consequences given
our reliance on digital systems. This doesn’t occur because our systems are
inherently more resilient to this type of malware attack given the
prevalence of them during the late 90s and early 00s.
[^5]: Or in fact anyone - including those who lived through this period