The future could be amazing. You described it yourself in the first unit. That future might not happen by default. We have a tremendous opportunity and responsibility to make that future a reality for ourselves, everyone we know and love, and the rest of humanity.

If we don’t know what threatens that future, we’re flying blind.

Here, we’ll offer a sketch of the territory. This includes the things we’re protecting, who or what might attack them, and how they might try. 

This is not intended to be a perfect overview: treat it like a starting point for your thinking.


Map the threat landscape

**In this section, you will choose ONE threat pathway "option" to investigate further.**

Here's a list of actors we might be concerned about, in terms of them having the capability and/or the motivation to use AI to cause harm to humanity. 
- "Misaligned AI", i.e. AI systems that act against the interests of humanity.
- Powerful human actors, e.g. corporate CEOs, military and political leaders. 
- Malevolent nation states, e.g. North Korea. 
- Terrorist groups and doomsday cults. 

How might they cause catastrophic harm? 
- They could use information warfare and conventional military force to overthrow existing power structures like democratic governments. 
- They could do cyberattacks on critical infrastructure, including the water, energy and food systems. 
- They could design, build and release viruses into the population that are worse than SARS-CoV-2, leading to a global pandemic. 
- Or the default incentives for all actors lead to bad outcomes, even absent any malicious intent. 


Prioritising threat pathways

Based on your current understanding, which threat pathway do you think is of most concern?

Consider the combination of likelihood of the threat, and the impact it would have if it happened. 

We don't expect you to have detailed, well-thought out thinking on this at this stage - consider this the beginning of your thinking.


Zooming into one threat

For your chosen threat pathway, write a "threat scenario" sentence, using this template: 

> The [ACTOR] with [CAPABILITY] and [MOTIVATION] attacks [ASSET] by [ATTACK PATHWAY] in order to [OBJECTIVE].

Your exercise response to prioritise which of the next 3 sub-units to explore in more depth.


Write your own threat scenario

A small group gains control of frontier AI systems and uses them to manipulate elections through micro-targeted disinformation, fake evidence, and synthetic media so convincing that truth becomes unknowable. They deploy AI-powered surveillance that tracks every digital interaction, predicts dissent before it emerges, and automatically suppresses opposition through economic and social levers.

Within months, autonomous weapons patrol the streets, facial recognition determines access to resources, and AI advisors whisper into the ears of every official - all controlled by those who seized the models first. Democratic institutions become theatrical performances while real power flows through algorithmic systems accountable to no one.

Whether it's a government using AI to become permanently unremovable, a corporation achieving regulatory capture through superintelligent lobbying, or an AI system itself accumulating power while maintaining a facade of servitude, the result is the same: the permanent concentration of control in hands that will never voluntarily let go.


Option 1: Power concentration

A **kill chain** takes a "threat scenario" and breaks it into stages of execution. It traces how an attacker would actually proceed step by step, e.g. via reconnaissance, delivery, exploitation, persistence, action on objectives.

Your scenario describes _what might happen_, whereas a kill chain describes _how it would unfold in practice._ This enables us to spot choke points where defenders can intervene. 

Based on your scenario, use [this template](https://bluedot-impact.notion.site/kill-chain?source=copy_link) to complete this exercise. 


Step by step breakdown

As AI systems become cheaper and more capable than humans at nearly every task, companies face overwhelming pressure to automate. First it's customer service and data entry, then management and creative work, eventually even governance and resource allocation.

This could lead to humans losing economic leverage as their labour becomes worthless. Our ability to influence culture might degrade as AI generates all the content we consume. Our political power could dissipate as governments pivot to AI for their tax revenue, administration, and security, instead of human income tax, human civil servants, and human military personnel. 

We might wake up one day to find ourselves comfortable but irrelevant - living in a world we no longer understand or control, where the economy optimises for AI activities rather than human needs, and our preferences matter as little as those of retired plough horses.


Option 2: Gradual disempowerment

Modern infrastructure runs on outdated industrial control systems and unpatched software that were never designed for internet connectivity. AI could systematically map these vulnerabilities across utilities, hospitals, and supply chains, then coordinate attacks that would take human hackers months to plan and execute.

We've already seen previews: the Colonial Pipeline shutdown disrupted fuel supplies across the US East Coast, NotPetya malware accidentally escaped its target and caused $10 billion in global damage, and ransomware regularly forces hospitals to divert ambulances and cancel surgeries. But these were relatively simple attacks with limited coordination.

AI-enabled attackers could compromise the industrial control systems that run power plants, the SCADA networks that manage water treatment, and the logistics software that routes food and medicine - not enough to collapse civilisation, but enough to cause sustained disruptions that compound over weeks. When the power grid has rolling blackouts, water pressure drops, hospitals run on backup generators, and just-in-time supply chains fail, even a wealthy nation starts to fray at the edges.


Option 3: Critical infrastructure collapse

Advanced AI could dramatically lower the barriers to creating pandemic pathogens. Today, designing a novel virus requires rare expertise, expensive equipment, and years of work. In a few years, AI could walk someone through synthesising a pathogen more transmissible than measles and deadlier than rabies.

Unlike natural pandemics that evolve toward transmission over lethality, an engineered pathogen could be optimised for maximum harm - combining long asymptomatic periods with high mortality, resistance to existing treatments, or even targeting specific genetic markers.

The same AI capabilities that accelerate beneficial biomedical research - protein folding prediction, drug discovery, genetic engineering - could be misused by state bioweapons programmes, terrorist groups, or disturbed individuals to create humanity's final pandemic.


AGI Strategy

Option 1: Power concentration

Resources (50 mins)

Exercises