What drives AI progress?

What can increasingly capable AI systems do?

How could progress accelerate from here?

Pathways to Harm

Loss of Control

Authoritarianism

Biological Misuse

Bring the arguments together

What’s at stake?

What future do you want?

Competition and coordination problems

Responding to the Hugging Face incident

Orienting to the strategy

The technical challenge

Following the transition

Pacing and alternative strategies

Examining the objections

Your assessment of the situation 

Choose your focus

Go deep in one area

Make a plan

Create your 1-pager (optional)

Optional advice

Next steps: Deeper-dive courses

Next steps: Programs

**How could increasingly capable AI systems cause catastrophic harm, and what would have to be true for those pathways to occur?**

In Unit 2, you explored how AI capabilities could develop. In this unit, we will examine three pathways to catastrophic harm from powerful AI: humans losing control of AI systems, people using AI to establish enduring authoritarian control, and bad actors using AI to cause catastrophic harm (with a focus on biological risk). 

For each, your task is to understand the causal argument. 
- What capabilities are required? 
- Why would a system or person act dangerously? 
- What access would they need? 
- Why might existing constraints/defenses fail? 
- And how does the resulting harm reach a catastrophic scale?

By the end of this unit, you should be able to:
- Reconstruct concrete pathways from AI capabilities to catastrophic outcomes.
- Distinguish deliberate human misuse from loss of control, while recognizing their possible interaction.


**How could AI systems come to act against human intentions, and become powerful enough that humans cannot stop them?**

For this unit, _loss of control_ means humans becoming unable to reliably direct, constrain or stop AI systems. We are particularly concerned with failures that could cause massive harm ('catastrophe') or permanently disempower humanity ('doom').


**How could AI enable a small group of people to acquire power that others cannot effectively challenge?**

Human control over AI would not guarantee good outcomes. Systems could faithfully serve people who use them to dominate others.

This section focuses on the claim that AI could enable unusually extensive and enduring authoritarian rule. Some of these pathways could involve deliberate power grabs. Others may combine ordinary adoption with institutional changes and longer-running trends. 


**How could AI change the feasibility of causing a catastrophic pandemic?**

This pathway requires two distinct arguments: why deliberately caused biological harm could be extraordinarily severe, and how AI might change someone’s ability to cause it. 


Prepare a short contribution for your discussion group. Bullet points are fine. Aim for **200–300 words total**.

**_Reconstruct one argument_**
Choose the pathway you most want to scrutinize. Explain:
1. The catastrophic outcome being proposed.
2. The causal steps connecting AI capabilities to that outcome.
    1. Writing this out as an actual chain: X -> X -> Z could be helpful! 
3. The strongest evidence supporting this argument.
4. One condition/event that could interrupt it/block its path. 

Mark where your reasoning is most uncertain or speculative.

**_Assess it_**
What is your current view of the argument’s strength? What evidence do you have available? Distinguish its plausibility from your confidence that it will actually occur.

**_Prepare a question_**
Write the one question that would help you resolve your biggest uncertainty. 


AGI Strategy

Bring the arguments together

Exercises