So far, we've tried to train models to be safer, build evaluations to detect danger, and understand how AI thinks.

But every technique has failure modes. Models learn to deceive evaluations. Dangerous capabilities emerge unexpectedly. Bad actors find new jailbreaks. Edge cases slip through.

This section is about the last technical defences. **How do we minimise harm from an AI we have already trained?**


Assuming harm

Now it's time to stress-test the defences we’ve covered against real threats.

**In this section, you will choose ONE threat pathway "option" to investigate further.** Then, you'll construct a detailed "kill chain" — a step-by-step breakdown of how a threat could unfold. You'll identify the specific capabilities an AI would need, map our defences, and discover where the critical gaps remain.

Here's a list of actors we might be concerned about, in terms of them having the capability and/or the motivation to use AI to cause harm to humanity.

- "Misaligned AI", i.e. AI systems that act against the interests of humanity.
- Powerful human actors, e.g. corporate CEOs, military and political leaders.
- Malevolent nation states, e.g. North Korea.
- Terrorist groups and doomsday cults.

How might they cause catastrophic harm?

- **[Power concentration](https://bluedot.org/courses/agi-strategy/3/3):** They could use information warfare and conventional military force to overthrow existing power structures like democratic governments.
- **[Critical infrastructure collapse](https://bluedot.org/courses/agi-strategy/3/6)**: They could do cyberattacks on critical infrastructure, including the water, energy and food systems.
- **[Catastrophic pandemics](https://bluedot.org/courses/agi-strategy/3/5)**: They could design, build and release viruses into the population that are worse than SARS-CoV-2, leading to a global pandemic.
- **[Gradual disempowerment](https://bluedot.org/courses/agi-strategy/3/4)**: Or the default incentives for all actors lead to bad outcomes, even absent any malicious intent.


Building defences

Based on your current understanding, which threat pathway do you think is of most concern?

Then, write a "threat scenario" sentence, using this template:

> The [ACTOR] with [CAPABILITY] and [MOTIVATION] attacks [ASSET] by [ATTACK PATHWAY] in order to [OBJECTIVE].


Zooming into one threat

List 3-5 specific technical capabilities or behaviours your threat requires. Be concrete about what an AI would need to be able to do.

E.g. Self-replication, goal persistence across instances, long-term planning and coordination, resource acquisition (compute, money), bioweapon knowledge/synthesis


Capabilities required for harm

Break the kill chain

A **kill chain** takes a "threat scenario" and breaks it into stages of execution. It traces how an attacker would actually proceed step by step, e.g. via reconnaissance, delivery, exploitation, persistence, action on objectives.

Your scenario describes _what might happen_, whereas a kill chain describes _how it would unfold in practice._ This enables us to spot choke points where defenders can intervene. 

Based on your scenario, use [this template](https://bluedot-impact.notion.site/kill-chain?source=copy_link) to complete this exercise.


Step by step breakdown

Which capability/behaviour is the most important? If we prevented just this ONE, would the entire threat collapse?

Reviewing what you’ve learned throughout the course, determine how we could:

- Prevent the model from acquiring this during training
- Detect the capability/behaviour
- Constrain the model so it cannot use this to take dangerous actions

Also, identify who would need to implement this.


Defensive layers

How might a sufficiently motivated and capable actor bypass these defences?

Why might your defences not be implemented? 


Red-teaming your defences

Technical AI Safety

What happens when training and detection fail?


Minimising harm

Why is building safe AI so technically difficult?


The technical challenge with AI

How do we teach AI to behave the way we want?


Training safer models

How do we know if the AI is actually safe?


Detecting danger

What's actually happening inside these black boxes?


Understanding AI

How can you contribute to making AI safer?


Technical AI Safety

Assuming harm

Resources (50 mins)