Your next steps

Choose your focus

Create your 1-pager

Next steps: Apply to roles

Next steps: Technical fellowships

Next steps: Policy fellowships

Next steps: Other fellowships

Next steps: Programs

How does AI think?

Interpretability in practice

Making AI go well

What might success look like?

Building AI safely is hard

What future do you want?

Assuming harm

Building defences

Break the kill chain

Evaluations: AI can, but will it?

How do AI companies test for safety?

Option 1: Anthropic

Option 2: OpenAI

Option 3: Google DeepMind

Option 4: Meta

Can we train AI to be safe?

Feeding AI ‘good’ data

Teaching AI right from wrong

More safety techniques

<Embed url="https://www.youtube.com/embed/RMB-7p\_cOnM"/>

So far, we've tried to train models to be safer, build evaluations to detect danger, and understand how AI thinks.

But every technique has failure modes. Models learn to deceive evaluations. Dangerous capabilities emerge unexpectedly. Bad actors find new jailbreaks. Edge cases slip through.

This section is about the last technical defences. **How do we minimise harm from an AI we have already trained?**


Now it's time to stress-test the defences we’ve covered against real threats.

**In this section, you will choose ONE threat pathway "option" to investigate further.** Then, you'll construct a detailed "kill chain" — a step-by-step breakdown of how a threat could unfold. You'll identify the specific capabilities an AI would need, map our defences, and discover where the critical gaps remain.

Here's a list of actors we might be concerned about, in terms of them having the capability and/or the motivation to use AI to cause harm to humanity.

- "Misaligned AI", i.e. AI systems that act against the interests of humanity.
- Powerful human actors, e.g. corporate CEOs, military and political leaders.
- Malevolent nation states, e.g. North Korea.
- Terrorist groups and doomsday cults.

How might they cause catastrophic harm?

- **[Power concentration](https://bluedot.org/courses/agi-strategy/3/3):** They could use information warfare and conventional military force to overthrow existing power structures like democratic governments.
- **[Critical infrastructure collapse](https://bluedot.org/courses/agi-strategy/3/6)**: They could do cyberattacks on critical infrastructure, including the water, energy and food systems.
- **[Catastrophic pandemics](https://bluedot.org/courses/agi-strategy/3/5)**: They could design, build and release viruses into the population that are worse than SARS-CoV-2, leading to a global pandemic.
- **[Gradual disempowerment](https://bluedot.org/courses/agi-strategy/3/4)**: Or the default incentives for all actors lead to bad outcomes, even absent any malicious intent.


Based on your current understanding, which threat pathway do you think is of most concern?

Then, write a "threat scenario" sentence, using this template:

> The [ACTOR] with [CAPABILITY] and [MOTIVATION] attacks [ASSET] by [ATTACK PATHWAY] in order to [OBJECTIVE].


Zooming into one threat

A **kill chain** takes a "threat scenario" and breaks it into stages of execution. It traces how an attacker would actually proceed step by step, e.g. via reconnaissance, delivery, exploitation, persistence, action on objectives.

Your scenario describes _what might happen_, whereas a kill chain describes _how it would unfold in practice._ This enables us to spot choke points where defenders can intervene. 

Based on your scenario, use [this template](https://bluedot-impact.notion.site/kill-chain?source=copy_link) to complete this exercise.


Step by step breakdown

List 3-5 specific technical capabilities or behaviours your threat requires. Be concrete about what an AI would need to be able to do.

E.g. Self-replication, goal persistence across instances, long-term planning and coordination, resource acquisition (compute, money), bioweapon knowledge/synthesis


Capabilities required for harm

Which capability/behaviour is the most important? If we prevented just this ONE, would the entire threat collapse?

Reviewing what you’ve learned throughout the course, determine how we could:

- Prevent the model from acquiring this during training
- Detect the capability/behaviour
- Constrain the model so it cannot use this to take dangerous actions


Technical AI Safety

Break the kill chain

Exercises