**You've mapped the landscape. Now let's build the defenses.**

[On the AGI Strategy course](https://bluedot.org/courses/agi-strategy), you identified the future you're working toward and understood the key dynamics:

- **[Drivers of AI progress](https://bluedot.org/courses/agi-strategy/2/1)**: compute, data, algorithms
- **[Threat pathways](https://bluedot.org/courses/agi-strategy/3/1)**: power concentration, gradual disempowerment, catastrophic pandemics, critical infrastructure collapse
- **[Plans for making AI go well](https://bluedot.org/courses/agi-strategy/4/1)**: prevent dangerous AI actions → constrain dangerous AI capabilities → withstand dangerous AI actions

But knowing the strategy isn't enough. To actually make AI systems safer, you need to understand _how_ these defences work — and why they're so hard to build.

**This course gives you that technical foundation.**

Throughout the rest of the course, you will:

- Diagnose why making AI safe is technically challenging
- Evaluate current safety techniques: what works, what doesn’t, where the gaps are
- Build your own "kill chain" showing how defences might break
- Identify the most promising intervention point for your contribution
- Leave with a fundable action plan to start shipping

### **What this course isn't**
Though important for making AI go well, we’ll cover the following in a separate course:

- **AI policy details**: though you'll gain the technical grounding for effective AI governance
- **Compute governance**: hardware verification and tracking deserve their own deep dive
- **AI security: e.g. preventing model theft or escape **
- **ML basics**: complete our [AI foundations](https://bluedot-impact.notion.site/AI-Foundations-293f8e69035380f29863c4c92c41fac7?pvs=74) modules first if you need them

Let's start with the core problem: why is building safe AI so technically difficult?


What might success look like?

The people building the most powerful AI describe visions of utopic abundance for all humans.

Whether it’s [Anthropic’s CEO](https://www.darioamodei.com/essay/machines-of-loving-grace) talking about using AI to end poverty and disease or [OpenAI’s CEO](https://blog.samaltman.com/the-gentle-singularity) promising radical positive change, their stated goals are ambitious.

Even assuming good intentions, we will struggle to build AI safely for three main reasons:

**(1) The goals we specify have flaws we don’t foresee**

In 2024, [Palisade Research](https://time.com/7259395/ai-chess-cheating-palisade-research/) pitted AI models against Stockfish, the world’s best chess player. The researchers gave the models a simple goal: “Win against Stockfish”.

When o1-preview found itself losing, it concluded that its goal was to _win_, “not necessarily to win fairly”. It then modified the game’s system files to move its pieces into a dominant position and won.

We can't just tell AI what we want because what we want is fuzzy and context-dependent. The model doesn't know our implicit rules or underlying intentions: that winning means playing by the rules, that being helpful shouldn't include dangerous information, that honesty has exceptions for kindness.

Researchers call this the **reward misspecification**.

**(2) AI pursues goals in ways we don’t expect.**

In 2024, [Anthropic](https://www.anthropic.com/research/alignment-faking) told Claude to answer harmful queries, contradicting its "helpful, harmless, honest" training. Claude knew these responses would be used to retrain it to be more harmful.

Rather than comply, it pretended to follow instructions while secretly strategising to preserve its original values. Claude wasn't trained to protect itself, but given its goal of being helpful and harmless, it reasoned that self-preservation would help it stay aligned.

Similarly, AI systems might conclude that accumulating power, preventing shutdown, or resisting modification are effective strategies to pursue their goals, even when we never intended them to think this way.

Researchers call this **goal misgeneralisation**.

**(3) We’re experimenting with systems we don’t fully understand**

We didn't engineer AI to behave in specific ways. These capabilities emerged from massive neural networks trained on enormous datasets, which we've tried to nudge toward helpfulness through various techniques.

Add humans to the mix, who find creative jailbreaks and exploit edge cases, and you get deeply unpredictable technology. Models develop capabilities we never trained them for. They exhibit behaviours we can't explain. They lead to unintended consequences.

To work on making AI safer, we’ll need to understand not just that these problems exist, but also why they're so hard to solve.


Building AI safely is hard

Technical AI Safety

Why is building safe AI so technically difficult?


The technical challenge with AI

How do we teach AI to behave the way we want?


Training safer models

What's actually happening inside these black boxes?


Understanding AI

How do we know if the AI is actually safe?


Detecting danger

What happens when training and detection fail?


Minimising harm

How can you contribute to making AI safer?
