We’ve tried to train safe models, but how do we know if we’ve succeeded?

There are two broad things we want to be able to detect through evaluations:

- **Capabilities**: can a model do X? It finds the upper bounds of what an AI can do, typically by giving the model tasks and seeing how reliably it solves them.
- **Propensities**: will a model do X? It tells us what the model’s behavioural tendencies are, typically by placing the model in different scenarios and seeing which behaviours it tends to exhibit more.


Evaluations: AI can, but will it?

But if dangerous capabilities or behaviours go undetected or an AI company releases the model anyway, it can still monitor how it is being used and flag suspicious behaviour.

As you’ll see, being able to detect dangerous capabilities requires you to have access to the model, which AI companies are wont to give for security and speed. The pressure for speed is especially dire when they are competing with other fast-moving companies for revenue share. Remember most of these companies are bleeding more money than they are earning.

The worry isn’t just with companies deploying dangerous models, but also about others attempting to steal the model or sufficiently powerful model attempting to escape.

This is the tradeoff that AI companies make is whether they are willing to delay deployment to ensure their models are safe.


Monitoring AI in the wild

**In this section, you’ll look into the safety practices of ONE AI company.**

The RSPs and system cards are the AI companies that are declaring how they’ve done this and what actions they've taken to prevent harm.


Safety practices at AI companies

- [Clio: Privacy-preserving insights into real-world AI use \ Anthropic](https://www.anthropic.com/research/clio)
- Anthropic:[ Threat Intelligence Report](https://www-cdn.anthropic.com/b2a76c6f6992465c09a6f2fce282f6c0cea8c200.pdf) (Aug 2025)


Option 1: Anthropic

- [How people are using ChatGPT | OpenAI](https://openai.com/index/how-people-are-using-chatgpt/)
- OpenAI:[ Disrupting Malicious Uses of AI](https://cdn.openai.com/threat-intelligence-reports/5f73af09-a3a3-4a55-992e-069237681620/disrupting-malicious-uses-of-ai-june-2025.pdf) (June 2025)


Option 2: OpenAI

- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations | Research - AI at Meta](https://ai.meta.com/research/publications/llama-guard-llm-based-input-output-safeguard-for-human-ai-conversations/)


Option 3: Meta

Technical AI Safety

How do we know if the AI is actually safe?


Detecting danger

Why is building safe AI so technically difficult?


The technical challenge with AI

How do we teach AI to behave the way we want?


Training safer models

What's actually happening inside these black boxes?


Understanding AI

What happens when training and detection fail?


Minimising harm

How can you contribute to making AI safer?
