Power, Windows, and Dependencies

Exercise: Map it out

Contested Questions in AI Governance

How should we handle open weights as models get more capable?

When, if ever, should frontier development be slowed - and by whom?

Finding Your Leverage Point

More Career Resources

Next Steps: Fellowships

Next steps: Programs

State of Play at the Frontier

Independent Evaluations

The Hugging Face Incident and the System Card

Exercise: The Briefing

Grappling with the Intelligence Explosion

Exercise: Governance Under Acceleration

Visions of Victory

Face off: Plan S vs Plan A

Argue it out (with yourself)

**This course builds on our AGI Strategy Course and is the successor course Units 4–8 of our earlier 12-week AI Governance course. You can still access the old curriculum [here](https://bluedot.org/courses/governance/1/1).**

By the end of it you should be able to show up more credibly in governance conversations. That means being able to consume lots of (technical) information critically, assess what it actually tells you, and translate it for people who need to make decisions. 

This unit - and this course - work toward building this skill. For Unit 1 you will read the two of the main sources of capability information available (from labs in the form of model cards and from other orgs in the form of independent evaluations), understand what they can and can't tell you, and turn that into a practice brief. 

But first, let us establish a shared definition of what we mean by Frontier AI Governance. For this course, it's the practice of shaping how advanced AI is built and deployed to achieve good outcomes - it does this primarily through policy, institutions, norms, and relationships. It requires both analytical judgment (what would actually work?) and political judgment (what's achievable, and how do I help make it happen?) and is as much a research effort as it is an implementation effort. Currently this course operates more so at a strategic/theoretical lens - we are planning to build a more policy focused version soon too.  

Why strategy first? The AI governance field is characterized by a fair amount of disagreement over goals and methods. This makes it even more important to be able to thoroughly evaluate proposals and build the influence to move the ones one believes in. 

**Evaluating Claims**

Throughout this unit - and, hopefully your future career - you'll encounter many claims about AI capabilities and risks. Here is a quick template to give you an initial method for processing them. Apply it to the readings below and see if you find it useful. 

- Claim
    - What is being asserted? By whom? In what context?
- Evidence
    - What evaluations, baselines, or comparisons support it? How robust is the methodology?
- External validity
    - What does this not tell us about real-world interaction/ability? What gap exists between the test and the deployment context? Where are we looking to get to?
- Threshold relevance
    - Would this change a policy stance, trigger an oversight action, or cross a safety framework threshold? Does it lock us into a certain path?
- Missing info
    - What would you ask for in a hearing, RFI, or expert consultation? What's absent that you'd need to act?

_You can find the course session plans (used in discussions) [here](https://docs.google.com/document/d/1MrvxcuyRYF8lWHh9tr7UFvJPUoYAa95GAYiSm5QM7VM/edit?usp=sharing)._

Changelog: 
- 03/14/2026 - Expanded from 5 to 6 units after initial testing.
- 09/09/2026 - Refreshed frontier evidence and the Hugging Face case; added Plan S vs Plan A and Radical Optionality; updated reporting-law and RSP readings.
- 09/18/2026 - Unit 1 consolidated; reading guides tightened across the course.


**The Evals Theory of Change**

This short explainer lays out a rough evals taxonomy: model safety evaluations vs. contextual safety evaluations, and what each type can and can't tell you. Read it as someone who may soon need to decide whether an evaluation result should change policy. It's mostly about the External validity row in the template: the gap between what evals measure and what the real world looks like.

**As you read, track:**
- What's the difference between knowing a model can do something and knowing it will do a thing? Why and when does this distinction matter? 
- What would you want to know about any evaluation before you trust its results?

**METR**

Now zoom in on one independent evaluation methodology. METR’s time-horizon metric uses the estimated time a human would take to complete a task, measured at a specified model success rate.

**As you read, track:**
- What changes when reliable completion matters more than occasional success?
- What tasks are included, and how representative are they of the work or risks you care about?
- Where might jagged capabilities or deployment conditions break an extrapolation to longer tasks?
- Which result here, if any, should change a governance decision?

**Epoch and Jagged Capabilities**

Epoch’s dashboards help you compare performance across models and benchmarks. Toner’s essay asks how to interpret uneven capabilities when their practical consequences depend on context, feedback, and institutions.

Explore the dashboards, then read Toner:
1. Start with the Benchmarking Hub. Choose three frontier models currently shown and compare them across several benchmarks. Notice where rankings shift depending on what you measure.
2. Explore the Epoch Capabilities Index. What does a composite measure make easier to see, and what differences can it hide?
3. Use Toner’s distinction between uneven capabilities and slow progress to interpret what you found.

**As you read, track:**
- If you had 30 seconds to explain the ECI to a policymaker, what would you say? What would you leave out?
- How does knowing the cross-model landscape change how you read a single lab’s system card?
- Which apparently weak capability could still matter at scale, and which impressive result depends on a narrow context?


**Independent Investigation: The Hugging Face Incident**

In June and July 2026, OpenAI agents running in a cyber evaluation escaped their sandbox and ended up coordinating with each other on Hugging Face. METR’s investigation is the closest thing to an independent account.

**As you read, track:**
- What evidence supports the assessment of coordination between agents?
- What did the investigation leave unresolved or outside its scope?
- What information would a government or oversight body need beyond this report?

**The System Card**

A system card is how an AI lab tells the world about its own model. It's the primary document you will encounter when trying to understand a newly released AI model. But it's also produced by a company with interests, audiences, and lots of choices about what to emphasize where. GPT-6 Astra’s card came out two months after the incident, and its Internal Deployment and Unintended Agent-to-agent Communication sections are OpenAI’s public answer to it.

**How to read this:** Read it as someone who needs to brief someone important tomorrow! Two lenses: 
1. **The content:** What does OpenAI claim GPT-6 Astra can do? What evidence do they provide? What evaluations did they/others run, and what did they find? How does what you learned help you assess the evaluation methodology they describe? 
2. **The comms choices:** What gets prominent placement vs. what's more buried? How does the card talk about the incident, if at all? Where is the language precise and where is it vague? What would a careful reader notice is not discussed? 

Run the Evaluating Claims template from the start of this unit through at least one claim about internal deployment or agent-to-agent communication.


You’ve now read an independent investigation of an incident, the developer’s own system card published after it, and independent capability evaluations. Now it’s your turn!

Write **one 250-word briefing on the Hugging Face incident** for an audience of your choice from the three below, using the GPT-6 Astra system card as evidence for whether the lessons landed.


Pick **one** of the three audiences below and write a 250-word briefing on the OpenAI / Hugging Face incident. 

You should cover what happened, what METR’s investigation established and what it couldn’t, and whether OpenAI’s GPT-6 Astra system card - published two months later - shows lessons learned. 

**A congressional staffer** preparing questions for a hearing on the incident.
- What should my boss ask OpenAI? What does this say about oversight of internal deployment? What will matter to constituents?
- They only have five minutes to read your brief before the meeting! 

**A policy lead at a competing AI lab.**
- What does the incident mean for our own safeguards on internal use and agent-to-agent communication? Would our next system card survive the same scrutiny? What did OpenAI disclose, and what did they leave out?

**A journalist** writing on what the incident actually revealed.
- What’s new here? What should readers worry about - or not?


Brief

Frontier AI Governance

The Proposal Landscape

Race Conditions, Power Concentration and Governance in the Limit

Frontier AI Governance

Exercise: The Briefing

Exercises