Power, Windows, and Dependencies

Exercise: Map it out

Contested Questions in AI Governance

How should we handle open weights as models get more capable?

When, if ever, should frontier development be slowed - and by whom?

Finding Your Leverage Point

More Career Resources

Next Steps: Fellowships

Next steps: Programs

State of Play at the Frontier

Evaluating Claims

The Evals Theory of Change

METR

Epoch and Jagged Capabilities

The System Card

Independent Investigation: The Hugging Face Incident

Exercise: Briefings

Grappling with the Intelligence Explosion 

Reframing the debate

Exercise: Governance Under Acceleration

Visions of Victory

Face off: Plan S vs Plan A

Argue it out (with yourself)

Optional but highly recommended

**This course is the spiritual successor to Units 4-8 of our old 12-week AI Governance course - Units 1-3 have been absorbed by the [AGI Strategy Course](https://bluedot.org/courses/agi-strategy). You can still access the old curriculum [here](https://bluedot.org/courses/governance/1/1).**

You’re coming in from the AGI Strategy Course - or somewhere equivalent - and know that AI governance matters. Now you will need to show up credibly in governance conversations. That means being able to consume lots of (technical) information critically, assess what it actually tells you, and translate it for people who need to make decisions - fast, accurately, and tailored to what they care about and what would change their mind in a way that matters.

This unit - and this course - work toward building this skill. For Unit 1 you will read the two main sources of capability information (lab disclosures in the form of model cards and independent evaluations), understand what they can and can't tell you, and turn that into briefs that move the right people in the right ways.

But first, let us establish a shared definition of what we mean by (Frontier) AI Governance. This course believes it is the practice of shaping how advanced AI is built and deployed to achieve good outcomes - it does this primarily through policy, institutions, norms, and relationships. It requires both analytical judgment (what would actually work?) and political judgment (what's achievable, and how do I help make it happen?) and is as much a research effort as it is an implementation effort. 

The field is characterized by a fair amount of disagreement over goals and methods. This makes it even more important to be able to thoroughly evaluate proposals and build the influence to move the ones one believes in. 

Fair warning: This course is intense, you will fight the readings and the clock. It will also teach you - in a [trial](https://ai.stewart.wtf/posts/bluedot-governance/) by [fire](https://heterotopiaai.substack.com/p/parrhesia-praxis-and-poetry-ai-governances) way - to read smart, to use AI well and get you to produce materials that decision-makers might actually use.  

_For feedback and questions, please email the Frontier AI Governance course lead [Josh Landes](https://www.linkedin.com/in/josh-landes12/) at [joshua@bluedot.org](mailto:joshua@bluedot.org)._

_You can find the course session plans (used in discussions) [here](https://docs.google.com/document/d/1MrvxcuyRYF8lWHh9tr7UFvJPUoYAa95GAYiSm5QM7VM/edit?usp=sharing)._

Changelog: 
- 03/14/2026
    - The course was updated to contain 6 units from 5 units after initial testing. 
- 04/05/2026
    - Unit 4 was fully updated 
- 04/13/2026
    - Unit 5 option on "When, if ever, should frontier development be slowed - and by whom?" was added

- 09/09/2026
    - Refreshed frontier evidence and the Astra/Hugging Face briefing case; added Plan S vs Plan A, Radical Optionality, and the CSET/OpenAI research-acceleration comparison.
    - Updated reporting-law and RSP readings, course guides, and the discussion template; added optional agent-coordination and career routes while preserving the six-unit arc.

Planned work
- Add additional contested questions


Throughout this unit - and, hopefully your future career - you'll encounter many claims about AI capabilities and risks. This template gives you an initial method for processing them. Apply it to the readings below and see if you find it useful. 

- Claim
    - What is being asserted? By whom? In what context?
- Evidence
    - What evaluations, baselines, or comparisons support it? How robust is the methodology?
- External validity
    - What does this not tell us about real-world interaction/ability? What gap exists between the test and the deployment context? Where are we looking to get to?
- Threshold relevance
    - Would this change a policy stance, trigger an oversight action, or cross a safety framework threshold? Does it lock us into a certain path?
- Missing info
    - What would you ask for in a hearing, RFI, or expert consultation? What's absent that you'd need to act?



This short explainer lays out a rough evals taxonomy: model safety evaluations vs. contextual safety evaluations, and what each type can and can't tell you. 

**How to read this:** Read it as someone who may soon need to decide whether an evaluation result should change policy.

**As you read, track:**
- What's the difference between knowing a model can do something and knowing it will do a thing? Why and when does this distinction matter? 
- What would you want to know about any evaluation before you trust its results?
- Pay special attention to the external row in the template. This reading is in part about the gap between what evals measure and what the real world looks like.



Now zoom in on one independent evaluation methodology. METR’s time-horizon metric uses the estimated time a human would take to complete a task, measured at a specified model success rate. It is not a measure of how long the AI ran, or a direct estimate of general workplace autonomy.

**How to read this:** Use the current chart and methodology. Compare results at 50% and 80% success. What changes when reliable completion matters more than occasional success? What tasks are included, and how representative are they of the work or risks you care about?

**As you read, track:**
- What claim does the evidence support, and which extrapolation would be overreach?
- What limitations, uncertainty ranges, and update dates matter?
- Where might jagged capabilities or deployment conditions break an extrapolation to longer tasks?
- Which result would change a governance decision, and what additional evidence would you require?


Epoch’s dashboards help you compare performance across models and benchmarks. Toner’s essay asks how to interpret uneven capabilities when their practical consequences depend on context, feedback, and institutions.

**How to engage with this:** Spend about 15 minutes exploring the dashboards, then read the assigned Toner essay.
1. Start with the Benchmarking Hub. Choose three frontier models currently shown and compare them across several benchmarks. Notice where rankings shift depending on what you measure.
2. Explore the Epoch Capabilities Index. What does a composite measure make easier to see, and what differences can it hide?
3. Use Toner’s distinction between uneven capabilities and slow progress to interpret what you found. Treat her examples as dated illustrations.

**As you read, track:**
- If you had 30 seconds to explain the ECI to a policymaker, what would you say? What would you leave out?
- How does knowing the cross-model landscape change how you read a single lab’s system card?
- Which apparently weak capability could still matter at scale, and which impressive result depends on a narrow context?


A system card is how an AI lab tells the world about its own model. It's the primary document you will encounter when trying to understand a newly released AI model. But it's also produced by a company with interests, audiences, and lots of choices about what to emphasize where. 

Use the selected sections of the GPT-6 Astra system card below. Note its publication date and subsequent revisions.

**How to read this:** Read it as someone who needs to brief someone important tomorrow! Pay attention to these two lenses: 
1. **The content:** What does OpenAI claim GPT-6 Astra can do? What evidence do they provide? What evaluations did they/others run, and what did they find? How does what you learned help you assess the evaluation methodology they describe? 
2. **The comms choices:** What gets prominent placement vs. what's more buried? What risks are discussed in detail vs. mentioned in passing? Where is the language precise and where is it vague? What would a careful reader notice is not discussed? 

**As you read, track:**
- What are the 3-5 things a policymaker must know from this document?
- What's the most important thing this document does not tell you?
- What do you think is missing? 
- Run the template from chunk 2 through at least one specific capability or safety claim.


The Hugging Face incident investigation adds a different kind of evidence: an independent assessment of agents’ behavior in an earlier incident. It helps you ask what an investigation can establish, what access it requires, and where its remit ends.

**How to read this:** Read METR’s core takeaways, scope and engagement terms, and limitations. Separate observed behavior from the investigators’ interpretations and from your own policy conclusions. The incident involved different models from GPT-6 Astra; use it to motivate questions about controls and evidence, not to claim that Astra committed the same actions.

**As you read, track:**
- What evidence supports the assessment of coordination between agents?
- What did the investigation leave unresolved or outside its scope?
- What information would a government or oversight body need beyond this report and the current system card?
- Which additional evidence, independent scrutiny, or deployment condition would change your recommendation?

The optional explainer offers orientation. The displaced corporate-promises reading remains available as a separate example of translating technical evidence into policy recommendations.


You’ve now read a developer’s system card, independent capability evaluations, and a bounded investigation of an earlier incident. Now it’s your turn!

Using the readings, write **three separate 150-word briefings** about **GPT-6 Astra’s release, in light of the evidence and unresolved questions from the earlier Hugging Face incident**. Each briefing is for a different audience with decisions to make.

Keep three things separate: claims and evidence about Astra, findings about the earlier incident involving different models, and your own inference. Identify what the audience should do or ask next, and which uncertainty would change that recommendation.


**For a congressional staffer** drafting questions for an upcoming hearing on AI safety.
- They are thinking: What should my boss ask about? What risks should Congress be aware of? What sounds important to constituents? 
- They have 5 minutes to read your brief before a meeting.



Brief 1

**For a policy lead at a competing AI lab** reviewing OpenAI’s GPT-6 Astra release.
- What they might want to know: What is OpenAI actually claiming vs. demonstrating? How does this compare to our approach? What does this mean for our safety framework? Are there evaluation choices that are generous to their model?



Brief 2

**For a journalist** writing a piece on the state of frontier AI capabilities. 
- What they care about: What's genuinely new here? What's the story for a general audience? What should readers worry about - or not worry about? Where is the hype and where is the substance? 



Frontier AI Governance

State of Play at the Frontier

Resources (1 hr 20 mins)