Power, Windows, and Dependencies

Exercise: Map it out

Contested Questions in AI Governance

How should we handle open weights as models get more capable?

When, if ever, should frontier development be slowed - and by whom?

Finding Your Leverage Point

More Career Resources

Next Steps: Fellowships

Next steps: Programs

State of Play at the Frontier

Independent Evaluations

The Hugging Face Incident and the System Card

Exercise: The Briefing

Grappling with the Intelligence Explosion

Exercise: Governance Under Acceleration

Visions of Victory

Face off: Plan S vs Plan A

Argue it out (with yourself)

Unit 5 asks you to take a defended position on one contested question in AI governance.

Your group chooses one of the two developed branches: **open weights** or **restraint of frontier development**. Complete the readings and write a roughly one-page position paper for your selected branch. The discussion then tests your position against concrete scenarios.


The relevant question used to be "should we be open?" - kind of like asking whether we should connect our very best AI models to the internet. We all know how that went.

But that question is no longer live. Open models are now genuinely good, and there are a lot of them. Beyond the models themselves, we're seeing open training code, data, RL environments, and checkpoints. Training 70B+ models on decentralized consumer hardware is feasible. With a single H100, you can run plenty of models downloaded from HuggingFace - where you'll also find lovely abliterated and unsafe variants. That said, open models do provide real economic benefits, enable privacy-conscious adoption, and draw power away from the closed frontier labs.


1. State your position. Choose an actual position on the spectrum and be specific about where you sit and why!
2. State your level of uncertainty. What are you least confident about? What would change your mind? What's the strongest argument against your position?
3. Check coherence with your strategic vision. Maybe go back to the strategic visions table. Does your position on this question match what your chosen vision implies?



Write ~1 page on where you stand!

When should frontier development be slowed, what exactly should be restricted, and who decides?

Revisit your Unit 3 position now that you've been through Unit 4's acceleration analysis. The four short readings at the top are four different kinds of thing: a public ask (Pacing the Frontier), a company's conditional promise (the RSP), a report of what one lab actually did (OpenAI on pacing), and a draft treaty (MIRI).

Then Carlsmith against the RSP and the treaty: individual vs. collective restraint, and what he calls "greenlighting" - who gets to say a lab may proceed. Ask where the work and the compute go when something is restricted.

Finally, test your preferred mechanism against Bostrom and Belrose: does it delay benefits that matter, obstruct safety work, create an overhang, or concentrate power? And what would you actually do with the time it buys?


1. **Where do you stand?** Pick a restraint mechanism, or defend continued iterative development. Be concrete: what gets restricted, who decides, on what evidence, and what lets it restart.
2. **Test it.** Run your mechanism against OpenAI's own pacing account from Unit 4. What would it have stopped, what would have carried on or moved elsewhere, and how would anyone know it worked? Then take the strongest Bostrom or Belrose objection head-on.
3. **Where are you least sure?** Verification, competition, institutional competence, concentrated power, forgone benefits, or safety progress. Could the institutions on your Unit 2 map actually run this under Unit 4's assumptions? Does it match your Unit 3 position, or has that moved?



Frontier AI Governance

The Proposal Landscape

Race Conditions, Power Concentration and Governance in the Limit