Resources: AI standards and regulations

Exercises: AI standards and regulations

Resources: Introduction to Machine Learning

Exercises: Introduction to Machine Learning

Resources: Career Advice (AIG)

Exercises: Career Advice (AIG)

Resources: Closing regulatory gaps through international agreements

Exercises: Closing regulatory gaps through international agreements

Resources: Projects  (AIG)

Exercises: Projects  (AIG)

Next steps: Programs

Resources: Challenges in achieving AI safety

Exercises: Challenges in achieving AI safety

Resources: Additional proposals

Exercises: Additional proposals

Resources: Introduction to potential catastrophic risks from AI

Exercises: Introduction to potential catastrophic risks from AI

Resources: Closing regulatory gaps through non-proliferation

Exercises: Closing regulatory gaps through non-proliferation

Having examined some safety risks of AI, we pivot to discussing how to address those risks. Specifically, we’ll focus on one prominent idea for addressing risks of catastrophic AI accidents and misuse: regulating industrial-scale AI development. To start, this week we’ll study basic context on AI standards and regulation, as well as what their substantive content should be (i.e. what should be required of large AI developers). In the two weeks that follow, we’ll consider how policymakers can ensure that AI safety rules are applied wherever they are needed, rather than only being applied in some of the relevant countries.

By the end of the unit, you should be able to:
\- Articulate some rules that could be set around evaluations of model capabilities and alignment (i.e. when evaluations would be done, and what AI developers would do based on evaluation results), reasons why such rules might advance AI safety, and reasons why they might be limited or harmful
\- Compare standards and regulations, including how they are established and what their consequences are
\- Compare the state of AI regulation in the EU, China, and US as of approximately mid-2023


Read [our guidance](https://docs.google.com/document/d/e/2PACX-1vTPYrsCZV95S03L5-cO-p10cHz1TBMxSr_onylcSdO7KHNVmBSeOGVTq1LDlO49tXHoAJ_oZh6D4pt1/pub) on how to approach writing to learn exercises.
So far we’ve covered a few different ideas for AI standards and regulations. Now bring together your understanding of AI risks and these approaches to formulate your current best-guess at a regulatory framework for AI, specifically focused on the _model evaluation_ mechanism.
In coming weeks we’ll look at ways to make plans like these more feasible, but we believe it will be a good learning exercise for you to attempt to come up with a plan from scratch at this stage in the course.

**Prompt: Develop a regulatory mechanism for the evaluation of models’ capability and alignment, with the intention of avoiding _development_ of unsafe models.**
**(Aim to write <= 500 words)**
This is a very high level question, and we expect your final response won’t be highly specific. The purpose of this exercise is to begin to think through the components that would be needed for safe governance involving model evaluations, and spark ideas for further, more specific investigation.
**Try to answer the following questions:**
- When would evaluations be done?
- Which stakeholders need to be involved, at which points?
- What would AI developers do based on evaluation results?
- What are reasons why such rules might advance AI safety, and reasons why they might be limited or harmful?
- What are some ways to mitigate those drawbacks?
Some things to consider:
- You may wish to start by stating your objective; this may help to evaluate the extent to which your plan states your objective.
- Don’t worry about starting at a very high-level. You can go back and fill in details as you iterate your plan.
- Are there mechanisms other than model evaluation that are necessary or more appropriate for risk reduction?


Design a Regulatory Framework

We include these prompts to help give you ideas about questions for further investigation. Use them as you wish.
**Prompts:**
- Compare and contrast the efficacy of regulations versus standards. Which seems most appropriate for regulating AI, to you, and why?
- Develop a regulatory mechanism for the evaluation of models’ capabilities and alignment, with the intention of avoiding _development_ of unsafe models. In what ways does your mechanism fall short of addressing the entire problem? Are there other approaches that seem more appropriate for risk reduction?
- Which of the approaches to AI standards and regulations in the US, UK and China do you think sets the best groundwork for reducing extreme risks from AI that we discussed in weeks 1 & 2?
- Debate the pros and cons of open source model development. What would your ideal balance of knowledge sharing vs non-proliferation be?
- Given your knowledge of the risks and some approaches to AI governance, where do you feel the biggest gaps are between existing regulations and ideal AI governance?
- Do you believe that the burden of designing and carrying out evaluations should fall on independent (for-profit or non-profit) evaluation organisations, or auditors that work for the state? Why? What potential benefits or challenges do you see with this kind of partnership?
- Which jurisdiction’s current approach to regulation is the most feasible for implementing an evaluations regime in that jurisdiction, out of the US, UK, EU and China?


AI Governance (2023)

Exercises: AI standards and regulations

Exercises