Resources: AI standards and regulations

Exercises: AI standards and regulations

Resources: Introduction to Machine Learning

Exercises: Introduction to Machine Learning

Resources: Career Advice (AIG)

Exercises: Career Advice (AIG)

Resources: Closing regulatory gaps through international agreements

Exercises: Closing regulatory gaps through international agreements

Resources: Projects  (AIG)

Exercises: Projects  (AIG)

Next steps: Programs

Resources: Challenges in achieving AI safety

Exercises: Challenges in achieving AI safety

Resources: Additional proposals

Exercises: Additional proposals

Resources: Introduction to potential catastrophic risks from AI

Exercises: Introduction to potential catastrophic risks from AI

Resources: Closing regulatory gaps through non-proliferation

Exercises: Closing regulatory gaps through non-proliferation

As noted previously, even if some governments enact excellent AI regulations, AI developers in countries that lack adequate guardrails could still cause global damage. Last week, we studied a relatively adversarial approach to mitigating this: non-proliferation. This week, we will study a more cooperation-oriented approach: states could establish international agreements on AI safety regulations. (These two approaches are complementary. For example, governments could cooperate with states that share an interest in guardrailing AI, while slowing the proliferation of advanced AI capabilities beyond this “cautious coalition.”.)
Historically, international agreements have taken many forms, and we can imagine a similarly wide range of possibilities for agreements on AI. They could involve just two states, several like-minded states, or many states. They could come about through enthusiastic cooperation, or through some states using their influence to incentivize others to participate. They could be formal or informal. States may expect others to comply due to common knowledge of AI’s risks, or because of formal procedures for verifying compliance. (Of course, these are all spectrums.) This week’s readings emphasize broader context on international diplomacy, as well as how enforcement/verification of compliance with AI safety agreements could be reliable, narrowly targeted, and privacy preserving. [1]
\-
[1] Reliable methods consistently identify harmful activities, enabling timely intervention. Narrowly targeted methods distinguish between acceptable and unacceptable AI activities, enabling beneficial AI activities to continue while overly risky ones do not. Privacy-preserving methods avoid revealing the sensitive information of individuals, companies, or governments.

By the end of the unit, you should be able to:
\- Describe, at a high level, a proposal for privacy-preserving verification of compliance with rules on large-scale AI development.
\- Identify policy tools that governments have historically used for nonproliferation of other technologies, as well as the outcomes of these nonproliferation efforts.
\- Explain various concepts relevant to international relations, such as the bureaucratic politics model, assurance games, and relative gains.


Read[ our guidance](https://docs.google.com/document/d/e/2PACX-1vTPYrsCZV95S03L5-cO-p10cHz1TBMxSr_onylcSdO7KHNVmBSeOGVTq1LDlO49tXHoAJ_oZh6D4pt1/pub) on how to approach writing to learn exercises.
The paper "What Does It Take To Catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring" presents one of the most fleshed-out plans to date for governing frontier AI development. However it often comes under scrutiny for its feasibility.
In this exercise practise critiquing high-level plans and considering their feasibility. We expect the resources from the course so far, as well as your background knowledge of other institutions will be helpful in answering this exercise.

**Prompt: which aspect of the proposal to achieve privacy-preserving, efficient verification of compliance with AI development regulations do you feel is likely to be the most challenging to implement?**
You may wish to pick one of the 3 stages of the plan (bolding added to indicate an implication of international relations), or take an alternative angle of your choosing:
1. Using on-chip firmware to occasionally save snapshots of the neural network weights stored in device memory, in a form that **an inspector could later retrieve **[and then verify]**.**
2. **Saving** sufficient information about each training run [e.g. the amount of training data, amount of compute, model size and the hardware used to replicate later] and **prove to inspectors the details of the training run** that had resulted in the snapshotted weights.
3. **Monitoring the chip supply chain** to ensure that no actor can avoid discovery by amassing a large quantity of untracked chips.
**Some things to consider:**
- Can you think of a similar policy or relevant organisation that might carry these proposals forward into standards or regulations?
- Which interest groups might pushback on particular aspects of the plan?
- Do you have any suggestions for further research, or mitigations to those challenges?
Responding to this exercise can be challenging without a background in international relations, arms control, and similar topics, or understanding of existing international institutions. It’s recommended to make assumptions in order to respond, and to be clear where you’re making assumptions so you can get feedback or investigate further later.


Feasibility of Compute Monitoring Proposal

Read[ our guidance](https://docs.google.com/document/d/e/2PACX-1vTPYrsCZV95S03L5-cO-p10cHz1TBMxSr_onylcSdO7KHNVmBSeOGVTq1LDlO49tXHoAJ_oZh6D4pt1/pub) on how to approach writing to learn exercises.
When considering how to design policy or cooperative endeavours to prevent uncontrolled proliferation of advanced Artificial Intelligence, governance researchers often turn to policies and models that have been successfully implemented in the past to seek ideas and analogy for what could work in the present, in the context of AI.
However no two technologies are exactly the same, so an important part of researchers’ jobs is to understand which parts are relevant, good ideas, and which are infeasible given the nature of Artificial Intelligence. Use the knowledge you’ve gained about AI from the course so far and what you’ve just read about nonproliferation efforts to do the following exercise.

**Prompt: Choose one policy tool that governments have historically used for nonproliferation of other technologies, and analyze its viability in the context of AI for preventing proliferation of advanced models in the coming decade. Explain any adaptations that would need to be made.**
**You may wish to consider:**
- What were the outcomes of the historical intervention? Did it achieve what it intended to, and what were its limitations or failures?
- Which aspects of the model might be analogous to, or desirable for, nonproliferation of advanced Artificial Intelligence, if any? Which aspects are infeasible?
As well as those ideas presented in the week’s resources, you may wish to search more widely and pick something we haven’t focused on.
We have produced a list of[ potential case studies for nonproliferation tools](https://docs.google.com/document/d/1PfXsg0yX-C5Bqki3VlP6U8LmIEOCXYcYUm93Cp-mYbg/edit?usp=sharing), which you may be interested in, in addition to the [resources](https://aisafetyfundamentals.com/governance-blog/historical-case-studies).


[Optional, recommended] Historical Nonproliferation Efforts

We include these prompts to help give you ideas about questions for further investigation. Use them as you wish.
**Prompts:**
- Considering "What Does It Take To Catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring," do you believe the proposed system for privacy-preserving, efficient verification of compliance with AI development regulations is feasible? What challenges might arise during implementation?
- After reading "A Tour of Emerging Cryptographic Technologies", discuss how these cryptographic methods could improve the verification process in AI regulation. Are there any risks or downsides associated with their use?
- Reflecting on the need for privacy-preserving verification, how can we balance the need for regulatory oversight and the protection of intellectual property or trade secrets in AI development, or even prevention of international espionage?
- Reflect on the history of nuclear nonproliferation as described in "The Nuclear Nonproliferation Regime." In which ways might lessons and policy tools from this era apply to the present-day challenge of AI governance, and in which ways are they less appropriate?
- Based on the models discussed, do you think current international relations (e.g., US-China relations) could hinder or facilitate the establishment of international agreements on AI safety?
- Would AI regulation agreements be more effective if they are formal and legally binding, or informal and based on mutual understanding? What would be the implications for compliance and enforcement in each case?
- Considering everything you've learned in this module, do you think a competition-oriented or cooperation-oriented approach would be more effective for ensuring global AI safety? Or do you believe a combination of both is necessary?
- Given the potential for AI to cause global damage, do you believe that international AI safety agreements should be enforced as strongly as nuclear nonproliferation agreements? Why or why not?
- Do you think an AI regulation agreement would be more effective if it began with a small number of states and then expanded, or should it start with many states? What are the pros and cons of each approach?
- Discuss how assurance games and relative gains, as concepts in international relations, might influence the dynamics of forming international agreements on AI safety.


AI Governance (2023)

Exercises: Closing regulatory gaps through international agreements

Exercises