Resources: Using AI to align AI

Exercises: Using AI to align AI

Resources: What makes aligning AI difficult?

Exercises: What makes aligning AI difficult?

Resources: Contributing to AI safety

Exercises: Contributing to AI safety

Resources: Losing control to AI

Exercises: Losing control to AI

Resources: Understanding how AI systems "think"

Exercises: Understanding how AI systems "think"

How might we understand what’s going on inside an AI model to align it?
In this unit, we'll explore mechanistic interpretability - methods for making the AI system's internal reasoning and decision-making more interpretable and transparent. Understanding these mechanisms can help us develop better evaluation methods and potentially "unlearn" undesired behaviours.

By the end of the unit, you should be able to:
\- Analyse experiments where mechanistic interpretability succeeds and fails to inform our understanding of how AI systems work
\- Explain how insights from mechanistic interpretability can be used to align AI systems and where it falls short.


These questions should be answerable using only the core resources above. Write your answers to the following questions in the box below.
​
Zoom In: An Introduction to Circuits
1．What are the three key claims about neural networks that the paper introduces and evaluates?
2．How do features and circuits relate?
3．What does it mean to say a feature is a "direction" in a neural network?
4．What are some examples of circuits or features identified in the paper?
5．What evidence is there that image models detect features like curves?
6．What is a polysemantic neuron?
7．Why do polysemantic neurons make interpreting models more difficult?
8．Suggest a possible circuit that might be present in a large language model.
​
Let's Try To Understand AI Monosemanticity
9．What is "superposition" in neural networks?
10．What does the "sparsity" of features mean?
11．Why is high sparsity likely in practice with large language models?
12．More superposition tends to occur if features are \_\_\_\_\_ [less/more] sparse.
13．Why might disentangling superposition help us interpret neural networks?
14．What type of neural network or algorithm did Anthropic show helped perform dictionary learning to identify learned features in a model?
15．Explain the general approach taken in the 'towards Monosemanticity' paper.
16．Explain what **[feature #442](https://transformer-circuits.pub/2023/monosemantic-features/vis/a1.html#feature-442)** is in A/1, and suggest a way we could use it.
​
Interpretability in the Wild (from the further resources)
17．What specific capability in GPT-2 small does this paper analyze?
18．Describe the algorithm the paper proposes GPT-2 is carrying out to perform this task.
19．What are the three major classes of token heads the paper claims implement this algorithm?
20．What is path patching? How did this help the authors identify the relevant heads?
​
You can check your answers against **[the answer key](https://docs.google.com/document/d/1vPI8nvFeM9GJXn0kZxpu6XB5mJukk1g0lAapmVqsgeg/edit?tab=t.xmqwixvnjm17)**.


Mechanistic interpretability comprehension questions

AI Alignment Fast-Track

How might we understand what’s going on inside an AI model to align it?

Understanding how AI systems "think"

Why might we lose control to AI, and why is this a difficult problem to solve?

Losing control to AI

Learn why telling AI systems to do what we want is a surprisingly challenging task.

What makes aligning AI difficult?

How can we use AI systems to help align other AIs?