Exercises: Decomposing Tasks for Better Supervision

Resources: Decomposing Tasks for Better Supervision

Exercises: Interpretability 9

Resources: Interpretability 9

Exercises: Track 1: Eliciting Latent Knowledge

Resources: Track 1: Eliciting Latent Knowledge

Exercises: Literature Review or Project Proposal

Resources: Literature Review or Project Proposal

Exercises: Track 3: Science and Deep Learning

Resources: Track 3: Science and Deep Learning

Exercises: Reasoning about Reasoning

Resources: Reasoning about Reasoning

Exercises: Further Understanding the Alignment Problem

Resources: Further Understanding the Alignment Problem

Exercises: Preventing Misgeneralization

Resources: Preventing Misgeneralization

Exercises: Track 2: Agent Foundations

Resources: Track 2: Agent Foundations

Guez et al. (2019) give three sources of evidence that a network might be doing implicit planning. What other evidence could you imagine getting about whether or not this is happening in a given network?



Implicit Planning

**Problem:** models may very robustly learn to reason in misaligned ways, without us being able to study this in advance (because those reasoning skills will only develop once they reach general intelligence). 
The readings for this session focus on some more speculative ideas which may help us understand and deal with this possibility.
### Assumed background
One of:
1. [Optimal policies tend to seek power: NeurIPS spotlight presentation (Turner et al., 2022)](https://nips.cc/virtual/2021/poster/28400) (15 mins)
    1. Note from the author: reinforcement learning algorithms do not necessarily train systems which will maximize reward. The reward maximization frame that this paper adopts is a leaky concept in general, although it may be a useful heuristic in terms of background for this session's resources. For another perspective on reinforcement learning, see [Reward is not the optimization target](https://www.lesswrong.com/posts/pdaGN6pQyQarFHXF4/reward-is-not-the-optimization-target).
2. [Excerpt from Superintelligence: Instrumental convergence (Bostrom, 2014)](https://drive.google.com/file/d/1KewDov1taegTzrqJ4uurmJ2CJ0Y72EU3/view?usp=sharing) (15 mins)


Alignment 201

Exercises: Reasoning about Reasoning

Exercises