Resources: Adversarial techniques for scalable oversight

Exercises: Adversarial techniques for scalable oversight

Resources: Reward misspecification and instrumental convergence

Exercises: Reward misspecification and instrumental convergence

Resources: Governance (Alignment 23)

Exercises: Governance (Alignment 23)

Resources: Task decomposition for scalable oversight

Exercises: Task decomposition for scalable oversight

Resources: Careers and Projects

Exercises: Careers and Projects

Next steps: Programs

Resources: Interpretability

Exercises: Interpretability

Resources: Agent foundations

Exercises: Agent foundations

Resources: Artificial General Intelligence

Exercises: Artificial General Intelligence

Resources: Introduction to Machine Learning (23)

Exercises: Introduction to Machine Learning (23)

Resources: Goal misgeneralisation

Exercises: Goal misgeneralisation

Machine learning has gone through a revolution over the last decade: almost all cutting-edge systems now make use of deep learning. 
This week focuses on the foundational concepts in machine learning you'll need to know to understand the rest of the course. If you’d like to learn about machine learning (ML) in more depth, see our 'resources' on this site.  Those already adept with machine learning may be able to skip this week entirely, or review some of the basics if there's an area you're less familiar with.
This week is designed to give you a high-level outline of the fields of artificial intelligence and machine learning, and the techniques used to train ML models. We'll examine the crucial techniques behind deep learning (the leading approach to machine learning): neural networks, gradient descent, and backpropagation. We'll also look at the types of tasks which machine learning is used for: supervised learning, self-supervised learning, and reinforcement learning.
For those with little background ML knowledge, this week’s core readings will take roughly 45 mins longer than in other weeks and more math-intensive than in other weeks. We recommend spending the time to work through them all, to provide a solid foundation for the rest of the course.
Optional additional exercises: here, answers here. 
<https://docs.google.com/document/d/1ChHiwLCDWpkwNDL77iRBc3D8tydXonJgaK2BvVYI3oE/edit?tab=t.0#heading=h.313oaiblxob8>
<https://docs.google.com/document/d/19AIWZKPqRoKYul2Pj_rF7wnT_3zLpBTebSiR-1ogtYE/edit?tab=t.0#heading=h.313oaiblxob8> 
We'll also go through the first part of the exercises in the synchronous sessions (below).
You can also find Richard's Intro to ML talk from a previous round of this course here. 
<https://www.youtube.com/watch?v=v5c2Ll9129k> 

By the end of the unit, you should be able to:
\- Explain the taxonomy of Artificial intelligence, Machine learning and deep learning
\- Be able to explain the following concepts to someone who's new to machine learning:
    \- Neural networks
    \- Gradient descent
    \- Backpropagation
\- Be able to evaluate a model, and discuss whether it was trained via one or more of the following tasks:
    \- Supervised learning
    \- Unsupervised learning
    \- Reinforcement learning
\- Understand broadly what transformers are, and how they're trained 


AI Alignment (2023)

Exercises: Introduction to Machine Learning (23)