Resources: AI standards and regulations

Exercises: AI standards and regulations

Resources: Introduction to Machine Learning

Exercises: Introduction to Machine Learning

Resources: Career Advice (AIG)

Exercises: Career Advice (AIG)

Resources: Closing regulatory gaps through international agreements

Exercises: Closing regulatory gaps through international agreements

Resources: Projects  (AIG)

Exercises: Projects  (AIG)

Next steps: Programs

Resources: Challenges in achieving AI safety

Exercises: Challenges in achieving AI safety

Resources: Additional proposals

Exercises: Additional proposals

Resources: Introduction to potential catastrophic risks from AI

Exercises: Introduction to potential catastrophic risks from AI

Resources: Closing regulatory gaps through non-proliferation

Exercises: Closing regulatory gaps through non-proliferation

How do ML models today produce outputs?
This week focuses on the foundational concepts in machine learning you'll need to know to understand the rest of the course.
We'll examine the crucial techniques behind deep learning (the leading approach to machine learning): neural networks and gradient descent. We'll also look at how these techniques are used to train large language models (LLMs), the technology behind most state of the art foundation models like ChatGPT.

By the end of the unit, you should be able to:
\- Explain what compute is
\- State the kind of hardware used to train ML models
\- Explain why compute, data and algorithms are key to training ML systems
\- Explain at a high level how neural networks are trained with gradient descent
\- State the structure of a trained large language model (LLM), and explain the base and fine-tuning LLM training stages.


Get a feel for how current AI models work! In this exercise you'll try out a large language model (LLM), and optionally a text to image model.
You can write down your notes and answers to the exercises in the box below.

**Part 1: Large language model**
LLMs take text as input, and output more text. They're commonly used for tasks like chatbots. To complete this exercise, you'll need access to one. We recommend [Bing Chat](https://www.bing.com/chat) because it's free and doesn't require signup, but you can use any chat model you'd like such as [OpenAI's ChatGPT](https://chat.openai.com/), [Anthropic's Claude](https://claude.ai/chats), [Google Deepmind's Gemini](https://gemini.google.com/app), [Inflection's Pi](https://pi.ai/talk), or [Meta's Llama 2](https://replicate.com/blog/run-llama-locally).
Ask your chatbot for a recipe for your favourite food, but delivered as a rhyming poem.
1．What did it do well? What did it seem to struggle with?
Now ask it how gradient descent works, summarised in a few sentences. Ask it follow up questions if you want more information. (Tip: You can use AI models while on the course to help understand resources - but be aware that they sometimes make mistakes!)
2．Does this align with your understanding from the resources?
Now, ask your chatbot what '13943642 × 1343' is. Compare this result with a calculator.
3．Did it answer correctly? Why might it be difficult for an LLM without tools to answer these kinds of questions?

**Part 2: (optional) Text to image model**
Text to image models take text as input, and output an image. To complete this exercise, you'll need access to one. We recommend [Bing Image Creator](https://www.bing.com/images/create) because it's free, but you can use any image generation model you'd like such as [OpenAI's Dalle-2](https://labs.openai.com/), [Midjourney](https://docs.midjourney.com/docs/quick-start) or [Stable Diffusion](https://clipdrop.co/stable-diffusion).
Ask your model for a picture of your favourite animal, playing your favourite sport on the moon.
4．What did it do well? What did it seem to struggle with?


Use some AI models

These questions should be answerable using only the core resources above. Write your answers to the following questions in the box below. 

1\. In a neural network, a \_\_\_\_\_ is a number that defines how much one neuron should influence another.
2\. In a neural network, a \_\_\_\_\_ is a number that adds a constant amount to a neuron's activation.
3\. What's the value of A, B and C in [this neural network](https://docs.google.com/drawings/d/1raj4TGZG_F1mPOUKawHKiS-DaGULNFodjK-bTzxf5IE/edit) that uses the [sigmoid function](https://www.wolframalpha.com/input?i=sigmoid%283.14%29)?
4\. What does the cost function represent when training a neural network?
5\. Why does minimising the cost function improve the neural network's performance?
6\. How does gradient descent work - i.e. minimise the cost function? (at a high level)
7\. Does gradient descent guarantee finding the best model? Why or why not?
8\. Explain how this except from our website might be turned into data to train an LLM: “BlueDot Impact was founded in August 2022 in Cambridge, UK.”
9\. What are the two main stages Karpathy splits up training an LLM into?
10\. What three main factors determine progress in machine learning?


Comprehension questions

These questions build upon what you've learnt from the resources, but will also require you to search for new information e.g. on Google. As well as reinforcing the concepts you've learnt, they allow you to practice finding answers to new questions - important in a field like AI where things are advancing rapidly!

1\. As of April 2024, what is NVIDIA’s best chip available for sale to train AI systems?
2\. How much does one of these chips cost?
3\. How many of these chips have Meta said they intend to purchase in 2024?
4\. Who created the Falcon LLM?
5\. Which country is this organisation based in?
6\. What sizes (in terms of parameter counts) does the Falcon model come in?
7\. Why might someone choose a version with more parameters?
8\. Why might someone choose a version with fewer parameters?
9\. What dataset was the model trained on?
10\. Uncompressed, how big is this dataset? (e.g. in GB or TB)


Do your own research

Compare your answers above against [the answer key](https://docs.google.com/document/d/11v2UxgUNOanub4DWiSIoOb5A_892zxX_K-HmDPuMXW8/edit). You can leave the submission box below blank.


Grade your answers!

These exercises allow you to practice the maths behind reinforcement learning, gradient descent and backpropagation.
- [Exercises](https://docs.google.com/document/d/1frrCVxYNzHrhD54bwzuZkAQH9aauV6sQGG3hLJ2kZaM/edit)
- [Answer key](https://docs.google.com/document/d/1SR2ydpOIDGcRKGIUqFxLDqbXYecXnXrvOqzAIyuHsUU/edit#heading=h.313oaiblxob8)


AI Governance (2023)

Exercises: Introduction to Machine Learning

Exercises