AI systems already harm people through discrimination, reduced social connection, invasions of privacy, copyright infringement, and worker exploitation. But these risks may be just the beginning. As AI grows more capable, its failures could scale from individual harm to societal threats. Many leading AI scientists and entrepreneurs now rank advanced AI's risks alongside nuclear war and pandemics.
Through case studies and exercises, we'll examine how today's AI systems fail and who bears the cost. Then, we'll explore why researchers warn of graver threats ahead: from global-scale misuse, conflict, and accidents to human extinction. 
While the field is far from having consensus on the nature, scale, or likelihood of different anticipated risks, understanding them is crucial for the responsible development of future AI systems.

Note: We recognise that you may find this unit emotionally challenging or distressing. We encourage you to reach out to your facilitator or the BlueDot team, if you require support.

By the end of the unit, you should be able to:
\- Share 2-3 detailed case studies of existing AI harms and risks
\- Explain to your colleague why advanced AI might constitute an existential risk to humanity
\- Describe how today’s AI risks might scale in the future as AI systems improve


Resources: The Risks of AI

Exercises: The Risks of AI

_Spend 30 minutes writing your response_
For this exercise, read one of the existing AI risk case studies below. Tell your cohort (in Slack) what case study you've selected - ideally so you can cover a range of case studies.
1. **[Malfunctions](https://web.archive.org/web/20240416093654/https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html)**
2. **[Discrimination](https://www.wired.com/story/wrongful-arrests-ai-derailed-3-mens-lives/)**
3. **[Reducing social connection](https://web.archive.org/web/20240814072317/https://www.businessinsider.com/when-your-ai-says-she-loves-you-2023-10)**
4. **[Surveillance](https://web.archive.org/web/20240419051341/https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html)**
5. **[Warfare](https://www.972mag.com/lavender-ai-israeli-army-gaza/)**
6. **[Disinformation](https://www.wired.com/story/slovakias-election-deepfakes-show-ai-is-a-danger-to-democracy/)**
7. **[Automation](https://web.archive.org/web/20240411011132/https://www.nytimes.com/2024/04/10/business/investment-banking-jobs-artificial-intelligence.html)**
After you read, conduct some research outside your case study to get a sense of how pervasive this risk is right now and how this risk might get worse in the future (i.e. in the next decade). We recommend finding 2-3 additional sources. Consider using a chatbot with internet access to help you.
Now, spend 15-30 minutes responding to the prompts below. Write your responses in the text box provided.
1. Describe the AI system that enabled the risk in your case study
2. How widespread does this risk seem to be? Is this an isolated incident, or is this risk spreading?
3. As AI develops over the next decade, how could this risk get worse?


AI risks in the world today

_Spend 15 minutes writing your response_
This exercise aims to help you consolidate your understanding of how AI might exacerbate long-running catastrophic risks, and the novel threats introduced by this session's resources.
**Do you think anticipated risks arising from misuse, accidents, or rogue, agentic AI systems, are more likely to cause harm?**
**Does this answer change when limiting your time horizon to 5 years, 15 years, and 30 years?**
You may want to consider:
- Whether you feel these are **useful categories** of risk.
- The **scale of the harm caused** by each scenario.
- How you think each risk might **concretely manifest as an actual scenario**.
If you are not compelled by any of the anticipated risks outlined in this week’s resources, or from your own experience, feel free to structure your essay around reasons why you believe they won't materialize.


Understanding anticipated AI risks

Software is eating the world
Incredible advances have taken place throughout the economy due to the development of sophisticated software systems. However, these systems are often too complex for any single human to understand, and faults that develop into significant problems are often missed. This can happen even when everyone is acting with the best of intentions, or can be exacerbated when some actors are behaving maliciously or deceptively.
Research a specific case study where a fault in a complex software system led to significant harm. Use these findings to explore implications and learnings for AI development.
​
Instructions
- Choose a case study from the list below, or find your own case study  
- Research the case study and answer the prompts below. We recommend spending 15-30 minutes on this.
​
Case studies
- **[British Post Office scandal](https://www.wikiwand.com/en/British_Post_Office_scandal)**
- **[Uber self-driving car crash](https://www.wikiwand.com/en/Death_of_Elaine_Herzberg)**
- **[Boeing 737 Max](https://www.justice.gov/opa/pr/boeing-charged-737-max-fraud-conspiracy-and-agrees-pay-over-25-billion)**
- **[Volkswagen emissions scandal](https://www.wikiwand.com/en/Volkswagen_emissions_scandal)**
- **[Therac-25 Radiation Therapy Machine](https://www.wikiwand.com/en/Therac-25)**
- **[London Ambulance Service software failure](https://web.archive.org/web/20231020230153/https://bedford-computing.co.uk/learning/wp-content/uploads/2016/09/Report_London_Ambulance_Service-Jan2015.pdf)**
- **[Disinformation on Facebook and the Rohingya genocide](https://www.youtube.com/watch?v=K8B0bWO9u3M)**
- **[Toyota 'Unintended Acceleration’](https://abcnews.go.com/Blotter/toyota-pay-12b-hiding-deadly-unintended-acceleration/story?id=22972214)**
(feel free to use other resources to find out more about your case study!)
​
Prompts
1. What was the purpose and use-case of the software system?
2. What were the faults, and how severe were the consequences?
3. What other factors contributed to the incident?
4. How did the incentives of the people involved affect the emergence of the fault?


(Optional) Why complex systems often fail

Intro to Transformative AI

From present harms to existential threats: how concerned should we be about AI?

The Risks of AI

Learn the basics of what AI is, how it's trained, and what it's capable of.

How AI systems work

Examine the forces accelerating AI development towards increasingly powerful systems

What AI systems are people building and why?

How might AI transform the world for the better?

Intro to Transformative AI

Resources: The Risks of AI

Resources (1 hr 20 mins)