Resources: Contributing to AI safety (ITAI)

Exercises: Contributing to AI safety (ITAI)

Next steps: Programs

Resources: What AI systems are people building and why?

Exercises: What AI systems are people building and why?

Resources: How AI systems work

Exercises: How AI systems work

Resources: The Promise of AI

Exercises: The Promise of AI

Resources: The Risks of AI

Exercises: The Risks of AI

During the course, you learned about the potential impact of transformative AI on the world. The next step will be figuring out how you can contribute towards protecting humanity from AI.
Figuring out what to do can be difficult, as there aren't many default paths to follow. This unit encourages you to think about your career and goals and propose your next steps after the course toward those goals, informed by the knowledge you've gained throughout this course.
The main focus of this week are the two exercises, which you can find after the resources section. Having any plan helps resolve decision paralysis, and you will likely revisit and adapt this plan as you take action and learn more. The important thing, for now, is to get started.

3 additional paths we currently offer after this course are: 
\- AI Alignment Fast Track which covers technical AI safety research agendas.
\- AI Governance Fast-Track which covers policy levers for steering AI development.
\- Writing Intensive to transform your learnings into a published article.

Please note that as of Feb 21st 2025 these are running on a self-study basis. Application are currently paused as we work on course improvements - - you can register interest in future rounds here!

By the end of the unit, you should be able to:
\- Evaluate relevant AI safety career opportunities
\- Take your next step in contributing to AI safety


To tackle a problem, it's important to understand it well. This **[writing-to-learn exercise](https://bluedot.org/blog/what-is-writing-to-learn/)** aims to help reflect on your uncertainties around what the alignment problem is, as well as develop more comprehensive threat models for misalignment scenarios. In the session, you'll discuss your answers to these prompts and other statements about AI safety with your cohort.
​
Spend ~30 minutes answering the question: **What are my key uncertainties about AI safety?**
​
You might want to consider:
- Are there any 'obvious' potential solutions to AI safety that come to mind? (We'd encourage brainstorming a long list first, before trying to analyse them)
- How likely do you think it is that human-level machine intelligence arises in the next 10 years? What about 100 years?
- Do you think AI systems that we build will actually pursue convergent instrumental goals (mentioned at **[13:16 in Intro to AI safety](https://www.youtube.com/watch?v=pYXy-A4siMw&t=796s)**)? Why and why not?
- Will AI systems be safe by default? Why and why not?
- Will we have one big AI system? Many copies of the same powerful AI? Lots of different AI systems?
- Who's intentions or which "values" should we be aligning AI systems with? How would you handle different stakeholders wanting to align AI systems with different intentions or values?
​
In particular, if you're convinced one way on a particular topic, we'd encourage really digging into the opposite argument. You're also encouraged to browse the optional resources (below), or do your own research to help fill in key gaps in your understanding.


Identifying uncertainties

Use a process similar to the one discussed in the [80,000 hours article](https://80000hours.org/career-guide/career-planning/) to think about your career in AI safety and write up your answers in the box below. This doesn’t necessarily have to involve switching roles, but could be about clarifying your _[theory of change](http://www.aaronsw.com/weblog/theoryofchange)_ in your current role.

1. **What’s your career stage?** Exploring, building career capital, or deploying your existing career capital?
2. **What’s your vision?** If you haven’t already, sketch out your best-guess shortlist of longer-term paths to aim towards and aspects of AI safety to work on.
3. **Now clarify: what’s the very next decision you need to make?** Then generate a big list of ideas for next steps. Lean towards including more rather than less, including some that seem like a stretch.
    1. Work backwards: what steps would most accelerate you towards your vision?
    2. Work forwards: what other interesting opportunities are you aware of?
4. **Now, make an initial guess at which 5–10 next steps are most promising.**
5. **Then make an initial guess at your:**
    1. **Plan A:** your top longer-term plan
    2. **Plan B:** nearby alternatives
    3. **Plan Z:** fallback options
6. **What are your most pressing key uncertainties about all the above?** What information would most change your rankings of options or your Plan A?
7. **How might you best resolve those key uncertainties?** If you have time, begin to look into these questions. If you want to know more about an area, who would you talk to, and what would you want to know? Ideally keep investigating until your best guesses stop changing.


Intro to Transformative AI

Next steps: Programs