Why does AGI matter?

What benefits could AGI bring?

What risks does AGI pose?

Terrorism

Authoritarianism

Loss of Control

Summary

Three ways you can contribute

1. Use your career to reduce AI risk 

2. Advocate for sensible AI regulation 

3. Donate to impactful AI safety organizations 

Your AI Safety Action Plan

What can the most powerful AI systems do today?

The speed of progress is unprecedented

What does agency look like?

So... how do AI systems work?

What is AGI?

The Vending-Bench evaluation

How could AGI get built?

AI Agents at Work

Teaching AI the next action

After learning about AI's potential and the challenges we face, you might be wondering: "What can I actually do about this?"

The good news is that **there are concrete ways you can help shape AI's development.** While the technology feels abstract and controlled by a few large companies, there are meaningful actions available to everyone, regardless of your background, location, or resources.
## We believe there are three primary ways you can make AI go well:

<Embed url="https://res.cloudinary.com/dfibju3df/image/upload/v1754351170/FoAI\_Unit\_4\_Chunk\_1\_Image\_1\_scjexh.png"/>

You can take action on all three approaches or focus on the ones that align best with your situation and strengths. 

**Here's what makes this moment special**: We're at a pivotal point in AI's development where individual actions can genuinely influence its trajectory. This means that anyone who chooses to get involved becomes part of shaping humanity's most important technology.

**If you decide to contribute in any of these ways, we thank you for it**. Your involvement matters more than you might think.

This Unit will explore each path, help you consider which approaches resonate with you, and then guide you in creating your own plan for making a difference.


Working on AI safety is extremely neglected. Current estimates are that while hundreds of thousands of people are working to make AI models more advanced and integrated into our lives, **only a few thousand people are trying to make them safer.**


<Embed url="https://res.cloudinary.com/dfibju3df/image/upload/v1754439155/UX\_suggestions\_via\_Yanni\_2\_x4stg6.svg"/>


↑ For every person working on AI safety, there are probably 100x racing to build more powerful systems.

**This creates an extraordinary opportunity.** In few fields can individual contributions have such outsized impact: when the entire global workforce focused on safety could fit in a large conference hall, every person who joins this effort genuinely matters.

**There are several ways people can use their careers to help navigate the safe transition to advanced AI systems**, and the paths are more diverse than you might expect.

## **Here are some career paths in AI safety you could follow:**

### 💻 1. Technical Contributor

Example Backgrounds:
- Software & hardware engineering
- Technical research
- Cybersecurity

Path to Impact:
- Research, build and implement interventions that help humanity understand AI capabilities and behaviours, ultimately making them safe to use

---

### 🎓 2. Non-Technical Research

Example Backgrounds:
- Non-technical researchers
- PhDs, postdocs, professors

Path to Impact:
- Conduct economic, legal, policy & social science research on how companies, countries and the world should govern powerful AI systems

---

### 🛠️ 3. Generalists

Example Backgrounds:
- Consultants
- Business operations

Path to Impact:
- Scale the organizational impact of AI safety organisations by building efficient systems that help them focus on their mission

---

### 🔐 4. Functional Experts

Example Backgrounds:
- Communications & marketing
- Program & product management
- Human resources

Path to Impact:
- Bring industry best practices from your domain to fill specific but crucial skill gaps in young AI safety organizations

---

### 🚀 5. AI Safety Startup Founder

Example Backgrounds:
- Previous founders
- Early stage startup employees
- CEO's of startups

Path to Impact:
- AI safety is an extremely new field with enormous unmet needs, creating countless opportunities for mission-aligned entrepreneurs to build essential infrastructure and capabilities that the ecosystem currently lacks

## **Some next steps:**
- [Express interest](https://web.miniextensions.com/aGd0mXnpcN1gfqlnYNZc) to join one of our other courses
- Attend events to build your network and learn more about the field
    - You can find BlueDot community events [here](https://lu.ma/bluedotevents?utm_source=foai)
    - And a comprehensive (perhaps intimidatingly large?) list of all AI safety events [here](https://www.aisafety.com/events-and-training)
- Check out what jobs in AI safety are [open now](https://jobs.80000hours.org/?refinementList%5Btags_area%5D%5B0%5D=AI%20safety%20%26%20policy). Which ones excite you?
- If you feel ready for it, consider applying for career advice from [Probably Good](https://probablygood.org/) and/or [80,000 Hours](https://80000hours.org/speak-with-us/?int_campaign=2021-08__primary-navigation)
- Finish this course and join the Future of AI Community!



- Does the idea of directly working on AI safety feel motivating?
- What are some things you think you're especially good at, that might be neglected in AI safety?
    - Tip: focus on tasks or projects, rather than job titles or functions
- While at work, when are you most in flow or energized? What are you doing in those moments?


Part 1 - Know Yourself

- What jobs do you think you could become good at?
    - Tip: we find that people often think too narrowly about where they might have an impact. Make a long list!
- What are you most uncertain or confused about, with regards to having a career in AI safety? What are some steps you could take to become more certain?
    - Tip: join the FoAI community and ask for feedback!


Part 2 - Explore Possibilities

The free market is racing us toward AGI without proper guardrails. Unlike typical consumer products where we can "vote with our wallets," AI development is driven by [massive corporate investments](https://openai.com/index/announcing-the-stargate-project/) betting that speed wins the race. **The market alone can't solve this coordination problem.**

**We need sensible government intervention**, whether that's international coordination, a slowdown, a pause, or something else entirely. **But governments move slowly, typically acting only after problems become undeniable**. We can't afford to wait for an AI crisis to trigger action because by then it may be too late.

The good news? Your government cares about winning elections, and AGI might arrive within their next election cycle. **As citizens, you have real power to create the political pressure needed for action.**

## **Three ways to advocate effectively:**

**📞 1. Contact Key Decision Makers Directly**

Best For:
- Individuals ready to act now and comfortable acting alone

Path to Impact:
- Email, call and/or meet your representatives about AI safety concerns
- Respond to public consultations
- Sign and share petitions

Even one voice matters when few are speaking up! [Here's](https://controlai.com/take-action/world) an email template from [ControlAI](https://controlai.com/) you could adapt for your own message.

---

**📢 2. Join Specialist Groups**

Best For:
- Those wanting structured support or a community

Path to Impact:
- Connect with organizations like [ControlAI](https://controlai.com/),[ PauseAI](https://pauseai.info/),[ Encode](https://encodeai.org/),[ The Midas Project](https://www.themidasproject.com/) or[ The AI Safety Awareness Project](https://aisafetyawarenessproject.org/).
- These provide talking points, coordinate campaigns, and amplify individual voices into collective action

---

**🤝 3. Integrate with Existing Groups**

Best For:
- Community leaders and organizers

Path to Impact:
- Bring AI safety concerns to groups you already influence. These could include;
    - Parent associations worried about AI in schools
    - Labor unions concerned about automation
    - Privacy advocates fighting deepfakes

[The AI Safety Awareness Project](https://aisafetyawarenessproject.org/) has provided some [useful resources](https://aisafetyawarenessproject.org/next-steps/community-statement/) to help you get started.

## **Policies to consider advocating for:**

In the below drop-downs you'll find examples of International and Local (National) policies that many AI safety advocates support, but we encourage you to research and think critically about what approaches make sense to you!

<Callout title="Local (National) Policies">

**🏢 Launch a national [AI Safety or Security Institute](https://en.wikipedia.org/wiki/AI_Safety_Institute)**

Policy: Build your country's own capability to evaluate AI risks rather than relying entirely on foreign assessments or company claims.

Example: The [UK AISI](https://www.aisi.gov.uk/about) has worked with OpenAI to make their recent Agent model safer for public use.

---

**⚖️ Pass an AI Act with safety requirements**

Policy: Create clear rules requiring AI companies to test for dangerous capabilities and be transparent about risks before releasing their systems.

Example: The [EU AI Act](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai) sets out a clear set of risk-based rules for AI developers and deployers regarding specific uses of AI.

---

**⚖️ Make AI developers liable for harms**

Policy: Hold companies legally responsible when their AI causes damage, creating strong incentives to prioritize safety over speed.

Example: [California's SB 1047](https://en.wikipedia.org/wiki/Safe_and_Secure_Innovation_for_Frontier_Artificial_Intelligence_Models_Act), though ultimately vetoed, proposed requiring developers to implement safety measures and face liability if their AI models caused catastrophic harms without proper precautions, offering a blueprint for future liability frameworks.

---

**🦠 Strengthen biosecurity against AI threats**

Policy: Improve laboratory safety, disease monitoring, and pandemic preparedness to defend against AI-enabled bioweapon development.

Example: Both the [Biden](https://bidenwhitehouse.archives.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/) and [Trump](https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Action-Plan.pdf) administrations have prioritized AI-biosecurity risks, establishing frameworks for biological synthesis screening that demonstrate bipartisan support for defending against AI-enabled bioweapons.

---

**🛡️ Upgrade national cybersecurity for the AI age**

Policy: Prepare defenses against AI-powered cyberattacks that could target critical infrastructure like power grids and hospitals.

Example: The U.S. Department of Homeland Security has created [a specific workstream](https://www.dhs.gov/archive/news/2023/10/30/fact-sheet-biden-harris-administration-executive-order-directs-dhs-lead-responsible) to guard against AI-enabled cyberattacks and is developing advanced threat detection systems to protect critical infrastructure from increasingly sophisticated AI-powered attacks.

</Callout>

<Callout title="International Policies">

**🔍 Create an "[IAEA](https://en.wikipedia.org/wiki/International_Atomic_Energy_Agency) for AI Safety"**

Policy: An international agency to monitor and inspect AI development globally, preventing any single country or company from building dangerous AI systems unchecked.

Example: Research published in [Oxford Academic](https://academic.oup.com/ia/article/101/4/1483/8141294?searchresult=1) and [Nature](https://www.nature.com/articles/s41599-024-03017-1) has proposed adapting the [IAEA's](https://en.wikipedia.org/wiki/International_Atomic_Energy_Agency) nuclear safety regulatory model for AI governance, emphasizing the need for international cooperation and standardized safety frameworks to manage AI development risks.

---

**⚡ Establish global compute limits**

Policy: Set international caps on how much computing power can be used to train AI, similar to nuclear non-proliferation treaties, to prevent an uncontrolled race to superintelligence.

Example: Researchers at the Centre for the Governance of AI [have explored](https://www.governance.ai/analysis/computing-power-and-the-governance-of-ai) how governments could use compute allocation and access controls to influence the pace of AI development, including coordinated efforts to manage compute resources globally.

---

**🔬 Build a "[CERN](https://en.wikipedia.org/wiki/CERN) for AI Safety"**

Policy: Create an international research facility where countries pool resources and expertise to solve AI safety challenges together.

Example: The Centre for Future Generations has developed [a comprehensive institutional blueprint](https://cfg.eu/building-cern-for-ai/) for such a facility, detailing organizational design, legal foundations, governance structures, and funding models for international AI safety collaboration.

</Callout>


- Does advocacy feel like a motivating way for you to help  shape the future of AI?
- Do you belong to any communities or groups that might care about AI's impact?
- How comfortable do you feel speaking up on these issues?
- Would you prefer working alone or with organized support?


Part 1 - Reflecting on advocacy approaches

- Which of the above suggested policies resonate most strongly with you and why?
- Are you better positioned to influence local/national policies or to support international efforts?


Part 2 - Reflecting on your policy preferences

AI safety funding is dramatically outnumbered. According to [this](https://www.founderspledge.com/research/how-philanthropists-can-help-slow-the-race-to-dangerous-ai) Founders Pledge report, **for every $250 spent making AI systems more powerful, only $1 goes toward making them safer.** 

<Embed url="https://res.cloudinary.com/dfibju3df/image/upload/v1754438336/2501\_3\_qgr2po.svg"/>
↑ We're pouring hundreds of times more money into racing toward superintelligence than into making sure we survive the journey.

**This creates both an urgent problem and a massive opportunity for individual donors.** Your donations can help change this balance and support critical work that could make the difference between AI catastrophe and AI flourishing.

## **Here are three approaches based on your available time and expertise:**

**🔍 1. Direct Research & Giving**

Best For:
- People with time and expertise to deeply understand the space

Path to Impact:
- Research individual organizations using resources like this [map](http://aisafety.com/map)
- Develop your own thesis about what matters most
- Donate directly to organizations whose approach you believe in

---

**🎯 2. Project-Based Giving**

Best For:
- Those wanting to support specific, highly transparent and concrete projects, without needing to do extensive research

Path to Impact:
- Support specific AI safety projects or trusted regrantors through platforms like [Manifund](http://manifund.com/)
- See exactly where your money goes, choose your involvement level, and make smaller donations count

---

**🏛️ 3. Expert-Managed Funds**

Best For:
- Donors who want professional evaluation without doing their own research

Path to Impact:
- Donate to funds run by experts who evaluate opportunities full-time. We recommend:
    - [The AI Risk Mitigation Fund](https://www.airiskfund.com/)
    - [The Long-Term Future Fund](https://funds.effectivealtruism.org/funds/far-future)
    - [Founders Pledge Global Catastrophic Risks Fund](https://www.founderspledge.com/funds/global-catastrophic-risks-fund)
    - [Longview Philanthropy’s Emerging Challenges Fund](https://www.longview.org/fund/emerging-challenges-fund/)



- Is financial support one of the ways you like to make a difference?
- How much time can you realistically spend researching donation opportunities?
- What's your risk tolerance? Do you prefer established organizations or experimental approaches?
- Would you rather support one organization deeply or spread smaller amounts across multiple bets?


Reflecting on different approaches to donating

Now that you've explored the three main ways to contribute to AI safety, it's time to create your personal action plan.

The goal isn't to commit to everything at once. Instead, **think about which approaches resonate with you and what concrete steps you'd like to make**. Many people find they're most effective when they start with one area and gradually expand their involvement.
## Creating Your Plan
- Below are example goals to spark your thinking. **Pick 2-4 that feel realistic and customize them to fit your situation.**
- Try to make your goals specific, measurable, and time-bound. Instead of "research AI safety careers," commit to "spend 3 hours this Saturday identifying 5 specific organizations I want to learn more about." **The more concrete your plan, the more likely you are to follow through.**


Example goals for the next 1-5 months (if you’re new to AI safety):

- Spend 2 hours this weekend browsing current AI safety job postings on the [80,000 hours jobs board](https://jobs.80000hours.org/) to identify 3 roles that match my background
- Email my local MP by Friday using the ControlAI [template](https://controlai.com/take-action/world) as inspiration, mentioning my concerns about AI regulation
- Donate $200 to the [Long-Term Future Fund](https://funds.effectivealtruism.org/funds/far-future) by month-end
- Have 5 one-on-one video calls with people working in AI safety through the Future of AI community by end of January
- Bring AI safety concerns to my book club/PTA/union meeting in February
- Attend one in-person or online AI safety [event](https://www.aisafety.com/events-and-training) by month-end


Your Short-Term Plan

Example goals for 6-12 months from now:

- Apply to 5 AI safety organizations by July 2026, starting with technical contributor roles
- Apply to the upcoming [AI Governance course](https://bluedot.org/courses/governance) facilitated by BlueDot Impact 
- Join an [Apart Research hackathon](https://apartresearch.com/) to collaborate on AI safety research
- Increase donations to $500/quarter after researching 3 additional organizations


Your Medium-Term Plan

Future of AI

Explore the challenge of building safe AGI amid heated global competition.

How can I help make AI go well?

Learn what the most advanced AI systems can do today, and how they work.

What can AI do today?

Learn what AGI means, and why it might arrive within the next decade.

Grapple with how AGI could cure diseases and automate boring work, while also posing severe risks.

Future of AI

2. Advocate for sensible AI regulation

Exercises