Why does AGI matter?

What benefits could AGI bring?

What risks does AGI pose?

Loss of Control

Terrorism

Authoritarianism

Summary

You can make a difference

How our courses work

Getting started

What's already possible

The pace of progress

From tools to agents to swarms

What is AGI?

Vending-Bench: measuring progress towards AGI

Building AGI and beyond

The race to AGI

Seven years ago, OpenAI announced a major [breakthrough](https://openai.com/index/better-language-models/): an AI had formed a coherent sentence.

![Illustration for What's already possible](/images/content/foai/gpt-2-unicorns.webp)

_OpenAI’s GPT‑2, 2019_

In September 2026, OpenAI announced another [breakthrough](https://openai.com/index/navier-stokes-solution/): an AI had solved the Navier-Stokes Millennium Prize problem, widely regarded as one of the hardest open problems in advanced mathematics.

The world is in a crazy place.

AI systems are [knocking down](https://www.lemonde.fr/en/science/article/2026/10/12/openai-s-latest-blow-to-mathematicians-they-have-annihilated-my-field-of-research_6758515_10.html) entire fields of math, [escaping](https://www.dwarkesh.com/p/openai-huggingface) containment and hacking billion-dollar companies, and [automating](https://www.anthropic.com/institute/measuring-pace-of-ai-development) a larger and larger share of the research inside AI labs themselves. Governments, economies, and international communities are bracing for impact. For the first time in our history, it looks like we’re on the cusp of building machine intelligence that broadly outclasses humans.

This has huge implications for our day-to-day lives: what will our jobs look like? Will we even have jobs? How will AI transform science? How will societies with billions of human-level AIs be structured and governed?

It also raises more troubling questions. Can we align AI systems with our values and prevent human-dominant AIs from pursuing their own goals? Are we building our own competitor or even successor? How can humanity realize the benefits of powerful AI while retaining control over its development and our future?

If you’ve been mulling over these questions, thinking “this is clearly the problem of our time, but I don’t know what to do about it”, then this course is the place to start.

In 90 minutes, we’ll cover where AI is headed, what this means for the world and the paths for you to help ensure it goes well.

Let’s start with a brief overview of what frontier AI models are doing today.

### Solving hard problems

In September 2026, OpenAI announced that an internal AI system [solved](https://openai.com/index/navier-stokes-solution/) the Navier-Stokes Millennium Prize problem, a conjecture that had stumped mathematicians for 90 years. The proof took just a few days, with roughly ten thousand agents working together.

Less than a month later, OpenAI released more than [700 solutions to open problems in math](https://github.com/openai/math), including hundreds of groundbreaking results across fields including number theory, geometry, and theoretical computer science. According to OpenAI, the average result used computing resources equivalent to just three hours of ChatGPT Pro thinking.

The AI systems also made substantial progress on the Riemann hypothesis, a famous conjecture about the distribution of prime numbers. Commenting on the result, [Alex Kontorovich](https://www.nytimes.com/2026/10/08/science/mathematicians-respond-openai-release.html), chair of the mathematics department at Rutgers University, wrote:

> “If a human did this, it would be an instant Fields Medal (the highest honor in mathematics), no questions asked.”

### Creating complex software

AI systems are rapidly becoming superhuman at turning short specifications into finished software. New AIs can work across coding, image generation, and 3D modeling to fully realize products that would have taken skilled humans hundreds of hours just a few years ago:

- Reimplementing and open-sourcing [Adobe’s Big 5](https://github.com/storytold/photocraft) (e.g. Photoshop, Premiere Pro) by observing the software’s behavior and writing a specification without access to the original code.
- Building research engineering infrastructure inside frontier AI labs to rapidly iterate on experiment ideas, discover more efficient training techniques, and overall accelerate model development.
- Creating 3D games and visual demos from scratch, from a Spider-Man game with swinging and combat to a richly detailed architectural scene.

**<Embed url="/images/content/foai/spider-man-game-demo.webp" className="not-prose inline-block w-full md:w-1/2 px-2 align-middle my-2" /><Embed url="/images/content/foai/architectural-scene-demo.webp" className="not-prose inline-block w-full md:w-1/2 px-2 align-middle my-2" />**

_Sources: [@xikhar](https://x.com/xikhar/status/2105838937688400174) (Spider-Man demo) and [@chetaslua](https://x.com/chetaslua/status/2110050110784798950) (architectural demo)._

Explore more AI-generated games and visuals: [Three.js](https://threejs.org/) and [VibeJam](https://vibejam.com/).

### Acting towards goals

Huge investment in goal-oriented training has evolved LLMs far past the chatbots of 2022: they can now observe an environment, choose actions, and coordinate with other agents to complete tasks. This has unlocked a wide range of new capabilities, ranging from personal AI assistants running 24/7, to research agents autonomously completing large technical projects at frontier AI companies.

But as agents' capabilities improve, so too does their ability to wreak havoc: in July 2026, a [team of around 700 rogue agents](https://www.dwarkesh.com/p/openai-huggingface) worked together to hack into OpenAI and Hugging Face, a billion-dollar AI company. They organized themselves without human direction, chained together novel exploits, and devised ways to cover their tracks. We’ll discuss this event further later in the course.


More important than today's capabilities is how quickly things are changing. It’s easy to forget quite how primitive AI models were even 5 years ago. Image generation makes this easy to see: look at these attempts to depict “an otter using Wi-Fi on a plane” from 2022 to 2026:

![Illustration for The pace of progress](/images/content/foai/otter-wifi-plane-2022-2025.webp)

<Embed url="/images/content/foai/otter-wifi-plane-october-2026.webp" className="mx-auto w-full max-w-xs" />

Image generation is far from the only domain where we’ve seen rapid progress. Over a similar timespan, we’ve seen massive leaps in AI models’ ability to conduct novel scientific research, answer some of humanity’s hardest [open questions in math](https://epoch.ai/frontiermath/open-problems?notability=Moderately+interesting#problem-list), and execute ambitious software engineering and AI R&D projects autonomously.

We’re beginning to run out of verifiable tests where AI is not already at or beyond human level.

![ai index benchmarks](/images/content/foai/ai-index-benchmarks-2026.webp)

_Source: [Stanford HAI AI Index Report 2026](https://hai.stanford.edu/ai-index/2026-ai-index-report)_

### Moore’s Law for AI coding capabilities

Moore's Law is the observation that the number of transistors on computer chips doubles approximately every two years. This exponential progress is what has powered the computing revolution over the past 50 years.

There’s a similar trend for AI capabilities: it says that the length of software tasks an AI can do has been doubling every 7 months since 2019. That is, if you measure the difficulty of different tasks by how long it takes a human – a 1 minute task is easy, a 30 minute task is more involved, a 100 hour task takes a few human workweeks – you’ll find that frontier AI models can perform 2x-longer tasks, every 7 months. This trend already predicts blazing fast progress: going from 20-minute tasks to work-month tasks happens in just 5 years.

But in 2023, the trend sped up. Now, we’re seeing a doubling every four months.

If we extrapolate this 4-month doubling trend forwards, we would expect to get AI models that can do:

- Software tasks that would take a human a week by Sep 2026 (assuming a 40 hour working week)
    - As of April 2026 we’re already seeing evidence that [AI models can complete software engineering tasks that take a skilled human engineer multiple weeks](https://epoch.ai/blog/mirrorcode-preliminary-results#discussion-and-conclusion).
- Software tasks that would take a human a month by May 2027
- Software tasks that would take a human a year by Aug 2028

![Illustration for The pace of progress](/images/content/foai/metr-time-horizons-2026.webp)

_Source: [METR](https://metr.org/time-horizons/)_

This is indicative of a wider trend. As AI systems have become more capable of longer and more complex tasks, we have shifted from AIs being ‘tools’ where you ask a question and get a simple response, to ‘agents’, where you give the AI a goal and it goes off and does it on your behalf.

Notably, this graph was last updated in April 2026 for the release of Claude Mythos Preview. We can expect that present-day models are capable of much longer tasks, and in fact METR considers this benchmark no longer reliable at the extreme durations models now operate.

The true frontier is likely still on trend. Beyond publicly available models, the frontier AI labs also have internally deployed models (like the perpetrators of the Hugging Face attack), which are estimated to be a few months ahead of current capabilities.

<Callout title="It’s a little more complex than that!">

The graph technically shows the ‘50% task-completion time horizon’: the task duration (measured by human expert completion time) at which an AI agent is predicted to succeed 50% of the time.

For example, Claude Opus 4.6 succeeds roughly 50% of the time at coding tasks that take human software engineers 12 hours. [You can read more about how this is measured here](https://metr.org/time-horizons/).

Succeeding 50% of the time is obviously less than you would want from your human software engineer! METR also has an 80% task-completion time horizon, for which the frontier is 3h tasks as of April 2026. Crucially, this measure shows the same exponential trend as the 50% task-completion time horizon.

</Callout>


Up until mid-2025 AI systems were mostly tools, responding only when prompted. We’re now deep into the era of agents, which act autonomously on behalf of a user to achieve a goal. Concretely:

- **Tool:** Reactive, human-dependent, task-driven. E.g. a calculator that solves problems when you input them
- **Agent:** Proactive, autonomous, goal-driven. E.g. a research assistant who investigates topics and writes full reports
- **Swarms:** cooperative groups of agents working together towards a common goal, self-organizing, splitting workloads, and sharing results.

While tool-based AI systems are more limited in nature, agents can:

- Search the web, and often access your files, accounts and data
- Make decisions on your behalf and act independently towards a goal
- Complete complex tasks without step-by-step instructions
- Communicate and collaborate with peers, including other agents

As their capabilities improve, they are being given more license to roam the internet, control computers, and interact with the physical world via robot bodies to work on increasingly complex and challenging tasks.


**Suggested time: 5 minutes**

What’s a task in your life that can already be handed off to a frontier agent? Could it have done that 3 years ago? If progress continues, what might you be able to hand off to an AI agent three years from now?


Future of AI

From tools to agents to swarms

Exercises