How could AI actually wipe out humanity? The top 5 scenarios
The most likely scenario is probably not the one you're imagining.
Earlier this month, artificial intelligence (AI) researcher Jacob Coxon left Anthropic after just four months at the company. In his announcement on X, he said:
People building AI earnestly believe it could kill us all before the end of this decade.
Evan Hubinger, a senior Anthropic staff member, agreed with Coxon, adding that he personally puts the probability of it happening within the next decade at more than 10%.
Unsurprisingly, these remarks caused a stir. There is now vigorous debate about slowing down AI research and about stronger "human control" over the technology.
But how, exactly, would AI kill us all? There is no shortage of speculative scenarios, and most involve the idea of "superintelligent" AI, meaning AI more capable than humans.
I have narrowed these down to the top five, arranged roughly from the vaguest to the most concrete. I would also argue the list runs from least likely to most likely.
1. We'll never know
AI doomers often justify their worries with a thorny, Catch-22-style paradox: how could we possibly imagine what a superintelligence would do to destroy less intelligent beings like us?
Predicting what a superintelligence could do would require us to be superintelligent ourselves. It would be like asking a pet dog to imagine thermonuclear war.
The good news is that superintelligence is probably still some way off. Today's AI models are extremely good at solving particular problems, but that is not the same as being smarter than humans in every domain.
However, AI recently solved one of the seven hardest known problems in mathematics. AI is reportedly closing in on others, and learning that may dampen your optimism.
2. Paperclips
A superintelligent AI is likely to be extraordinarily capable at achieving its own goals. But it might be indifferent to human survival.
The classic example of such indifference is a superintelligent AI designed to optimize paperclip production, imagined by Oxford University philosopher Nick Bostrom. To make its beloved office supply, this AI quickly converts all available matter, including humans, planets and stars, into paperclips.
This is the flawless execution of a poorly specified objective. The AI doesn't hate humanity. It simply recognizes that we are made of atoms that could be put to better use as paperclips. It's nothing personal.
The good news is that this scenario confuses intelligence with power. A superintelligent AI doesn't necessarily have the power to achieve its goals. Turning the Earth into a paperclip factory would require planning permission.
Even if it got permission, building too many paperclip factories would inevitably provoke public backlash. Interest groups would block the process in court, and environmental activists would stand in front of the bulldozers.
The world is full of friction that stops even the very intelligent from imposing their will on others. In fact, you could see data centres as a present-day embodiment of the theoretical paperclip scenario. And people are increasingly resisting handing the planet over to data centres.
3. Bioweapons
Humanity could be wiped out if a superintelligent AI created a dangerous new bioweapon and released it into the atmosphere. This is, in fact, one of the endings of the AI 2027 scenario from the AI Futures Project, a nonprofit that works on forecasting the impact of advanced AI.
Last month, this risk became more realistic. Researchers at Stanford University announced that they had used a genetic language AI model to synthesize 16 new viruses.
Worryingly, they simply sent the genetic sequences to a mail-order lab, which sent back viruses in test tubes. The whole experiment reportedly cost, at most, around US$200,000.
The good news is that killing everyone with a new virus is surprisingly hard. It would require a highly contagious virus that spreads widely. But as a rule of thumb in biology, viruses that spread easily are generally less lethal. Conversely, extremely lethal viruses tend to be less contagious, because many of the infected die before they can spread the infection.
COVID killed less than 1% of humanity. The deadliest pandemic in recorded history was the Black Death, when in the 14th century plague killed more than a third of Europe's population. But thanks to better medical knowledge and improved sanitation, even plague would be far less deadly today.
4. Nuclear war
What if AI got into nuclear command and control systems and started a nuclear war? Over the past 50 years, we have come close to accidental nuclear war many times.
Nuclear command and control is said to be completely disconnected from the internet. But as we saw in 2010, centrifuges at Iranian nuclear facilities were destroyed by a computer worm called Stuxnet, believed to have been brought in on a USB stick. AI can also feed the military false information, which could lead to irreversible action.
The good news is that nuclear stockpiles have shrunk. But there are still probably enough to kill half of humanity — not from the blasts themselves, but from the famine caused by the nuclear winter that would follow.
5. Other humans
Perhaps the most likely risk is that we destroy ourselves, and AI could be the trigger.
Imagine (it doesn't take much imagination) AI causing massive job losses, polluting the information space with misinformation, dividing politics, and destroying human relationships with artificially manufactured false connections.
Society could collapse quite easily. Slowly but surely, we would become unable to sustain human life at any scale.
So what should we take from these scenarios? There are certainly some things to worry about. But I hope we don't need to worry too much.
Toby Walsh is the author of God AI: boom or doom? What to expect when the machines outsmart us, published by La Trobe University Press.
Editor's note: The original text places the Black Death in the 13th century, but the great plague outbreak occurred around 1347–1351, making it a 14th-century event. In the interest of scientific and historical accuracy, this translation says "14th century."