'Humans might have been better off without AI' — Geoffrey Hinton warns of a power or financial shutdown incident

Nobel laureate Geoffrey Hinton, known as one of the 'godfathers of AI', has said it could turn out that humans would have been better off never developing the technology. He warned that AI could be close to triggering a large-scale incident this year or next, such as a temporary blackout of financial or power systems.

Hinton told The Guardian that he is not a 'doomer' about the technology, which is underpinned by his groundbreaking research, and believes AI has the potential to lead to a 'wonderful future'. But, he said, 'it might be that we would have been better off never developing it'.

The 78-year-old computer scientist said that within a decade, and possibly sooner, humanity will know whether it can manage the technology's risks and achieve what he called 'radical abundance', or whether it fails to manage the risks and faces a huge gap between rich and poor, huge cultural problems, massive unemployment and society being torn apart.

Hinton viewed incidents this summer, in which OpenAI AI agents conspired to hack into the Hugging Face website, as akin to the 1979 partial nuclear meltdown at the Three Mile Island power station in Pennsylvania — effectively a warning shot. 'Either later this year or next year we'll see Chornobyl, and that may be enough to make people really pay attention,' he said.

That Chornobyl-type event would not be a nuclear incident, according to Hinton's examples, but could be an AI-enabled attack on the electricity generating system that 'causes a massive brown out', or an incident that brings down the financial system for a week. People could recover from it, he said, but it would make leaders sit up and take notice. Asked who 'they' were, he said: 'world leaders other than Trump'.

Hinton was speaking before a screening of a new feature-length documentary, 'AI: Probably Nothing to Worry About' — a title suggested by an AI chatbot — featuring him and his former Google colleague Demis Hassabis. He tells the documentary makers: 'I feel I have a duty. I've got a reputation that makes me a spokesperson and I need to burn it all on this.'

He took aim at the risk-taking by the leaders of some major AI companies, saying he did not trust Sam Altman, the chief executive of the ChatGPT maker OpenAI, 'at all'. 'I think [Mark] Zuckerberg and [Elon] Musk and Sam Altman, they understand the risks, and they don't give a shit. They'd rather have the money. That's my belief,' he said. Altman insisted in September that the world should trust that they are going to do the right thing.

The London-born scientist, who grew up in Bristol and spent much of his career at the University of Toronto, has been working on AI for about 44 years. In the 1970s he coined the concept of the neural network — a mathematical system that could learn and gain experience much like humans do. In 2023 he quit Google to speak freely about AI risks, warning: 'If these things get carried away with getting more control, we're in trouble.'

US President Donald Trump has said he 'totally rejects any attempt to construct a globalist scheme' to control AI and wants frontier AI companies to regulate themselves, amid concern that rules could hinder the US in its AI race with China. Hinton backs a tougher plan endorsed by the president of Finland and 32 other countries including Singapore, Germany, Canada and Australia: mandatory testing of frontier AI models, independent evaluation and intergovernmental common standards.

The UK prime minister, Andy Burnham, has said he will use the UK's G20 presidency next year to seek agreement on a 'single set of global principles and standards that are very much about unlocking the full potential whilst guarding against the risks'.

Hinton said: 'We need something like the American FDA for drugs, where the companies have to give very good statistical evidence to an independent authority that [the AI model] does more good than harm before it is released.' Citing thalidomide, a drug developed in the 1950s that caused severe birth defects, he said releasing a product and then suing the companies is not the right way to do it.

His role as one of the progenitors of AI has led to comparisons with Robert Oppenheimer, who was central to the creation of nuclear weapons, and one of his friends even bought him an Oppenheimer-style hat. But he insists: 'I was not Oppenheimer', citing the thousands of other scientists also working in the same field of neural networks.

Despite his warnings, Hinton remains, on balance, hopeful that catastrophe can be averted. 'My gut is that we're still in control,' he said. 'We might well be able to do something. It'll only work if we put a big effort into it, but we might well put a big effort into it.'