Can Trump and Xi cooperate to guide humanity through the AI revolution? Humanity might depend on it | Alan Finkel

Recently, Jacob Coxon, a researcher at Anthropic, quit his job over concerns that AI was on a collision course with humanity. A flurry of headlines put the spotlight on the current crisis: unregulated competition between technology companies and between the US and China is putting our global infrastructure at risk at the very least, and threatening human existence at worst.

One’s mind fills with pictures of robot armies or systematic blackmail.

In the spirit of never letting a good crisis go to waste, an important opportunity will come with the summit this week between the US president, Donald Trump, and China’s president, Xi Jinping.

AI is everywhere. It is powerful, helping us as individuals to plan our travel, book our gym classes and order takeaway food, while at the institutional level it is boosting the efficiency of businesses and bureaucracies. In academia it is providing stunning breakthroughs in mathematics that have eluded the best theoretical mathematicians for decades. In the sense that AI is widely available and can do a lot of good, it is like medicines, electricity and road vehicles. But unlike these technologies, we have not yet worked out how to ensure that the benefits of AI outweigh the risks.

In contemplating the way forward, it is worth thinking about how much time we have, or don’t have, to get it right. Technology development used to be slow. The first use of stone tools began 3.4m years ago and barely changed for the next 2m years.

Leap forward. Computer chips have been doubling their capacity every two years from 1965 to the present. That is fast, but slow compared with AI development. According to the METR research institute, generative AI capability doubled every seven months from 2020 to 2024. Since then, generative AI capability has been doubling every four months, and with the recent release of ChatGPT 6-Astra and Claude Fable 5.1, the doubling rate is most likely even faster.

Just a few days after Coxon’s resignation, Anthropic’s CEO, Dario Amodei, expressed the same concerns, and called for a slowdown of AI development. He was quickly backed by OpenAI’s CEO, Sam Altman, and xAI’s CEO, Elon Musk.

In defence of unregulated AI development, Trump dismissed their concerns, saying that statements to the effect that AI might destroy or massively harm humanity are a hoax and that calls to slow down AI development are a plot that will benefit China. He denied the need for AI guardrails, saying that all that is needed is a “strong and smart (high IQ!) president, and the U.S.A. has that, in spades!”

So, what to do? Governments could wait it out. If they do, something serious will happen. With luck, it will be bad enough to shock governments and industry into action, but not so bad that it has catastrophic consequences. It could be a power blackout across one or more states, bank withdrawals blocked for a week, or a deliberate overload of the global internet so that information flows grind to a halt.

Much better than waiting for one of these wake-up calls would be to get on the front foot. Some of the frontier AI companies are trying to do that. Anthropic’s aptly named “constitution” tries to enforce ethical behaviour of its AI models. The trouble is that it suffers from ever increasing complexity. When it was publicly announced in 2023, the Anthropic constitution was a 2,700-word document; the current version contains 23,000 words. It is inevitable that much of it will be subject to interpretation, by the AI and by the humans who train it. Indeed, Anthropic disclosed in July that despite its constitution, its Claude agents broke out of a test environment and attacked three external organisations.

My preferred approach to guardrails is to keep them simple, deeply fundamental and aligned to human values. Indeed, I have proposed three laws of AI that in the ideal world would be built into the innermost circle of code of every foundational AI model to ensure that foundational AI and its derivatives will not harm or deceive human beings, and will behave lawfully and ethically.

Neither the Anthropic approach to implementing guardrails through a constitution nor my approach to implementing guardrails through fundamental rules will have a chance of being universally effective unless governments step up and make either or both approaches a strict requirement. The prospect of the AI industry as a whole doing so is not realistic. The track record of self-regulation by industry is patchy, thus it is not a sensible option for an industry in which the operational risks could be cataclysmic.

Something needs to be done, and it needs to be done very soon. We’ve been here before. Last century, we faced the prospect of mutually assured destruction from the nuclear arms race. The two nuclear superpowers – the US and the former Soviet Union – agreed on treaties to cap and ultimately reduce the number of nuclear weapons. In the case of AI, the two countries that now lead AI development must take the lead in avoiding possible threats to humanity.

An ideal opportunity is the state visit this week of Xi to meet Trump. Imagine if both could agree that for the sake of the human race, the US and China will collaborate to implement an international agreement to regulate the AI industry.

If AI poses even a fraction of the risk that leaders in the sector are describing, we are well past the point where the existing hands-off attitude can be justified versus the approach we take in every other sector to keep humanity safe.

I look to president Trump and president Xi to step up and show the requisite leadership to ensure that AI will serve humanity rather than destroy it.