'Poorly managed AI poses existential risk': Tech leaders warn UN, urge global regulation

Silicon Valley’s top architects stepped before the United Nations Security Council with a grim warning: without global safeguards, artificial intelligence could slip out of human control and imperil the planet.

Leaders of several major tech companies — OpenAI, Anthropic, Hugging Face — addressed the Security Council on September 23 regarding recent developments surrounding AI and spotlighted both the potential and the problems inherent in the emerging technology.

In an unprecedented high-stakes session at the UN headquarters, executives behind the world’s most advanced models urged global powers to establish binding international controls to curb the unchecked proliferation of autonomous systems.

DON'T MISS | Could AI really kill us all? Here’s what MIT researchers fear and what they don’t

“If managed poorly, I even believe AI could be a risk to humanity as a whole,” said Dario Amodei, chief executive officer of Anthropic.

Sam Altman, CEO of OpenAI, echoed the urgency of the moment, telling the council, “We could lose control of the future to AI.”

The testimony highlighted a growing consensus among technology leaders that individual corporations and nations cannot self-regulate an escalating technological arms race.

MUST READ | AI safety warning: Sam Altman, Elon Musk back Anthropic CEO’s call to slow down frontier AI. Why?

Both executives called on the Security Council to establish verifiable guardrails to prevent AI capabilities from getting too powerful to rein in, while ensuring that access to the technology does not become concentrated within a few nations or corporations.

The summit follows escalating warnings from global diplomats over autonomous weaponry. A day earlier, UN Secretary-General Antonio Guterres cautioned against the emergence of “killer robots.” Ukrainian President Volodymyr Zelenskyy noted the real-world immediacy of the threat, stating, “There is already a real possibility that AI, not only people, will begin to decide what happens on the battlefield. And we need peace before we reach that point.”

During his testimony, Amodei called on global leaders to explicitly prohibit the use of AI in building biological weapons, establish international protocols to verify model capabilities, and formulate baseline standards to evaluate potential losses of control.

United Kingdom Foreign Secretary Ed Miliband emphasised that governments cannot ignore the alarms raised by the technology's creators. Those tech leaders “are telling us in stark terms that it cannot be left to them to prevent disaster from befalling the world,” Miliband said. “We cannot ignore them because they are right.”

DO CHECKOUT | 'Market forces can't guarantee AI safety': Obama warns profit motives driving premature AI deployment

However, views on binding global governance remain split. White House Science Adviser Michael Kratsios urged caution against over-regulation that might stifle innovation or delay economic benefits, arguing that worries about control are “not a reason to pause its further development or to constrain it with new global governance structure.”

Fu Cong, Permanent Representative of China, rejected the notion of “banding together into a petty us-versus-them clique in the tech sector” to serve “certain major Powers’ attempt to maintain their technological edge through monopoly in their quest for digital hegemony”. He called on all countries to enhance cooperation, promote open-source ecosystems and work together for global scientific and technological advancement.