OpenAI warns of autonomous AI risks, calls for global rules before AI starts improving itself

After Anthropic’s CEO Dario Amodei shared an article on the growing risk of autonomous Artificial Intelligence (AI), OpenAI has stepped forward, sharing the company’s perspective on AI safety, research progress, and the need for global technical standards. The AI startup urged the US government to lead the effort in building a global framework for AI safety and technical standards.

OpenAI shared a blog post titled “Building standards for the next phase of AI,” where it highlighted the company’s core mission and goals for advanced AI systems. The company laid out three main objectives: Navigating AI progress by creating an automated AI researcher, delivering broad scientific and economic benefits through advanced AI, and providing personalised AGI for everyone.

Must read: AI safety or market control? Anthropic, OpenAI, Google SpaceXAI accused of slowing AI progress

OpenAI’s concerns for advanced AI systems

In the blog post, OpenAI stressed concerns about AI’s Recursive Self-Improvement (RSI) capabilities. While acknowledging AI progress, safety research, and defensive capabilities, OpenAI also highlighted that fully autonomous RSI should not be pursued without safeguards and human oversight.

“Fully autonomous RSI is not happening today, and we should not pursue it unless and until it can be done safely,” the blog post wrote. “Whether and how to proceed must depend on our ability to preserve human control and on informed democratic choices⁠(opens in a new window) about the benefits and risks.”

The blog also mentioned that if RSI happens without sufficient safeguards, AI could advance faster than humans can understand or supervise. It also pointed out that if AI starts doing more research and development work, humans may struggle to monitor or direct the process. This may lead to humans losing practical control over how AI is developed.

Must read: OpenAI, Anthropic and Google have been discussing AI safety and risks for weeks: All details

“From here, AI could become more dangerous, less aligned, and, on the whole, a danger to people,” the company said.

OpenAI’s plan for International standards

In the blog post, OpenAI also urged international standards to manage frontier AI development globally. It claims that a global standard will help prevent fragmented national rules, solve collective action problems, and address uneven technical capacity across countries.

“Standards can create shared definitions of high-quality evidence and agreed-upon baselines for the rigor of technical safeguards,” OpenAI wrote.

OpenAI says institutions such as CAISI, state-level AI laws, and a federal AI framework can help regulate AI development, and that countries need to work together on common standards. The company says both open and closed AI models need to be covered by these discussions.