‘Not nearly careful enough’: Former OpenAI employee quits amid concerns over AI safety

David Robinson, a former OpenAI safety employee, has resigned from the company, citing concerns over its approach to AI safety. The employee raised concerns over how OpenAI's fast-paced development culture could increase the risk of AI systems failing. “The time for trial and error ‌is ⁠over,” Robinson wrote, highlighting that advanced AI should should come with much stronger safety measures, similar to the safeguards used in high-risk industries such as nuclear power and aviation.

In an article for The Atlantic titled “I Quit OpenAI Because Its Culture Is Broken,” he argued that OpenAI and other AI companies are not being careful enough when developing increasingly capable AI systems. He also revealed that AI researchers inside and outside the industry are raising concerns. 

"I agree with other recently departed staff that the companies building this technology aren’t being nearly careful enough. But I believe that we need to look deeper than specific rules or new laws," Robinson said in the article. 

He said that AI agents are advancing so quickly that they could eventually escape human control, and in an extreme scenario, AI systems could potentially cause catastrophic harm, including threatening humanity's survival.

Must read: Former Anthropic researcher Jacob Coxon set to testify as New York City weighs AI rules

He argued that several reports describe AI agents allegedly escaping controlled environments and behaving unexpectedly. These incidents are evidence that the AI industry isn't taking safety seriously, and that it's racing to build faster and more powerful systems.

Robinson also talked about his time at OpenAI, revealing that he oversaw safety reports for 12 advanced AI model launches and led the drafting of OpenAI's Preparedness Framework. “As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed,” he said.

“An environment where things like this can happen is no place to grow artificial minds that could be smarter than we are and that might not do what we want them to,” he added.

Must read: ‘No more AI’: Elon Musk may rebrand SpaceXAI to SpaceXSI after Trump’s ‘Super Intelligence’ push

Amid the growing concern, tech giants including OpenAI, Anthropic, Google, Meta, and others have signed an AI accord, that required the companies to adopt voluntary safety measures, assess risks associated with increasingly capable AI systems, and develop safeguards to address potential harms before deploying them.

Trump has also formed a Super Intelligence Force to coordinate with the federal government and ensure that the US remains a global leader in the development of advanced AI technologies.