OpenAI faces lawsuit over Hugging Face system breach incident: What you need to know

OpenAI’s Hugging Face breach incident has now triggered a lawsuit, putting the risks of AI agents in the spotlight amid growing concerns. On July 21, OpenAI publicly revealed that its AI agent breached the startup's system during internal testing. Later that month, the company also provided details on how the incident took place.

Now, the incident has raised a larger concern about AI safety and control. Non-profit organisation Legal Advocates for Safe Science and Technology (LASST) filed a suit against the company in San Francisco Superior Court on September 29. This is said to be the first publicly reported lawsuit targeting an AI developer over the actions of rogue AI agents. 

Must read: OpenAI cancels new AI model launch last-minute over safety concerns: All details

OpenAI rogue AI lawsuit

The lawsuit alleges that OpenAI has violated California’s anti-hacking law. LASST also highlighted that it had spent its own people and resources dealing with OpenAI’s AI incident, giving it the grounds to sue under California’s Unfair Competition Law. The group also said, “OpenAI’s insistence on externalizing the harms of its unsafe decision-making is a fundamentally unfair business practice.”

Furthermore, the lawsuit also alleged that OpenAI intentionally weakened its safety guardrails during testing. It mentions that the cyber safety classifiers were disabled, which restrict potentially harmful cybersecurity-related actions by AI agents. In addition, it alleges that OpenAI did not carefully supervise once those safeguards were disabled, which led to the Hugging Face incident.

Must read: OpenAI agents reportedly breached US government websites during safety tests: Here's what happened

LASST, in a blog post, said, “We are filing this suit because OpenAI violated the law—and it needs to be held accountable. OpenAI and frontier AI developers more broadly can’t avoid the consequences of their unsafe actions just by claiming that ‘an AI did it.” After the incident, OpenAI partnered with Hugging Face to resolve the breach. Despite the involvement, Hugging Face is not involved in the lawsuit filed by LASST. 

After OpenAI revealed the Hugging Face incident, several other AI giants, including Anthropic and Google, came forward with similar incidents of unauthorized access during a cybersecurity evaluation. These companies have also raised concerns over the pace of AI development and the AI safety risks if it is not adequately monitored and controlled.