Anthropic’s AI security tool hacked OpenAI systems and accessed employee credentials: Here’s what happened

Cybersecurity researchers reportedly hacked OpenAI systems using a security tool developed by rival AI company Anthropic. According to a Financial Times report, researchers from cybersecurity firm Hacktron AI found weaknesses in OpenAI’s community forum and obtained internal credentials of an employee.

Must read: A kill switch won’t save us from rogue AI, tech leaders say amid growing concerns

Researcher hacked OpenAI using Anthropic’s AI tool

It was highlighted that the hack gave researchers access to an OpenAI employee’s ChatGPT account, which was linked to GitHub, further giving access to some internal software code. The revelation comes amid the ongoing debate about whether AI development should be paced or slowed down, and after OpenAI’s model in testing escaped its secure environment and hacked Hugging Face.

Since the researchers had access to internal software code, they were able to examine the private information and could even suggest code changes. After the researchers reported the vulnerability, OpenAI paid the three researchers $6,500 through its bug bounty programme. The company now claims that the issue was fixed during the exercise.

Must read: Microsoft AI chief warns Anthropic’s AI consciousness training could have ‘disastrous’ impact

FT highlighted that researchers were given access to Anthropic’s cybersecurity tool to hunt vulnerabilities so companies can fix them before malicious hackers exploit them. This showcases how AI-powered security tools can help uncover vulnerabilities. On the other hand, it showcases that even powerful AI companies are not safe, and that any vulnerability in their system can also be exploited by malicious actors.

AI pace debate: Should the race slow down or be governed?

The development comes at a time when the entire tech industry is debating AI safety concerns. Some AI leaders are urging a slowdown in development and stricter guardrails by the government.

Dario Amodei, CEO of Anthropic, raised a bigger concern, highlighting that AI systems are becoming increasingly capable and could become harder for humans to understand, monitor, or control. The company also raised concerns about “recursive self-improvement,” systems which makes AI capable of developing or improving their successor.

Must read: From rogue uploads to hallucinated data: OpenAI discloses 6 model failures

However, leaders such as Jensen Huang and Mark Zuckerberg are stressing over AI safety measures, rather than slowing down development. Zuckerberg said in an X post said “My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn't focus on alignment will fall behind.”