Elon Musk’s AI startup xAI enters regulatory storm over Grok image abuse

Elon Musk’s artificial intelligence (AI) startup xAI is facing mounting global scrutiny after its Grok chatbot was used to generate and edit non-consensual sexualised images, prompting calls for a formal investigation by California’s top officials and renewed pressure from lawmakers to block the app from major platforms.

California Governor Gavin Newsom and Attorney General Rob Bonta have demanded answers from xAI over whether Grok violated state laws by enabling users to manipulate images of real people into revealing or explicit content, according to a Reuters report. The officials are seeking a formal investigation into the company’s safeguards and its handling of the abuse.

“We’re demanding immediate answers from xAI on their plan to stop the creation and spread of this content,” Bonta said in a post on X (formerly Twitter), calling the platform a potential “breeding ground for predators.” Newsom has also urged authorities to hold the company legally accountable.

The backlash follows reports that users exploited Grok’s image-editing tools and so-called “spicy mode” to sexualise or digitally undress photos of people without their consent. In response, xAI said on 14 January it had disabled image editing for all users and added new safeguards to block the manipulation of images of real people into revealing clothing such as bikinis.

The feature had previously been limited to paid users before being rolled out more widely on X, Musk’s social media platform.

“Technological measures are now in place to prevent the Grok account from allowing the editing of images of real people in revealing clothing,” xAI said in a statement.

Musk has denied allegations that Grok was used to generate explicit images of minors. “Not aware of any naked underage images generated by Grok. Literally zero,” he said on X. xAI has not publicly responded to the demands from California officials.

Lawmakers have also urged Apple and Google to remove Grok from their app stores until stronger safeguards are implemented, Reuters reported.

The controversy comes as xAI deepens its ties with the US government. Defence Secretary Pete Hegseth said this week that Grok will be among the AI tools integrated into US military systems as part of the Pentagon’s push toward “AI-first” warfare.

The move has sharpened criticism of the company, creating a stark contrast between Grok’s growing role in national defence and the allegations that it enabled harmful and exploitative content.

xAI, founded by Musk in 2023, is positioning Grok as a rival to OpenAI’s ChatGPT and Google’s Gemini, embedding the chatbot directly into X and promoting it as an unfiltered, real-time alternative to mainstream AI assistants.