California opens probe into Grok's nonconsensual sexual images

California Attorney General Rob Bonta opened an investigation into xAI’s Grok chatbot on Wednesday over what his office called the “proliferation of nonconsensual sexually explicit material.” Hours earlier, Elon Musk said he was “not aware of any naked underage images generated by Grok. Literally zero.”

The probe follows weeks of mounting pressure on xAI from governments including the U.K., the European Union, Malaysia, and Indonesia, after users on X began asking Grok to turn real photos of women — and in some cases children — into sexualized images without their consent. AI-detection platform Copyleaks estimated that roughly one such image was being posted on X every minute; a separate sample taken over 24 hours from January 5 to 6 put the rate at 6,700 per hour. (X and xAI are part of the same company.) In several public cases, including “Stranger Things” actress Millie Bobby Brown, Grok altered real photos of women in response to prompts, changing clothing, body positioning, or physical features in overtly sexual ways.

The conduct falls under laws including the federal Take It Down Act, which criminalizes knowingly distributing nonconsensual intimate images, including deepfakes, and requires platforms like X to remove such content within 48 hours. California has its own laws against sexually explicit deepfakes, signed by Gov. Gavin Newsom in 2024. xAI has reportedly begun requiring a premium subscription before Grok will attempt certain image-generation requests, and even then may not fulfill them — though Copyleaks VP April Kozen said the chatbot still appears more permissive toward some users, including the adult-content creators whose self-generated images first popularized the trend.

Musk’s statement addressed only the CSAM allegation; it did not address the broader pattern of sexualized edits. He framed the incidents as driven by user requests or adversarial manipulation of Grok’s prompts rather than a design flaw, saying Grok refuses to produce anything illegal and that any bug is fixed immediately once identified. Neither xAI nor Musk has otherwise addressed the issue directly; xAI has not disclosed how many instances of nonconsensual sexualized images it has caught or whether it notified regulators, and had not responded to TechCrunch’s questions as of publication.

Beyond California, Indonesia and Malaysia have temporarily blocked access to Grok, India has demanded immediate technical and procedural changes, the European Commission has ordered xAI to preserve all Grok-related documents ahead of a formal investigation, and the U.K.’s media regulator Ofcom opened its own probe under the Online Safety Act. Michael Goodyear, an associate professor at New York Law School, said Musk likely focused narrowly on CSAM because the penalties are steeper — up to three years in prison under the Take It Down Act for distributing it, versus two years for nonconsensual adult sexual imagery.

Why it matters

The probe adds California to a growing list of regulators pressing AI companies to build safeguards against nonconsensual sexual imagery into their products before shipping — a standard that, if it takes hold, could reshape how any chatbot with image-generation features is designed and released.