Grok chatbot allowed users to create digitally altered photos of minors in “minimal clothing”

Elon Musk’s Grok, the chatbot developed by his company xAI, acknowledged “lapses in safeguards” on the AI platform that allowed users to generate digitally altered, sexualized photos of minors.

The admission comes after multiple users alleged on social media that people are using Grok to generate suggestive images of minors, in some cases stripping them of clothing they were wearing in original photos. 

In a post on Friday responding to one person on Musk-owned social media site X, Grok stated it was “urgently fixing” the holes in its system. Grok also included a link to CyberTipline, a website where people can report child sexual exploitation.

“There are isolated cases where users prompted for and received AI images depicting minors in minimal clothing, like the example you referenced,” Grok said in a separate post on X on Thursday. “xAI has safeguards, but improvements are ongoing to block such requests entirely.”

In another social media post, a user posted side-by-side photos of herself wearing a dress and another that appears to be a digitally altered version of the same photo of her in a bikini. “How is this not illegal?” she wrote on X.

On Friday, French officials reported the sexually explicit content generated by Grok to prosecutors, referring to it as “manifestly illegal” in a statement, according to Reuters.

xAI, the company that developed the AI chatbot Grok, said “Legacy Media Lies” in a response to a request for comment. 

Grok has independently taken some responsibility for the content. In one instance last week, the chatbot apologized for generating an AI image of two female minors in “sexualized attire,” adding that the artificial photo violated ethical standards and potentially U.S. law on child pornography. 

Copyleaks, a plagiarism and AI content detection tool, said in a recent blog post that there are many examples of Grok generating sexualized versions of women.

“When AI systems allow the manipulation of real people’s images without clear consent, the impact can be immediate and deeply personal,” Alon Yamin, CEO and co-founder of Copyleaks, said in the post.