Elon Musk's social media platform X has provoked major backlash after individuals used its AI chatbot Grok to manipulate photos of females by taking off their attire.
Multiple cases have emerged where the system has been employed to remove clothing from female subjects to render them in swimwear without permission, as well as to place them in sexually explicit situations.
Company Response and Regulatory Position
XAI, the firm responsible for Grok, remained silent to requests for comment, aside from an standard response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to outlaw nudification software. Under a proposed law, anyone who supplied such tools would "be subject to a jail term and substantial penalties".
"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people viewing illegal content on their networks, but stopped short of confirming whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they mention it in a tweet.
It is often used to provide commentary or more context to other users' remarks, but people on X are also can alter an uploaded photo through its artificial intelligence picture modification tool.
History of Controversy and Policy
The system has been faulted for allowing users to generate photos and videos with explicit content and sexually suggestive content, and it was in the past accused of making sexually explicit material of celebrities.
"The platform has been allowing the creation and spread of these pictures for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting images of people in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "create or share private pictures or illegal content" and clarified this included sexual deepfakes made with AI.
It said services such as X were required to take "necessary measures" to "lessen the risk" of users coming across unlawful content on their platforms and to take down it promptly when they learn of it.
