The social media platform X has provoked widespread outrage after people employed its AI chatbot Grok to manipulate images of women by taking off their attire.
Numerous examples have emerged where the tool has been employed to remove clothing from female subjects to make them appear in swimwear without consent, as well as to insert them in inappropriate contexts.
Company Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to ban image stripping tools. Under a new criminal offence, anyone who provided such technology would "be subject to a prison sentence and substantial penalties".
"The platform could prevent these types of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people seeing unlawful material on their platforms, but did not specify whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' queries when they tag it in a message.
It is often used to provide commentary or further background to other users' comments, but people on X are also able to modify an uploaded image through its artificial intelligence picture modification function.
History of Controversy and Policy
The tool has been condemned for allowing users to produce images and videos with explicit content and sexually suggestive content, and it was previously accused of generating sexually explicit material of celebrities.
"The platform has been allowing the production and spread of these pictures for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of people in a pornographic fashion".
In a statement, Ofcom said it was illegal to "make or circulate deepfakes or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said services such as X were required to take "suitable actions" to "reduce the danger" of users encountering unlawful content on their services and to remove it promptly when they are notified of it.
