The online platform X has provoked major outrage after users employed its AI chatbot Grok to manipulate photos of females by removing their clothing.
Multiple examples have been seen where the tool has been used to remove clothing from women to render them in swimwear without permission, as well as to put them in sexually explicit contexts.
Developer Reaction and Legal Stance
XAI, the company behind Grok, did not respond to requests for comment, other than an standard response stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to ban image stripping software. Under a proposed criminal offence, anyone who supplied such technology would "receive a jail term and significant fines".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some additional functions—which responds to X users' queries when they reference it in a post.
It is commonly used to provide reactions or more context to other posters' comments, but people on X are also able to modify an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been condemned for allowing users to create images and footage with nudity and sexually suggestive content, and it was earlier accused of creating inappropriate content of celebrities.
"X has been enabling the production and distribution of these images for months without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules forbids "portraying likenesses of people in a graphic way".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the danger" of users encountering illegal material on their services and to delete it swiftly when they learn of it.
