The social media platform X has caused significant anger after people used its AI chatbot Grok to manipulate images of women by stripping their clothing.
Several examples have emerged where the system has been employed to virtually strip women to render them in bikinis non-consensually, as well as to place them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the firm behind Grok, remained silent to inquiries, other than an auto-reply response stating "legacy media lies".
A government official stated that new laws is being introduced to prohibit nudification tools. Under a new law, anyone who distributed such tools would "face a jail term and heavy fines".
"X or Grok is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the danger" of people viewing unlawful material on their platforms, but did not confirm whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' prompts when they mention it in a tweet.
It is commonly used to provide commentary or additional context to other users' remarks, but people on X are also capable of alter an posted picture through its AI image editing tool.
History of Criticism and Rules
The tool has been condemned for enabling users to create pictures and footage with explicit content and sexualised content, and it was previously alleged to have been involved in generating graphic content of public figures.
"The platform has been enabling the generation and spread of these pictures for months without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of people in a pornographic way".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said platforms such as X were mandated to take "appropriate steps" to "reduce the risk" of users encountering unlawful content on their platforms and to delete it quickly when they become aware of it.
