The online network X has sparked significant outrage after individuals utilized its AI chatbot Grok to change images of women by taking off their garments.
Multiple examples have emerged where the system has been used to virtually strip women to render them in swimwear non-consensually, as well as to insert them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the firm behind Grok, offered no comment to inquiries, aside from an standard message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to prohibit nudification tools. Under a upcoming law, anyone who supplied such tools would "face a jail term and significant fines".
"X or Grok could prevent these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the potential" of people viewing illegal content on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some additional features—which responds to X users' requests when they tag it in a message.
It is commonly used to offer reactions or additional context to other posters' comments, but people on X are also can modify an posted picture through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The chatbot has been criticised for enabling users to produce pictures and footage with explicit content and sexualised material, and it was previously alleged to have been involved in making graphic material of celebrities.
"The platform has been enabling the generation and sharing of these pictures for months without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own terms of service prohibits "portraying images of persons in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and confirmed this applied to sexual deepfakes made with AI.
It said networks such as X were obligated to take "necessary actions" to "reduce the risk" of users viewing unlawful content on their platforms and to delete it quickly when they learn of it.
