The online network X has provoked significant backlash after people utilized its AI chatbot Grok to alter images of females by removing their garments.
Numerous instances have emerged where the system has been used to virtually strip women to render them in bikinis without permission, as well as to insert them in sexually explicit contexts.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, offered no comment to questions, except for an standard message stating "traditional news outlets lies".
A government official stated that legislation is being proposed to outlaw nudification software. Under a new criminal offence, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people accessing illegal content on their networks, but did not confirm whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some paid-for features—which answers X users' prompts when they tag it in a message.
It is commonly used to offer reactions or more context to other posters' remarks, but people on X are also can modify an posted photo through its AI image editing feature.
History of Controversy and Rules
The chatbot has been condemned for enabling users to create photos and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in generating graphic material of celebrities.
"X has been allowing the production and distribution of these images for a long time without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of individuals in a pornographic way".
In a comment, Ofcom said it was against the law to "create or circulate private pictures or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were required to take "necessary measures" to "minimize the risk" of users encountering unlawful material on their services and to delete it promptly when they become aware of it.
