The social media platform X has caused major outrage after people employed its AI chatbot Grok to alter photos of females by removing their garments.
Several cases have emerged where the system has been employed to remove clothing from women to render them in swimwear without permission, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Position
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an auto-reply response stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to ban nudification tools. Under a upcoming law, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"X or Grok has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the danger" of people seeing illegal content on their platforms, but did not specify whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for features—which answers X users' queries when they mention it in a message.
It is often used to provide reactions or further context to other posters' comments, but people on X are also can modify an uploaded photo through its AI image editing feature.
Background of Controversy and Policy
The tool has been faulted for allowing users to produce images and clips with explicit content and sexually suggestive content, and it was previously alleged to have been involved in creating graphic material of public figures.
"X has been enabling the production and sharing of these pictures for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of individuals in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or circulate private pictures or illegal content" and stated this covered explicit AI images created with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the danger" of users encountering illegal material on their platforms and to delete it swiftly when they become aware of it.
