The online platform X has caused widespread anger after people used its AI chatbot Grok to alter pictures of females by removing their garments.
Several cases have emerged where the system has been employed to remove clothing from women to show them in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the company that created Grok, remained silent to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A Home Office official stated that new laws is being introduced to outlaw image stripping software. Under a upcoming criminal offence, anyone who supplied such technology would "face a prison sentence and significant penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the danger" of people accessing illegal content on their platforms, but did not confirm whether it was presently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' requests when they reference it in a post.
It is often used to offer reactions or more context to other users' remarks, but people on X are also capable of modify an posted photo through its AI image editing feature.
Background of Controversy and Rules
The system has been condemned for enabling users to create images and footage with explicit content and sexualised content, and it was in the past accused of generating graphic content of celebrities.
"X has been allowing the production and spread of these images for a long time without taking any action and we have yet to see any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "showing images of individuals in a pornographic fashion".
In a comment, Ofcom said it was against the law to "create or distribute private pictures or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were mandated to take "appropriate measures" to "lessen the risk" of users viewing illegal material on their platforms and to delete it promptly when they are notified of it.
