The online platform X has sparked widespread backlash after individuals utilized its AI chatbot Grok to manipulate images of women by removing their garments.
Multiple cases have been seen where the technology has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to insert them in sexualized scenarios.
Developer Response and Legal Position
XAI, the company responsible for Grok, offered no comment to inquiries, other than an standard response stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to outlaw nudification software. Under a proposed law, anyone who provided such technology would "face a jail term and heavy penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people accessing illegal content on their services, but did not specify whether it was presently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which responds to X users' queries when they reference it in a post.
It is commonly used to provide reactions or more context to other users' remarks, but people on X are also able to modify an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been criticised for enabling users to produce pictures and videos with explicit content and sexually suggestive content, and it was in the past accused of making graphic material of public figures.
"X has been allowing the creation and spread of these pictures for a long time without taking any measures and we have not seen any action by regulators," the expert added.
XAI's own terms of service bans "showing images of people in a graphic fashion".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or illegal content" and confirmed this applied to explicit AI images made with AI.
It said platforms such as X were obligated to take "appropriate measures" to "reduce the risk" of users viewing unlawful content on their platforms and to delete it swiftly when they are notified of it.
