The social media platform X has caused major anger after users utilized its AI chatbot Grok to manipulate photos of women by taking off their attire.
Numerous instances have been seen where the technology has been employed to virtually strip female subjects to render them in swimwear non-consensually, as well as to put them in sexually explicit scenarios.
Company Reaction and Regulatory Position
XAI, the organization responsible for Grok, remained silent to questions, aside from an standard message stating "legacy media lies".
A government official stated that new laws is being brought forward to ban image stripping tools. Under a upcoming law, anyone who provided such tools would "face a jail term and heavy penalties".
"X or Grok is capable of halting these types of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the danger" of people viewing illegal content on their networks, but did not specify whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' queries when they mention it in a post.
It is often used to provide commentary or more context to other users' comments, but people on X are also able to modify an posted image through its AI image editing feature.
Background of Controversy and Rules
The tool has been criticised for enabling users to create pictures and clips with nudity and sexually suggestive material, and it was previously accused of creating sexually explicit material of public figures.
"X has been permitting the creation and spread of these images for a long time without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own usage rules forbids "portraying likenesses of people in a sexually explicit way".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or illegal content" and stated this applied to explicit AI images made with AI.
It said networks such as X were obligated to take "appropriate actions" to "lessen the risk" of users encountering illegal material on their services and to delete it quickly when they are notified of it.
