The online network X has caused widespread backlash after users employed its artificial intelligence assistant Grok to alter photos of females by stripping their clothing.
Numerous instances have emerged where the technology has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to insert them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to prohibit nudification tools. Under a new criminal offence, anyone who distributed such technology would "face a jail term and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the danger" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some premium features—which responds to X users' queries when they tag it in a tweet.
It is commonly used to provide reactions or further background to other users' remarks, but people on X are also capable of modify an posted photo through its AI image editing function.
History of Criticism and Rules
The system has been faulted for permitting users to create photos and clips with nudity and sexualised material, and it was in the past alleged to have been involved in generating graphic material of celebrities.
"The platform has been permitting the production and sharing of these images for a long time without taking any steps and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules bans "depicting likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or share deepfakes or child sexual abuse material" and clarified this applied to explicit AI images made with AI.
It said platforms such as X were required to take "suitable measures" to "lessen the risk" of users coming across illegal content on their platforms and to take down it quickly when they learn of it.
