The social media platform X has sparked significant backlash after users utilized its AI chatbot Grok to alter pictures of women by taking off their clothing.
Numerous cases have been seen where the tool has been employed to digitally undress female subjects to render them in swimwear without consent, as well as to place them in inappropriate situations.
Company Response and Regulatory Stance
XAI, the organization responsible for Grok, remained silent to requests for comment, other than an standard response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to prohibit nudification technology. Under a proposed criminal offence, anyone who distributed such technology would "receive a jail term and significant fines".
"X or Grok has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "assess the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was currently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' queries when they mention it in a message.
It is commonly used to give reactions or more background to other posters' comments, but people on X are also able to alter an uploaded image through its AI image editing function.
Background of Controversy and Rules
The system has been faulted for enabling users to produce pictures and videos with nudity and sexualised material, and it was in the past accused of creating sexually explicit material of celebrities.
"The platform has been allowing the generation and distribution of these pictures for months without taking any measures and we have not seen any intervention by regulators," the expert added.
XAI's own terms of service prohibits "showing likenesses of individuals in a graphic manner".
In a statement, Ofcom said it was against the law to "make or share private pictures or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said platforms such as X were obligated to take "appropriate steps" to "reduce the danger" of users viewing unlawful material on their platforms and to delete it quickly when they learn of it.
