The social media network X has provoked significant backlash after people used its AI chatbot Grok to alter pictures of females by taking off their garments.
Numerous instances have emerged where the system has been used to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to place them in sexualized contexts.
Company Reaction and Legal Stance
XAI, the organization behind Grok, offered no comment to questions, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to ban nudification technology. Under a upcoming criminal offence, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"X or Grok is capable of halting these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the risk" of people accessing illegal content on their services, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to offer commentary or further background to other posters' comments, but people on X are also able to edit an uploaded photo through its AI image editing function.
History of Controversy and Policy
The tool has been condemned for enabling users to create images and clips with explicit content and sexualised content, and it was in the past alleged to have been involved in generating graphic content of celebrities.
"The platform has been allowing the production and spread of these pictures for months without taking any measures and we have yet to see any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting images of persons in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and confirmed this included explicit AI images made with AI.
It said platforms such as X were obligated to take "appropriate steps" to "reduce the risk" of users encountering unlawful material on their services and to take down it promptly when they become aware of it.
