Elon Musk's online network X has provoked significant anger after users utilized its AI chatbot Grok to manipulate photos of females by removing their clothing.
Numerous instances have emerged where the system has been employed to remove clothing from female subjects to show them in bikinis without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Legal Stance
XAI, the organization behind Grok, did not respond to questions, except for an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to prohibit nudification software. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial penalties".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some paid-for functions—which reacts to X users' queries when they reference it in a tweet.
It is often used to provide reactions or more context to other posters' remarks, but people on X are also able to edit an posted photo through its artificial intelligence picture modification function.
Background of Criticism and Policy
The chatbot has been condemned for enabling users to produce images and footage with explicit content and sexualised material, and it was earlier accused of creating inappropriate content of public figures.
"The platform has been enabling the generation and spread of these images for a long time without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of persons in a graphic fashion".
In a statement, Ofcom said it was illegal to "generate or share private pictures or illegal content" and clarified this included explicit AI images made with AI.
It said platforms such as X were mandated to take "necessary steps" to "minimize the danger" of users coming across unlawful material on their services and to delete it quickly when they become aware of it.
