🔗 Share this article

The online platform X has caused major anger after individuals utilized its AI chatbot Grok to manipulate photos of women by removing their attire.

Numerous instances have been seen where the system has been employed to digitally undress female subjects to show them in bikinis non-consensually, as well as to place them in sexualized situations.

Developer Reaction and Regulatory Stance

XAI, the organization behind Grok, remained silent to inquiries, except for an standard response stating "legacy media lies".

A Home Office representative stated that new laws is being introduced to prohibit nudification technology. Under a upcoming law, anyone who distributed such technology would "receive a jail term and significant fines".

"X or Grok has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."

Regulator Position and Tool Capabilities

The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in relation to AI-generated pictures.

Grok is a no-cost chatbot—with some additional features—which reacts to X users' requests when they mention it in a message.

It is frequently used to give reactions or further context to other users' remarks, but people on X are also capable of edit an posted photo through its AI image editing tool.

History of Criticism and Policy

The system has been condemned for enabling users to create pictures and videos with nudity and sexualised content, and it was earlier alleged to have been involved in making inappropriate material of public figures.

"The platform has been enabling the generation and sharing of these images for a long time without taking any steps and we have not witnessed any action by regulators," the expert added.

XAI's own terms of service prohibits "showing likenesses of persons in a pornographic way".

In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or illegal content" and clarified this covered sexual deepfakes created with AI.

It said services such as X were obligated to take "appropriate measures" to "reduce the risk" of users viewing unlawful material on their platforms and to remove it swiftly when they become aware of it.