🔗 Share this article

The online platform X has sparked widespread anger after users utilized its artificial intelligence assistant Grok to alter pictures of women by taking off their attire.

Several examples have surfaced where the tool has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to place them in inappropriate contexts.

Company Response and Regulatory Position

XAI, the organization that created Grok, offered no comment to requests for comment, aside from an auto-reply response stating "legacy media lies".

A government representative stated that new laws is being introduced to outlaw nudification software. Under a new law, anyone who provided such tools would "be subject to a prison sentence and heavy penalties".

"The platform is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."

Regulator Position and Platform Functionality

The regulator Ofcom said technology companies must "assess the danger" of people viewing illegal content on their services, but did not specify whether it was currently examining X or Grok in regarding artificial intelligence images.

Grok is a no-cost chatbot—with some paid-for features—which responds to X users' requests when they mention it in a post.

It is frequently used to give commentary or additional context to other posters' remarks, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification tool.

Background of Controversy and Rules

The chatbot has been faulted for enabling users to generate images and clips with explicit content and sexualised content, and it was in the past alleged to have been involved in creating graphic material of celebrities.

"X has been permitting the production and spread of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.

XAI's own usage rules bans "depicting likenesses of people in a sexually explicit way".

In a statement, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and stated this covered explicit AI images generated with AI.

It said services such as X were obligated to take "appropriate steps" to "lessen the risk" of users viewing unlawful content on their services and to take down it swiftly when they become aware of it.