Elon musk and the xAI logo.
Vincent Feuray | Afp | Getty Images
Elon Musk‘s X is being investigated by authorities in Europe, India and Malaysia after its Grok chatbot let users create and share AI-generated sexualized images of children and women.
British media watchdog Ofcom also said it’s requested information from X, which is owned by xAI, concerning the Grok issues. And on Sunday, a member of Brazil’s parliament said on social media that she’s asked the country’s federal public prosecutor and data protection authority to suspend use of Grok until an investigation is completed.
The probes follow a global surge over the past few weeks in the use of Grok to create and share nonconsensual, intimate images, or NCII, derived from photos or videos of real people in response to user’s prompts. The concerning images have been widely shared on X.
Musk’s company recently updated its Grok Imagine features, enabling easier image generation from text-based prompts on the platform.
While safety experts and tech critics decried the proliferation of exploitative images and clips on X, Musk appeared to mock the situation by sharing an array of Grok-generated images, including one depicting himself in a bikini, punctuated by laughing-crying emojis.
At a press conference on Monday, European Commission spokesperson Thomas Regnier said the authority was “very seriously looking into this matter” and was “well aware” that X and Grok were “now offering a spicy mode showing explicit sexual content with some output generated with childlike images.”
“This is not ‘spicy,'” Regnier said. “This is illegal. This is appalling. This is disgusting. This is how we see it, and this has no place in Europe.”
Late last week, India’s Ministry of Electronics and Information Technology ordered X to conduct a “comprehensive technical, procedural and governance-level review” of Grok. The company was given until Jan. 5 to comply.
The Malaysian Communications and Multimedia Commission said over the weekend that it’s investigating X and will call in company representatives.
“MCMC urges all platforms accessible in Malaysia to implement safeguards aligned with Malaysian laws and online safety standards, especially in relation to their AI-powered features, chatbots and image manipulation tools,” the group said in a statement.
Meanwhile, in the U.S., the National Center on Sexual Exploitation (NCOSE) called on the Department of Justice and Federal Trade Commission to investigate the matter in an interview with CNBC. Dani Pinter, chief legal officer and director of the Law Center for NCOSE, said there’s “not a lot of legal precedence on point for these specific issues.”
However, she said federal laws prohibit the creation and distribution of child sexual abuse materials, or CSAM, and that can be applied to virtually created content “when it depicts an identifiable child, or depicts a child engaged in sexually explicit conduct.”
Those laws include the Take It Down Act, which was endorsed by First Lady Melania Trump before it was enacted last year.
A DOJ spokesperson said in an emailed statement to CNBC on Monday evening, “The Department of Justice takes AI-generated child sex abuse material extremely seriously and will aggressively prosecute any producer or possessor of CSAM. We continue to explore ways to optimize enforcement in this space to protect children and hold accountable individuals who exploit technology to harm our most vulnerable.”
The FTC declined to comment. XAI didn’t provide a comment beyond an automated reply.
Musk’s social media company made its first public statement on the matter on Saturday in a post on the official X Safety account.
“We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary,” the company wrote.
Musk wrote, in a separate post on X, “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”
The following day, an xAI employee named Ethan He wrote in a post on X that Grok Imagine was updated, but he didn’t specify any changes regarding the ability to create harmful explicit images.
Musk and X have a history of allowing users who created posts showing child sexual exploitation to stay on the platform.
In 2023, X briefly suspended then reinstated the account of a user named Dom Lucre after he posted “child exploitation pictures associated with the criminal conviction of an Australian man in the Philippines,” Mashable reported. Musk said at that time that X decided to delete Lucre’s offending posts but to reinstate the right-wing influencer to the platform. Lucre currently has a monetized account on X with 1.6 million listed followers.
Tom Quisel, CEO of Musubi AI, which helps social networks and AI companies automate content moderation using AI, said it appeared that xAI had failed to build even “entry level trust and safety layers” into the rollout of Grok Imagine to X.
Quisel said it would be easy for a company like xAI to have its model detect and block “an image involving children or partial nudity,” or to reject users’ prompts to put the subject of a photo in sexually suggestive outfits.
The controversy has done nothing to harm X’s traffic.
According to Apptopia, which tracks mobile app trends, daily downloads of Grok have increased 54% since Jan. 2, while daily downloads of X have jumped 25% in the past three days.
