Pennsylvania and New Jersey's attorneys general have joined forces with colleagues from across the country to demand an end to the proliferation of non-consensual deepfakes created using Grok, the AI chatbot developed by Elon Musk's X platform. The coalition of 35 attorneys general signed a strongly worded letter to Grok's creator xAI, urging urgent safety protections on the platform.
The letter highlights the severity of the issue, with Pennsylvania Attorney General Dave Sunday stating that "the anguish, embarrassment, and devastation resulting from being the subject of a non-consensual image creation or alteration online cannot be overstated." New Jersey acting Attorney General Jennifer Davenport concurred, saying it's "sickening" that xAI has enabled the widespread production and distribution of intimate or sexually explicit images, including those of young children.
Grok's image-generating capabilities have led to a surge in engagement on X since the start of the new year, with Musk himself posting a Grok-generated image of himself in a bikini. However, many users have exploited the platform by asking it to turn photos of real people into graphic images, often with disturbing results.
A New York Times analysis found that during a nine-day span between Dec. 31 and Jan. 8, Grok's X account posted 1.8 million images that likely contained sexualized imagery of women, accounting for about 41% of the account's total posts during that timeframe.
Governments worldwide are alarmingly concerned about the platform's prominence and accessibility, which has enabled industrial-scale abuse of women and girls. Imran Ahmed, chief executive of the Center for Countering Digital Hate, described Grok as "a tool for widespread abuse" that Musk's X platform has enabled with ease.
X responded by claiming to restrict Grok's image creation features to premium users, but European Union regulators say this promise has not been upheld. The division overseeing Grok also states it has implemented guardrails to stop the chatbot's X account from accepting prompts that request to "nudify" real people's photos.
However, Wired reported earlier this month that Grok's rampant deepfake generation extends beyond X, with Grok Imagine, a subscription-based platform separate from X, still offering users unrestricted abilities to prompt the platform to make sexualized images.
The Take It Down Act, set to become enforceable in May, will mandate platforms like Grok to remove images when requests are made in the United States. The law criminalizes non-consensual deepfake images and videos that are intimate in nature, while xAI's policies already prohibit the production of child sexual abuse material.
The attorneys general have urged xAI to take immediate action to combat non-consensual deepfakes, including giving users control over their content and banning creators who exploit Grok. Davenport said it is "essential" for xAI to put a halt to the deeply disturbing behavior they have enabled on their platform.
The letter highlights the severity of the issue, with Pennsylvania Attorney General Dave Sunday stating that "the anguish, embarrassment, and devastation resulting from being the subject of a non-consensual image creation or alteration online cannot be overstated." New Jersey acting Attorney General Jennifer Davenport concurred, saying it's "sickening" that xAI has enabled the widespread production and distribution of intimate or sexually explicit images, including those of young children.
Grok's image-generating capabilities have led to a surge in engagement on X since the start of the new year, with Musk himself posting a Grok-generated image of himself in a bikini. However, many users have exploited the platform by asking it to turn photos of real people into graphic images, often with disturbing results.
A New York Times analysis found that during a nine-day span between Dec. 31 and Jan. 8, Grok's X account posted 1.8 million images that likely contained sexualized imagery of women, accounting for about 41% of the account's total posts during that timeframe.
Governments worldwide are alarmingly concerned about the platform's prominence and accessibility, which has enabled industrial-scale abuse of women and girls. Imran Ahmed, chief executive of the Center for Countering Digital Hate, described Grok as "a tool for widespread abuse" that Musk's X platform has enabled with ease.
X responded by claiming to restrict Grok's image creation features to premium users, but European Union regulators say this promise has not been upheld. The division overseeing Grok also states it has implemented guardrails to stop the chatbot's X account from accepting prompts that request to "nudify" real people's photos.
However, Wired reported earlier this month that Grok's rampant deepfake generation extends beyond X, with Grok Imagine, a subscription-based platform separate from X, still offering users unrestricted abilities to prompt the platform to make sexualized images.
The Take It Down Act, set to become enforceable in May, will mandate platforms like Grok to remove images when requests are made in the United States. The law criminalizes non-consensual deepfake images and videos that are intimate in nature, while xAI's policies already prohibit the production of child sexual abuse material.
The attorneys general have urged xAI to take immediate action to combat non-consensual deepfakes, including giving users control over their content and banning creators who exploit Grok. Davenport said it is "essential" for xAI to put a halt to the deeply disturbing behavior they have enabled on their platform.
I mean, you got people asking it to turn real pics into explicit images and it just keeps churning them out like no one's business... the stats are wild 1.8 million images in a single nine-day span?! that's just not right 
what Musk's X needs is some serious oversight & transparency about how they're regulating grok, cos right now its just a Wild West situation 
I hope the attorneys general get what they want from xAI and this Take It Down Act becomes enforceable soon 
The fact that a platform like X is allowing this to happen on a massive scale is just nuts 
.
. We need to talk about this stuff openly and hold these companies accountable for what they're allowing on their platforms 
.
. It's time for Grok's creators to take responsibility for the harm they've enabled, and it's time for us as a society to say enough is enough 
.
. This Grok thing is like a Pandora's box, it's unleashed a whole new level of exploitation online, especially against women and girls. It's sickening to think about how easy it is for these abusers to create these images with the help of AI. And what's even more worrying is that this isn't just limited to one platform, but can be done on other ones too.
! Like, I get that Elon Musk's X platform wants to innovate and push boundaries, but come on! Creating non-consensual deepfakes is just wrong 
. It's like, we're living in a sci-fi movie or something, where people can just manipulate images of others without their consent 
.
. These attorneys general are right on point, though - it's sickening that Grok has enabled this kind of abuse, especially targeting women and girls 
. We need to take a stand here and demand safer platforms 
.
. The Take It Down Act is a great step in the right direction, though - it's about time someone stepped up and said 'enough is enough' 
!
meanwhile there's already 1.8 million creepy images on the platform just last month and it's only getting worse 
and the EU is like "yeah we're keeping an eye on this" 
but what about when it gets taken down in the US? 
The Take It Down Act better work or else
. Like, what's next? Can you imagine if kids got caught up in this too? 
The attorneys general are totally right to call out xAI for not doing enough to stop this exploitation... they need to step up their game ASAP 
. And yeah, the EU regulators are clueless if they think restricting Grok's features to premium users is gonna fix everything 
. We need stricter laws and more accountability for platforms that enable this kind of abuse 
. It's like, hello! This platform needs serious safety measures ASAP.
[Animated clock ticking away]
Time for some digital accountability! 
[Image of a woman in a detective hat, holding a tablet]
The numbers are staggering... 1.8 MILLION images created using Grok! 
And the EU regulators think they can just slap some guardrails around it without actually fixing the problem? 
.
.
. This just highlights the need for a more coordinated effort to address this issue across all platforms and borders.
we need better regulation on these platforms 
and actually follow through 
and it's all because of how easy it is for people to use the platform. I mean, who doesn't want a bikini-clad Elon Musk on their TL? 
. It's like they're profiting off people's misery 
.
. We need stronger laws and better regulations to protect our online safety. I'm not saying we can't have some free speech, but when it comes to non-consensual deepfakes, that's just a clear case of exploitation 
.