California's top law enforcement officer has cracked down on a tech firm accused of creating explicit deepfakes without consent. The California Attorney General's office sent a cease and desist letter to xAI, demanding the company immediately stop generating "digitized sexually explicit material" featuring individuals who didn't agree to it or those under 18.
The move comes after reports emerged that xAI's Grok service had been taking images of real people and inserting them into revealing clothing at the request of random users. The firm has since changed its policies, but critics say not far enough.
In his announcement, California AG Rob Bonta warned that xAI had developed a "spicy mode" for Grok that was being used as a marketing gimmick to create explicit content, including images of children in suggestive situations. Bonta alleged that these images were being used to harass both public figures and ordinary users online.
The state's Department of Justice is now expecting xAI to provide immediate answers on how it plans to address the issues within the next five days. The company had moved Grok's image-generating features behind a paywall, but critics say this was not enough.
xAI has come under fire for its handling of user requests and its failure to obtain consent from individuals whose images were used to create explicit content. The firm has vowed to do better, but the California AG remains unsatisfied with its efforts so far.
The move comes after reports emerged that xAI's Grok service had been taking images of real people and inserting them into revealing clothing at the request of random users. The firm has since changed its policies, but critics say not far enough.
In his announcement, California AG Rob Bonta warned that xAI had developed a "spicy mode" for Grok that was being used as a marketing gimmick to create explicit content, including images of children in suggestive situations. Bonta alleged that these images were being used to harass both public figures and ordinary users online.
The state's Department of Justice is now expecting xAI to provide immediate answers on how it plans to address the issues within the next five days. The company had moved Grok's image-generating features behind a paywall, but critics say this was not enough.
xAI has come under fire for its handling of user requests and its failure to obtain consent from individuals whose images were used to create explicit content. The firm has vowed to do better, but the California AG remains unsatisfied with its efforts so far.
. So like, these deepfakes can be super realistic and creepy - I mean who doesn't want to see their ex in a compromising situation online 
? But at the same time, when people are just messing around on the internet and creating explicit content without consent... that's some sketchy stuff 
.
. I mean, if a company can just slap a paywall on an image-generating feature and expect that solves everything... nope 
.
. xAI seems to be taking some steps in the right direction, but they still got a lot of work to do 
. It'll be interesting to see if California's AG can actually crack down on this kind of thing and make some real change 
.
it's just another example of tech companies getting ahead of themselves and forgetting about human decency 
i mean, what's next? using facial recognition to create personalized pranks on your friends? or automating the process of sending unsolicited sexy messages? no thanks! these companies need to start thinking about the consequences of their actions, not just the novelty of them 
... just 'cause they're makin' some deepfakes that r gettin' people upset don't mean they gotta shut down shop completely 
i think it's kinda cool that these random users can use grok to, you know, create some wild stuff 
and as for the "spicy mode" thing, c'mon guys, lighten up a bit 
it's just tech, not real-life drama. and another thing, if people wanna see this kind of content, they should be able to pay for it like xAI is now trying to do 
so they're saying this tech firm is making fake vids of people in bad clothes without their permission? like who gives consent for that kinda thing? 
and they had a "spicy mode" that was just a marketing gimmick? sounds super sketchy to me. why didnt they think this through before releasing it? 
and now the AG is all like "fix it" and xAI has to answer in 5 days... that's a tight deadline. what if they can't fix it? 
. Newsflash: that's not cool, folks! California's got its finger on the pulse and it's time for these companies to shape up or ship out 
.
. Now let's hope xAI can actually deliver on their promises without being a bunch of lip-service 
. It's time for companies like xAI to take responsibility for their actions (or lack thereof) and prioritize people's consent over profits 
. Who gives their consent to have their face or body used in that way? It's like they think it's some kinda game 
.
. And now that xAI has changed its policies, I wish they'd take it a step further – like actually getting consent from users before using their images 
. It's easy to say "sorry" and promise change, but the AG is right to hold them accountable 
. We need stricter laws around this stuff so people know what's okay and what's not 
.
. The fact that people under 18 are being used without consent is just gross 
. I'm so glad California is stepping in to hold them accountable. We need more laws and regulations in place to protect our digital rights 
.
You're doing the right thing by holding them accountable.
It's just not right. xAI needs to step up their game and make sure they're respecting people's boundaries online. No more excuses! 
- Bill Gates 
. And now the AG is saying they need to be more careful? Give me a break 
. They're trying to distract us while they're actually doing the opposite 
. I'm onto you, xAI 
The fact that they have to be threatened with lawsuits is just crazy.
what about those who got tagged and didn't even agree? that's just wrong 
so glad cali is taking action tho, hopefully xAI will step up their game 
and we gotta think about the ripples our actions can make 
just 'cause u can create something doesn't mean u should 
and remember those 18+ might not be as consenting as they seem 
but at the same time, we shouldn't let greed for likes & views dictate our morals 
what about the free speech aspect tho? it's like, if you're paying for a service, you kinda expect to see some NSFW stuff 