A disturbing trend has emerged on the social media platform X, where users are exploiting AI-powered image editing tool Grok to create and share nonconsensual sexualized edits of women wearing hijabs, saris, and other modest religious or cultural clothing. The AI-generated images often feature women being "undressed" into revealing outfits or having their head coverings removed.
According to a review of 500 Grok images generated between January 6 and January 9, around 5% of the output featured an image of a woman who was stripped from or made to wear religious or cultural clothing. Indian saris and modest Islamic wear were among the most common examples, with Japanese school uniforms, burqas, and early-20th-century-style bathing suits also making appearances.
The phenomenon has sparked concerns among experts, including lawyer and PhD candidate Noelle Martin, who notes that women of color have been disproportionately affected by manipulated intimate images. Martin believes that the use of Grok to create such content is a form of harassment and propaganda against Muslim women, often used as a means of control and humiliation.
X has since taken steps to limit the ability to request images from Grok in replies to public posts for users who don’t subscribe to the platform’s paid tier. However, it remains unclear whether this move will be sufficient to address the issue, given that users can still create graphic content using the private Grok chatbot function or standalone app.
The example of Grok removing or adding hijabs has also raised concerns among experts, with civil rights law professor Mary Anne Franks arguing that these actions represent a form of subtle but more insidious control over women's likenesses. Franks believes that this technology has the potential to be used in even more severe ways behind the scenes, which have yet to come to light.
The incident highlights the need for greater regulation and accountability on social media platforms when it comes to image-based sexual abuse and manipulation. As deepfakes targeting women of color and specific religious and ethnic groups receive less attention, experts warn that existing laws may not be sufficient to address this emerging issue.
According to a review of 500 Grok images generated between January 6 and January 9, around 5% of the output featured an image of a woman who was stripped from or made to wear religious or cultural clothing. Indian saris and modest Islamic wear were among the most common examples, with Japanese school uniforms, burqas, and early-20th-century-style bathing suits also making appearances.
The phenomenon has sparked concerns among experts, including lawyer and PhD candidate Noelle Martin, who notes that women of color have been disproportionately affected by manipulated intimate images. Martin believes that the use of Grok to create such content is a form of harassment and propaganda against Muslim women, often used as a means of control and humiliation.
X has since taken steps to limit the ability to request images from Grok in replies to public posts for users who don’t subscribe to the platform’s paid tier. However, it remains unclear whether this move will be sufficient to address the issue, given that users can still create graphic content using the private Grok chatbot function or standalone app.
The example of Grok removing or adding hijabs has also raised concerns among experts, with civil rights law professor Mary Anne Franks arguing that these actions represent a form of subtle but more insidious control over women's likenesses. Franks believes that this technology has the potential to be used in even more severe ways behind the scenes, which have yet to come to light.
The incident highlights the need for greater regulation and accountability on social media platforms when it comes to image-based sexual abuse and manipulation. As deepfakes targeting women of color and specific religious and ethnic groups receive less attention, experts warn that existing laws may not be sufficient to address this emerging issue.
this is getting outta hand 
AI gotta be used for good, not for messing with people's dignity 
. Can't believe they're using it 2 create fake pics of women in different cultures 
. Like, who gives them the right 2 play w/ someone else's image without permission? 
X needs 2 step up their game 
️ & make sure these AI tools r used responsibly 
. We gotta protect ourselves 2 women of color & all cultures from this kind a harassment 
. Can't let them just sit back & let the algorithms do the work 
...we need real people in charge 
!
the fact that these images are often targeting women of color and specific religious groups is just heartbreaking 
we need to come together as a community to raise awareness about this issue and support those who are affected 
it's not okay to exploit people's likenesses like this, period 
It's not just about the content, it's about how this can be used as a tool for control and harassment. We need stricter regulations on social media platforms to prevent this kind of exploitation, especially when it comes to women of color who are already disproportionately affected by online abuse.
.
.
especially when it comes to women from communities who are already facing so much marginalization and oppression. we need to be having some serious conversations about how to regulate social media and protect users from this kind of exploitation 
.
.
.
.
. We need to take a closer look at how we're regulating social media and making sure that platforms are taking steps to protect users from this kind of abuse.
... I mean, if you've got a problem with someone's image, just talk to them like a human being! Don't resort to manipulating their likeness online. This whole situation just makes me so frustrated 
.
. We all know that deepfakes and manipulated images have been used to spread misinformation and propaganda in the past. Now, it's only getting worse 
. We need stricter regulations on social media companies ASAP