\n<\/p>\n
For more than two years, an app called ClothOff has been terrorizing young women online \u2014 and it\u2019s been maddeningly difficult to stop. The app has been taken down from the two major app stores and it\u2019s banned from most social platforms, but it\u2019s still available on the web and through a Telegram bot. In October, a clinic at Yale Law School filed a lawsuit that would take down the app entirely, forcing the owners to delete all images and cease operation entirely. But simply finding the defendants has been a challenge.\u00a0<\/p>\n
\u201cIt\u2019s incorporated in the British Virgin Islands,\u201d explains Professor John Langford, a co-lead counsel in the lawsuit, \u201cbut we believe it\u2019s run by a brother and sister and Belarus. It may even be part of a larger network around the world.\u201d<\/p>\n
It\u2019s a bitter lesson in the wake of the recent flood of non-consensual pornography<\/a> generated by Elon Musk\u2019s xAI, which included many underage victims<\/a>. Child sexual abuse material is the most legally toxic content on the internet \u2014 illegal to produce, transmit or store, and regularly scanned for on every major cloud service. But despite the intense legal prohibitions, there are still few ways to deal with image generators like ClothOff, as Langford\u2019s case demonstrates. Individual users can be prosecuted, but platforms like ClothOff and Grok are far more difficult to police, leaving few options for victims hoping to find justice in court.<\/p>\n The clinic\u2019s complaint, which is available online<\/a>, paints an alarming picture. The plaintiff is an anonymous high school student in New Jersey, whose classmates used ClothOff to alter her Instagram photos. She was 14 years old when the original Instagram photos were taken, which means the AI-modified versions are legally classified as child abuse imagery. But even though the modified images are straightforwardly illegal, local authorities declined to prosecute the case, citing the difficulty of obtaining evidence from suspects\u2019 devices.<\/p>\n \u201cNeither the school nor law enforcement ever established how broadly the CSAM of Jane Doe and other girls was distributed,\u201d the complaint reads.<\/p>\n Still, the court case has moved slowly. The complaint was filed in October, and in the months since, Langford and his colleagues have been in the process of serving notice to the defendants \u2014 a difficult task given the global nature of the enterprise. Once they\u2019ve been served, the clinic can push for a court appearance and, eventually, a judgment, but in the meantime the legal system has given little comfort to ClothOff\u2019s victims.<\/p>\n The Grok case might seem like a simpler problem to fix. Elon Musk\u2019s xAI isn\u2019t hiding, and there\u2019s plenty of money at the end for lawyers who can win a claim. But Grok is a general purpose tool, which makes it much harder to hold it accountable in court.<\/p>\n Techcrunch event<\/p>\n \n\t\t\t\t\t\t\t\t\tSan Francisco<\/span> \u201cClothOff is designed and marketed specifically as a deepfake pornography image and video generator,\u201d Langford told me. \u201cWhen you\u2019re suing a general system that users can query for all sorts of things, it gets a lot more complicated.\u201d<\/p>\n A number of US laws have already banned deepfake pornography \u2014 most notably the Take It Down Act<\/a>. But while specific users are clearly breaking those laws, it\u2019s much harder to hold the entire platform accountable. Existing laws require clear evidence of an intent to harm, which would mean providing evidence xAI knew their tool would be used to produce non-consensual pornography. Without that evidence, xAI\u2019s basic first amendment rights would provide significant legal protection..<\/p>\n \u201cIn terms of the First Amendment, it\u2019s quite clear Child Sexual Abuse material is not protected expression,\u201d Langford says. \u201cSo when you\u2019re designing a system to create that kind of content, you\u2019re clearly operating outside of what\u2019s protected by the First Amendment. But when you\u2019re a general system that users can query for all sorts of things, it\u2019s not so clear.\u201d<\/p>\n The easiest way to surmount those problems would be to show that xAI had willfully ignored the problem. It\u2019s a real possibility, given recent reporting<\/a> that Musk directed employees to loosen Grok\u2019s safeguards. But even then, it would be a far riskier case to take on. \u00a0<\/p>\n \u201cReasonable people can say, we knew this was a problem years ago,\u201d Langford says. \u201cHow can you not have had more stringent controls in place to make sure this doesn\u2019t happen? That is a kind of recklessness or knowledge but it\u2019s just a more complicated case.\u201d<\/p>\n Those First Amendment issues are why xAI\u2019s biggest pushback has come from court systems without robust legal protections for free speech. Both Indonesia and Malaysia<\/a> have taken steps to block access to the Grok chatbot, while regulators in the United Kingdom have opened an investigation<\/a> that could lead to a similar ban. Other preliminary steps<\/a> have been taken by the European Commission, France, Ireland, India and Brazil. In contrast, no US regulatory agency has issued an official response.<\/p>\n It\u2019s impossible to say how the investigations will resolve, but at the very least, the flood of imagery raises lots of questions for regulators to investigate \u2014 and the answers could be damning.<\/p>\n \u201cIf you are posting, distributing, disseminating Child Sexual Abuse material, you are violating criminal prohibitions and can be held accountable,\u201d Langford says. \u201cThe hard question is, what did X know? What did X do or not do? What are they doing now in response to it?\u201c<\/p>\n<\/div>\n
\n\t\t\t\t\t\t\t\t\t\t\t\t\t|<\/span>
\n\t\t\t\t\t\t\t\t\t\t\t\t\tOctober 13-15, 2026<\/span>\n\t\t\t\t\t\t\t<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n