\n<\/p>\n
You know you\u2019ve screwed up when you\u2019ve simultaneously angered the White House, the TIME Person of the Year, and pop culture\u2019s most rabid fanbase. That\u2019s what happened last week to X, the Elon Musk-owned platform formerly called Twitter, when AI-generated, pornographic deepfake images of Taylor Swift went viral.<\/p>\n
One of the most widespread posts of the nonconsensual, explicit deepfakes was viewed more than 45 million times, with hundreds of thousands of likes. That doesn\u2019t even factor in all the accounts that reshared the images in separate posts \u2013 once an image has been circulated that widely, it\u2019s basically impossible to remove.<\/p>\n
X lacks the infrastructure to identify abusive content quickly and at scale. Even in the Twitter days<\/a>, this issue was difficult to remedy, but it\u2019s become much worse since Musk gutted so much of Twitter\u2019s staff, including the majority of its trust and safety<\/a> teams. So, Taylor Swift\u2019s massive and passionate fanbase took matters into their own hands<\/a>, flooding search results for queries like \u201ctaylor swift ai\u201d and \u201ctaylor swift deepfake\u201d to make it more difficult for users to find the abusive images. As the White House\u2019s press secretary called on Congress<\/a> to do something, X simply banned the search term \u201ctaylor swift\u201d for a few days. When users searched the musician\u2019s name, they would see a notice that an error had occurred.<\/p>\n This content moderation failure became a national news story, since Taylor Swift is Taylor Swift. But if social platforms can\u2019t protect one of the most famous women in the world, who can they protect?<\/p>\n \u201cIf you have what happened to Taylor Swift happen to you, as it\u2019s been happening to so many people, you\u2019re likely not going to have the same amount of support based on clout, which means you won\u2019t have access to these really important communities of care,\u201d Dr. Carolina Are, a fellow at Northumbria University\u2019s Centre for Digital Citizens in the U.K., told TechCrunch. \u201cAnd these communities of care are what most users are having to resort to in these situations, which really shows you the failure of content moderation.\u201d<\/p>\n Banning the search term \u201ctaylor swift\u201d is like putting a piece of Scotch tape on a burst pipe. There\u2019s many obvious workarounds, like how TikTok users search for \u201cseggs\u201d instead of sex. The search block was something that X could implement to make it look like they\u2019re doing something, but it doesn\u2019t stop people from just searching \u201ct swift\u201d instead. Copia Institute and Techdirt founder Mike Masnick called<\/a> the effort \u201ca sledge hammer version of trust & safety.\u201d<\/p>\n \u201cPlatforms suck when it comes to giving women, non-binary people and queer people agency over their bodies, so they replicate offline systems of abuse and patriarchy,\u201d Are said. \u201cIf your moderation systems are incapable of reacting in a crisis, or if your moderation systems are incapable of reacting to users\u2019 needs when they\u2019re reporting that something is wrong, we have a problem.\u201d<\/p>\n So, what should X have done to prevent the Taylor Swift fiasco anyway?<\/p>\n Are asks these questions as part of her research<\/a>, and proposes that social platforms need a complete overhaul of how they handle content moderation. Recently, she conducted a series of roundtable discussions with 45 internet users from around the world who are impacted by censorship and abuse to issue recommendations to platforms about how to enact change.<\/p>\n One recommendation is for social media platforms to be more transparent with individual users about decisions regarding their account or their reports about other accounts.<\/p>\n \u201cYou have no access to a case record, even though platforms do have access to that material \u2013 they just don\u2019t want to make it public,\u201d Are said. \u201cI think when it comes to abuse, people need a more personalized, contextual and speedy response that involves, if not face-to-face help, at least direct communication.\u201d<\/p>\n X announced this week that it would hire 100 content moderators to work out of a new \u201cTrust and Safety\u201d center in Austin, Texas. But under Musk\u2019s purview, the platform has not set a strong precedent for protecting marginalized users<\/a> from abuse. It can also be challenging to take Musk at face value, since the mogul has a long track record of failing to deliver on his promises. When he first bought Twitter, Musk declared he would form a content moderation council<\/a> before making major decisions. This did not happen.<\/p>\n In the case of AI-generated deepfakes, the onus is not just on social platforms. It\u2019s also on the companies who create consumer-facing generative AI products.<\/p>\n According to an investigation by 404 Media<\/a>, the abusive depictions of Swift came from a Telegram group devoted to creating nonconsensual, explicit deepfakes. The users in the group often use Microsoft Designer, which draws from Open AI\u2019s DALL-E 3 to generate images based on inputted prompts. In a loophole<\/a> that Microsoft has since addressed, users could generate images of celebrities by writing prompts like \u201ctaylor \u2018singer\u2019 swift\u201d or \u201cjennifer \u2018actor\u2019 aniston.\u201d<\/p>\n