\n<\/p>\n
Twitter\u2019s new owner, Elon Musk, is feverishly promoting his \u201cTwitter Files\u201d:<\/a> selected internal communications from the company, laboriously tweeted out by sympathetic amanuenses. But Musk\u2019s obvious conviction that he has released some partisan kraken is mistaken \u2014 far from conspiracy or systemic abuse, the files are a valuable peek behind the curtain of moderation at scale, hinting at the Sisyphean labors undertaken by every social media platform.<\/p>\n For a decade, companies like Twitter, YouTube, and Facebook have performed an elaborate dance to keep the details of their moderation processes equally out of reach of bad actors, regulators, and the press.<\/p>\n To reveal too much would be to expose the processes to abuse by spammers and scammers (who indeed take advantage of every leaked or published detail), while to reveal too little leads to damaging reports and rumors as they lose control over the narrative. Meanwhile, they must be ready to justify and document their methods or risk censure and fines from government bodies.<\/p>\n The result is that while everyone knows a little<\/em> about how exactly these companies inspect, filter, and arrange the content posted on their platforms, it\u2019s just enough to be sure that what we\u2019re seeing is only the tip of the iceberg.<\/p>\n Sometimes there are expos\u00e9s of the methods we suspected \u2014 by-the-hour contractors clicking through violent and sexual imagery, an abhorrent but apparently necessary industry<\/a>. Sometimes the companies overplay their hands, like repeated claims of how AI is revolutionizing moderation and subsequent reports that AI systems for this purpose are inscrutable and unreliable.<\/p>\n What almost never happens \u2014 generally companies don\u2019t do this unless they\u2019re forced to \u2014 is that the actual tools and processes of content moderation at scale are exposed with no filter. And that\u2019s what Musk has done, perhaps to his own peril, but surely to the great interest of anyone who ever wondered what moderators actually do, say, and click as they make decisions that may affect millions.<\/p>\n The email chains, Slack conversations, and screenshots (or rather shots of screens) released over the last week provide a glimpse at this important and poorly understood process. What we see is a bit of the raw material, which is not the partisan illuminati some expected \u2014 though it is clear, by its highly selective presentation, that this is what we are meant to perceive.<\/p>\n Far from it: the people involved are by turns cautious and confident, practical and philosophical, outspoken and accommodating, showing that the choice to limit or ban is not made arbitrarily but according to an evolving consensus of opposing viewpoints.<\/p>\n Leading up to the choice to temporarily restrict the Hunter Biden laptop story \u2014 probably at this point the most contentious moderation decision of the last few years, behind banning Trump \u2014 there is neither the partisanship nor conspiracy insinuated by the bombshell packaging of the documents.<\/p>\n Instead we find serious, thoughtful people attempting to reconcile conflicting and inadequate definitions and policies: What constitutes \u201chacked\u201d materials? How confident are we in this or that assessment? What is a proportionate response? How should we communicate it, to whom, and when? What are the consequences if we do, if we don\u2019t limit? What precedents do we set or break?<\/p>\n The answers to these questions are not at all obvious, and are the kind of thing usually hashed out over months of research and discussion, or even in court (legal precedents affect legal language and repercussions). And they needed to be made fast, before the situation got out of control one way or the other. Dissent from within and without (from a U.S. Representative no less \u2014 ironically, doxxed in the thread along with Jack Dorsey in violation of the selfsame policy) was considered and honestly integrated.<\/p>\n \u201cThis is an emerging situation where the facts remain unclear,\u201d said Former Trust and Safety Chief Yoel Roth. \u201cWe\u2019re erring on the side of including a warning and preventing this content from being amplified.\u201d<\/p>\n Some question the decision. Some question the facts as they have been presented. Others say it\u2019s not supported by their reading of the policy. One says they need to make the ad-hoc basis and extent of the action very clear since it will obviously be scrutinized as a partisan one. Deputy General Counsel Jim Baker calls for more information but says caution is warranted. There\u2019s no clear precedent; the facts are at this point absent or unverified; some of the material is plainly non-consensual nude imagery.<\/p>\n \u201cI believe Twitter itself should curtail what it recommends or puts in trending news, and your policy against QAnon groups is all good,\u201d concedes Rep. Ro Khanna, while also arguing the action in question is a step too far. \u201cIt\u2019s a hard balance.\u201d<\/p>\n Neither the public nor the press have been privy to these conversations, and the truth is we\u2019re as curious, and largely as in the dark, as our readers. It would be incorrect to call the published materials a complete or even accurate representation of the whole process (they are blatantly, if ineffectively, picked and chosen to fit a narrative), but even such as they are we are more informed than we were before.<\/p>\n Even more directly revealing was the next thread, which carried screenshots of the actual moderation tooling used by Twitter employees. While the thread disingenuously attempts to equate the use of these tools with shadow banning, the screenshots do not show nefarious activity, nor need they in order to be interesting.<\/p>\n Image Credits:<\/strong> Twitter<\/p>\n<\/div>\n On the contrary, what is shown is compelling for the very reason that it is so prosaic, so blandly systematic. Here are the various techniques all social media companies have explained over and over that they use, but whereas before we had it couched in PR\u2019s cheery diplomatic cant, now it is presented without comment: \u201cTrends Blacklist,\u201d \u201cHigh Profile,\u201d \u201cDO NOT TAKE ACTION\u201d and the rest.<\/p>\n Meanwhile Yoel Roth explains that the actions and policies need to be better aligned, that more research is required, that plans are underway to improve:<\/p>\n The hypothesis underlying much of what we\u2019ve implemented is that if exposure to, e.g., misinformation directly causes harm, we should use remediations that reduce exposure, and limiting the spread\/virality of content is a good way to do that\u2026 we\u2019re going to need to make a more robust case to get this into our repertoire of policy remediations \u2013 especially for other policy domains.<\/p>\n<\/blockquote>\n Again the content belies the context it is presented in: these are hardly the deliberations of a secret liberal cabal lashing out at its ideological enemies with a ban hammer. It\u2019s an enterprise-grade dashboard like you might see for lead tracking, logistics, or accounts, being discussed and iterated upon by sober-minded persons working within practical limitations and aiming to satisfy multiple stakeholders.<\/p>\n As it should be: Twitter has, like its fellow social media platforms, been working for years to make the process of moderation efficient and systematic enough to function at scale. Not just so the platform isn\u2019t overrun with bots and spam, but in order to comply with legal frameworks like FTC orders and the GDPR. (Of which the \u201cextensive, unfiltered access\u201d outsiders were given to the pictured tool may well constitute a breach. The relevant authorities told TechCrunch they are \u201cengaging\u201d with Twitter on the matter<\/a>.)<\/p>\n A handful of employees making arbitrary decisions with no rubric or oversight is no way to moderate effectively or meet such legal requirements; neither (as the resignation<\/a> of several on Twitter\u2019s Trust & Safety Council today testifies) is automation. You need a large network of people cooperating and working according to a standardized system, with clear boundaries and escalation procedures. And that\u2019s certainly what seems to be shown by the screenshots Musk has caused to be published.<\/p>\n What isn\u2019t shown by the documents is any kind of systematic bias, which Musk\u2019s stand-ins insinuate but don\u2019t quite manage to substantiate. But whether or not it fits into the narrative they wish it to, what is being published is of interest to anyone who thinks these companies ought to be more forthcoming about their policies. That\u2019s a win for transparency, even if Musk\u2019s opaque approach accomplishes it more or less by accident.<\/p>\n<\/p><\/div>\nPay no attention to the honest, complex conversation behind the curtain<\/h2>\n
Tools of the trade<\/h2>\n
\n