Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

AI systems fail to accurately identify disinformation attacks – conflicting algorithm labels are too complicated

Wooley 20 (Samuel Wooley; January 8, 2020; an assistant professor at Moody School of Communication at the University of Texas-Austin, “We’re fighting fake news AI bots by using more AI. That’s a mistake,” https://www.technologyreview.com/2020/01/08/130983/were-fighting-fake-news-ai-bots-by-using-more-ai-thats-a-mistake/)

When I was at the Oxford Internet Institute, meanwhile, we looked into how and whether Twitter bots were used during the Brexit debate. We found that while many were used to spread messages about the Leave campaign, the vast majority of the automated accounts were very simple. They were made to alter online conversation with bots that had been built simply to boost likes and follows, to spread links, to game trends, or to troll opposition. It was gamed by small groups of human users who understood the magic of memes and virality, of seeding conspiracies online and watching them grow. Conversations were blocked by basic bot-generated spam and noise, purposefully attached to particular hashtags in order to demobilize online conversations. Links to news articles that showed a politician in a particular light were hyped by fake or proxy accounts made to post and repost the same junk over and over and over. These campaigns were wielded quite bluntly: these bots were not designed to be functionally conversational. They did not harness AI. There will also need to be policies for handling situations in which an AI system makes exactly the wrong decision. Because of the limited data available in the early stages of a rapid disinformation attack, the need to quickly make a determination might lead an algorithm to invert truth and falsity and conclude that the disinformation is accurate and that the attempts to debunk it are themselves a disinformation attack. This is a less far-fetched outcome than it might initially appear to be. Algorithms, like the people who design them, can be influenced by a confirmation bias effect, leading to a boost in confidence in a wrong conclusion by selectively giving greater weight to inputs bolstering that conclusion. Particularly given the short time scales of rapid disinformation attacks, this could lead an algorithm to quickly converge on an incorrect conclusion that would need human intervention to identify and invert. For most topics and events, there simply won’t be the resources to supply staffing dedicated to individually monitoring each of the essentially limitless list of situations in which disinformation might arise. Attempts to use AI to identify disinformation will likely need to confront noisy data for the simple reason that intentional deception is involved. Most people who post an image of a car to the internet wouldn’t choose to label it “bicycle” just to throw off machine learning algorithms. But disinformation attacks will be associated with a set of conflicting claims about whether online statements are true. Returning to the Election Day example from above, in response to a tweet falsely stating that a polling location is closed, someone who has actually just voted at that location might reply with a tweet stating that the initial tweet is false and that the polling location was in fact open. That reply is, in effect, a label. An account controlled by the attackers might also reply to the initial tweet by asserting that it is true. That reply is also a label, though one that directly contradicts the reply from the real voter. Over short time scales, it would be exceedingly difficult for an algorithm—or a human—to know which label to trust. Responding quickly to disinformation thus requires addressing the twin hurdles of limited data and unreliable—and in some cases, intentionally wrong—labels of that data. This is especially true given that companies such as Facebook and Twitter operate globally; there are literally billions of accounts in nearly two hundred countries that could potentially be employed to disseminate disinformation. As a result, for the vast majority of instances of disinformation, human intervention at the social media companies will of necessity occur only after a problem is flagged either algorithmically or through manual reporting channels. Public policy will play a central role in both the human and technological aspects of the response to rapid disinformation attacks. At the technology level, policies will need to be embedded into the algorithms in relation to questions such as: What confidence level that a rapid disinformation attack is occurring should trigger notification to human managers that an activity of concern has been identified? Over what time scales should the AI system make that evaluation, and should that time scale depend on the nature and/or extent of the disinformation? For example, suspected disinformation regarding violence should clearly receive a higher priority for immediate resolution than disinformation associated with conflicting online characterizations of what a politician said at a recent campaign speech. Other questions that can drive policies to be embedded in AI disinformation detection systems include: Under what circumstances should an AI system preemptively shut down accounts suspected of originating a rapid disinformation attack? What types of autonomous actions, if any, should be taken to address posts from legitimate accounts that unwittingly contribute to propagating disinformation? Policy considerations will be an important driver for the human response as well. When an AI system identifies a potential rapid disinformation attack, managers at social media companies will need a set of guidelines for how to proceed. Policies can also guide the extent to which people at social media companies should arrange in advance to be “on call” to watch for rapid disinformation attacks. It is clear that for a short duration, high stakes events like a national election, social media companies will need to have people standing by ready to step in and address disinformation. For events in that category, the question is not whether disinformation will be present, but rather how much of it there will be, and how sophisticated the attacks will be.

Sign in to recut, update, or save this card.