The aff can’t solve - AI solutions cannot quickly identify disinformation attacks
Villasenor 20 (John Villasenor; November 23, 2020; a nonresident senior fellow in Governance Studies and the Center for Technology Innovation at Brookings. He is also a professor of electrical engineering, law, public policy, and management at UCLA, as well as co-director of the UCLA Institute for Technology, Law, and Policy. Villasenor’s work considers the technology, policy, and legal issues arising from key technology trends including the growth of artificial intelligence, the increasing complexity and interdependence of today’s networks and systems, and continued advances in computing and communications., “How to deal with AI-enabled disinformation,” https://www.brookings.edu/research/how-to-deal-with-ai-enabled-disinformation/)
One hurdle to any learning algorithm, whether supervised or unsupervised, is access to a sufficiently large set of training data. Information suitable for use as training data regarding a particular issue or question can take significant time to accumulate on social media. To be useful in AI systems for detecting disinformation, the data would in many (though not all) instances require at least some degree of manual coding at the outset. Such an approach works if the topic at issue is one—such as false claims regarding medical cures—for which the time over which it is important to continue to combat the disinformation is much longer than the time it would take to build and use a large set of training data. But it is far less effective for situations when disinformation defenses need to be deployed very quickly, and in which there will typically be a smaller amount of data that can be used as a basis for the algorithm to learn. Another issue, both in rapid and in less time-constrained attempts to identify disinformation, is the accuracy of the data labels on which an AI algorithm is relying during the learning process. “Noisy” data—data in which the labels are not necessarily accurate—is a well-known problem in machine learning. To take a simple example unrelated to disinformation attacks, consider a machine learning algorithm that is attempting to learn to automatically distinguish images of cars from images of bicycles. To do this, the algorithm might scour the internet and find millions of images that are labeled “car” and millions of other images labeled “bicycle.” In most cases, those labels will be correct. But in some instances, the labels will be incorrect; an image labeled “car” might show a truck, a bicycle, or content completely unrelated to vehicles. The higher the fraction of incorrect labels, the more difficult and slower it will be for the algorithm to learn to accurately distinguish between cars and bicycles. Working with noisy data is an active area of research, and there are emerging approaches that can help mitigate—though not completely eliminate—the loss in accuracy that results when a machine learning algorithm learns based data in which there a is substantive fraction of labeling inaccuracies.[8] Attempts to use AI to identify disinformation will likely need to confront noisy data for the simple reason that intentional deception is involved. Most people who post an image of a car to the internet wouldn’t choose to label it “bicycle” just to throw off machine learning algorithms. But disinformation attacks will be associated with a set of conflicting claims about whether online statements are true. Returning to the Election Day example from above, in response to a tweet falsely stating that a polling location is closed, someone who has actually just voted at that location might reply with a tweet stating that the initial tweet is false and that the polling location was in fact open. That reply is, in effect, a label. An account controlled by the attackers might also reply to the initial tweet by asserting that it is true. That reply is also a label, though one that directly contradicts the reply from the real voter. Over short time scales, it would be exceedingly difficult for an algorithm—or a human—to know which label to trust. Responding quickly to disinformation thus requires addressing the twin hurdles of limited data and unreliable—and in some cases, intentionally wrong—labels of that data. Responding quickly to disinformation thus requires addressing the twin hurdles of limited data and unreliable—and in some cases, intentionally wrong—labels of that data.” Researchers have recognized these issues and are developing new approaches that do not rely on a large set of pre-existing training data. In April 2020, a team of researchers from Microsoft and Arizona State University posted a pre-publication version of a paper[9] describing new results on techniques for quickly detecting fake news.[10] In the paper the authors note that traditional approaches to detecting fake news “rely on large amounts of labeled instances to train supervised models. Such large labeled training data is difficult to obtain in the early phase of fake news detection.”[11] To address this, the authors introduce a method that requires only a “small amount of manually-annotated clean data,”[12] which can be used to rapidly and automatically label a larger set of data based on posts and comments on news articles by social media users. User credibility is one of the factors considered in forming the labels. According to the authors, this approach “outperforms state-of-the-art baselines for early detection of fake news.”[13] Frameworks like this can not only help solve the problem of limited data, but could potentially also help mitigate labeling accuracy issues. As the above examples help to convey, one common theme in research addressing disinformation is the importance of measuring the credibility of online sources. Approaches to establish and then leverage credibility will be critical to quickly identifying truth in the presence of a well-constructed rapid disinformation attack. For instance, in the Election Day scenario, it would be advantageous to give high credibility weight to the social media accounts of local television and news stations and the reporters who work at those stations. That way, as soon as those stations are able to identify that the claims of closed polling places are false and disseminate that fact on social media, the AI system can calibrate truth and falsity and move to the next step of addressing the posts known to contain disinformation.[14] It is also important to recognize the limits of what AI can be expected to accomplish. Earlier this year, Samuel Woolley of UT Austin published an excerpt of his book “The Reality Game” in MIT Technology Review. In it, Woolley noted that “There simply is no easy fix to the problem of computational propaganda on social media.”[15] It would be unreasonable to expect any AI solution that will be available in the near future would be able to quickly and unambiguously identify a rapid disinformation attack. However, AI will certainly be able to provide insight into the dynamics of emerging disinformation attacks, pinpoint at least some of the social media accounts at the source, and compute confidence levels regarding the likely truth or falsity of a claim making the rounds on social media. After that, the response will need to be overseen by humans making decisions based on a combination of the AI outputs and guidance from policy frameworks.