AI can’t produce stable deterrence---deepfakes and misinformation

Artificial intelligence is already used today to automate and assist in the production and spread of online disinformation, eroding trust in democratic institutions and processes and sowing confusion and polarisation among populations. Disinformation is not new, but in a digital era where most of the global population is connected, disinformation is gaining traction as an effective tool of asymmetric warfare to influence and destabilise states, while remaining below the threshold of war (Kaspersen and Rickli 2016). Russia notoriously influenced the 2016 US presidential election by purchasing ads on Facebook and Google, utilising automated bot accounts on Twitter, and using troll farms to create and spread disinformation online ahead of the election and disrupt the American democratic process (Polyakova 2018; Mueller 2019). The deliberate targeting of democratic discourse is a new type of political warfare which can be “employed to disrupt free societies anywhere in the world” (Arquilla 2021, 2).

Advances in Natural Language Processing (NLP), the branch of artificial intelligence that focuses on training computers to understand, process, and replicate human language, can have very concerning applications. Open AI recently developed the ChatGPT, which can automatically generate text about any subject by learning from massive amounts of English text articles online.3 The texts can be persuasive and nearly impossible to distinguish from text written by humans. Such capabilities have the potential to scale up a generation of misleading news articles, the impersonation of others online, the automation of the production of abusive or fake content posted on social media, or the automated production of spam and phishing content (Open AI 2019). AI enables increased dissemination of propaganda and provides “mechanisms to narrowly tailor propaganda to a target audience” (Horowitz et al. 2018, 4). With advances in machine learning and the growing amount of individual data collected on the internet, AI can profile individuals with extreme precision, predicting their preferences and behaviours (Mansted and Rosenbach 2018). This enables the kind of impact seen in the 2016 elections. It is now possible to identify those most susceptible to being influenced, targeting their fears, characteristics, vulnerabilities, and beliefs (Mansted and Rosenbach 2018). The seriousness of the threat has prompted the USAF to develop a programme with Primer (a startup company specialising in NLP) to develop a software tool to detect false narratives in public space (Tucker 2020).

Deepfake technology, which uses deep learning to alter images, videos, and audio content or even to create them from scratch in an uncanny way, further compounds AI’s disruptive potential. It is, in fact, possible today to make policymakers say things they never said or create convincing portraits of non-existent people.4 While lying and deceit are not new to military affairs, AI is compounding the problem by simultaneously democratising and sophisticating the creation of fake and misleading content (Whyte 2020, 213). This lowers the entry bar to realistic disruptive media, multiplying the numbers of actors with access to this technology and increasing their impact.

While deepfake technology is predominantly used today to create pornographic content (around 96% of deepfake videos online are pornographic), its potential for military application is clear (Ajder et al. 2019, 1). Deepfake technology fosters an environment of “plausible deniability”, where the possibility of an image or video being fake erodes our trust in what we see or hear. The distinction between what is real or not is increasingly blurred (Schick 2020). In Gabon in 2018, a video of the president suspected to be a deepfake led to an attempted coup. In 2019, the pretext of a video being a deepfake shielded the Malaysian Minister of Economic Affairs from political scandal (Ajder et al. 2019, 10). In these cases, a growing awareness of deepfakes damaged political discourse and “undermined the perceived objectivity of recorded video in politically charged contexts” (Ajder et al. 2019, 10). The end of “seeing is believing” would dramatically impact political uncertainty and crisis, accelerating the spread of propaganda and disinformation online and reducing trust in democratic institutions (Smith and Mansted 2020, 11). This phenomenon is further amplified by the fact that lies spread faster (six times) and reach a wider audience (up to a thousand times) than the truth (Vosoughi et al. 2018).