Algorithms’ superior abilities of “perception” creates rash reactionary politics
Amoore 20 (Louise Amoore, Professor of Political Geography at Durham University, “Cloud Ethics: Algorithms and the Attributes of Ourselves and Others”, Duke University Press, 8/30/20, https://doi.org/10.1080/1369118X.2020.1800782) // EL
Critical accounts of the rise of algorithms have placed great emphasis on the power of algorithms to visualize, to reprogram vision, or indeed even to “see” that which is not otherwise available to human regimes of visuality. Similar to the spatial arrangement, this primacy of the visual register has also annexed what could count as the ethics and politics of algorithms. There are two curiously twinned accounts of contemporary algorithms in relation to regimes of sight and vision. The first is that algorithms operate on a plane in excess of human visibility and at scales that are inscrutable to the human. The second is that algorithms themselves have an enhanced capacity to visualize the invisible, to see, scan, and search volumes and varieties of data heretofore unavailable to human senses. Indeed, this intersection of machinic and human vision comes to the fore in the espoused ethics of public inquiries into the state’s deployment of automated algorithms for the government of the population. For example, in the UK parliamentary inquiry following the Edward Snowden disclosures of widespread automated data analysis, a peculiar kind of virtue was found in the notion that, in machine learning intelligence, “only a tiny fraction of those collected are ever seen by human eyes.” Similarly in the United States, the former director of national intelligence James Clapper likened the nsa’s algorithmic analysis of citizens’ data to a form of library in which few books are “actually read” and where the output of the system supplies “the books that we need to open up and actually read.” There is an acute problem, then, with the widespread appeal to ethical codes that regulate what algorithms or humans do or do not see. It is a problem, I suggest, with its roots in the privileging of sight and vision over other forms of making things perceptible. “Vision cannot be taken,” writes Orit Halpern in her wonderful book Beautiful Data, “as an isolated form of perception” but must be considered “inseparable from other senses.” To act and to be responsible for action, an algorithm need not “see” or “read” but need only make something or someone perceptible and available to the senses. In this book I situate the ethics and politics of algorithms within a genealogy of technologies of perception. Contemporary algorithms are changing the processes by which people and things are rendered perceptible and brought to attention. This is definitively not merely a matter of making things amenable to vision and indeed is frequently a matter of sustaining something beneath the visual register and yet perceptible. As art historian Jonathan Crary writes, “Ideas about perception and attention were transformed” alongside the historical “emergence of new technological forms of spectacle, display, projection, attention, and recording.” Understood in this way, the transformation of perception involves changes in how the perceiving subject thinks about what could be brought to attention, changes in the horizon of possibility of human action. As with the advent of the technologies of printing press, camera, or cinema, so the advent of the machine learning algorithm implies a reworking of what it means to perceive and mediate things in the world. This is not a process that is effectively captured by the idea that automated systems are undermining or superseding human forms of perception and action. To foreground instruments of perception, or what Henri Bergson terms “organs of perception,” is to breach conventional distinctions between humans and machines and acknowledge the entangled nature of all forms of perception. Bergson insists on the shared limits of perception across science and ordinary everyday experience, so that “ordinary knowledge is forced, like scientific knowledge,” to divide up time into perceptible slices, to “take things in a time broken up into particles.” Whether the organ of perception is microscope, telescope, eye, camera, or algorithm, perception is attuned to action, to the dividing up of movement into points on a trajectory so that they can be acted on. “What you have to explain,” he writes, is not “how perception arises, but how it is limited, since it should be the image of the whole, and is in fact reduced to the image of that which interests you.” Following Bergson’s insight on how an organ of perception seizes the object of interest from its environment, to consider algorithms as instruments of perception is to appreciate the processes of feature extraction, reduction, and condensation through which algorithms generate what is of interest in the data environment. Confronted by something of a moral panic surrounding the expansive volumes of “big data” and powers of surveillance of automated systems, my emphasis on practices of perception foregrounds precisely the opposing process: the reducing, distilling, and condensing of particles of interest from a whole. A defining ethical problem of the algorithm concerns not primarily the power to see, to collect, or to survey a vast data landscape, but the power to perceive and distill something for action. Algorithms function with something like an aperture—an opening that is simultaneously a narrowing, a closure, and an opening onto a scene. Let us consider, for example, an algorithm designer working in the UK defense sector, demonstrating the capacity of his deep neural network algorithms to recognize a mistaken civilian target amid a crowded data environment of drone images. He shows a slice through time as a vehicle travels away from the center of Kandahar, Afghanistan. He explains the problem for the decision: that this could be a suspect vehicle or, crucially, a school bus taking children home to villages outside the city. The algorithms had learned to recognize a school bus through training data that supplied images of predominantly yellow US-style buses. The designer explains that his algorithm is aggregated with many others to generate a single actionable output— target/no target—but to do this he must necessarily reduce and condense the patterns of interest from a volume of input data. The training data—and the humans who labeled it—have conditioned the cnn algorithm to carve out and value some objects and to discard others. In this part of Afghanistan, some of the school buses are indeed in the spatial form of a US-style school bus, but others still are open-back trucks repurposed for transporting scholars. My point is that a potential act of violence, such as a school bus wrongfully targeted by a drone strike, resides not primarily in the vertical surveilling and collecting of data, but in fact in the horizontal arraying of possible patterns of interest lodged within the algorithm itself. As an aperture instrument, the algorithm’s orientation to action has discarded much of the material to which it has been exposed. At the point of the aperture, the vast multiplicity of video data is narrowed to produce a single output on the object. Within this data material resides the capacity for the algorithm to recognize, or to fail to recognize, something or someone as a target of interest. The ethical stakes of what Mark Hansen calls “potential perceptual reconfiguration” applied to my cloud ethics necessarily involves something like a reopening of the process of the algorithm’s reduction of a multiplicity to one. What is happening in this process of condensing plural possible pathways to a single output? When an algorithm determines whether a vehicle is a military or a civilian target, or when it decides if a public protest contains latent dangerous propensities, it reduces the heterogeneity of durational time to perceive the attributes of an object and their differences of degree from other objects encountered in a past set of data. The question of what this crowd could be, what this vehicle might do, the frustrations or discomforts of the actual lived experience of waiting or gathering persist as indeterminacies in the hidden layers of the algorithm. Even within the archive of training data—sometimes just a Google ImageNet dataset of labeled images of school buses—are the residual contingencies of durational time, with all the past lived moments supplying norms and anomalies for the algorithm to learn. “My own duration, such as I live it in the impatience of waiting,” reflects Gilles Deleuze, “serves to reveal other durations that beat to other rhythms, that differ in kind from mine.” To respond to the perceptual power of the algorithm and to prize open the aperture of the single output is to trace the other durations that continue to beat in the discarded data, the multiple other potential pathways that could be mapped between fragments.