Dystopia outweighs everything.
Harel and Brownsword, 19—law professor at the Hebrew University of Jerusalem AND Professor in Law at King's College London (Alon and Roger, “Law, liberty and technology: criminal justice in the context of smart machines,” International Journal of Law in Context, Volume 15, Special Issue 2, June 2019, pp. 107-125, dml) [language modifications denoted by brackets]
Famously, Stephen Hawking (2018, p. 188) remarked that ‘the advent of super-intelligent AI would be either the best or the worst thing ever to happen to humanity’. At best, smart machines, smart policing and smart cities of the kind contemplated by Elizabeth Joh might signal the end of crime; but, at worst, we can imagine various dystopian futures where the existential and agential threats presented by AI have been realised. Given, in James Bridle's (2018, p. 2) words, that our technologies are complicit in ‘an out-of-control economic system that immiserates many and continues to widen the gap between rich and poor; the collapse of political and societal consensus across the globe resulting in increasing nationalisms, social divisions, ethnic conflicts and shadow wars; and a warming climate, which existentially threatens us all’, then Vincent Chiao might well be right in claiming that the turn to smart technology might not be the smartest [best] way of trying to achieve the end of crime.
In this collection, our contributors have not highlighted concerns of an existential nature. Nevertheless, we might fear that, in our quest for crime-free societies, for greater safety and well-being, we will develop and embed ever more intelligent devices to the point that there is a risk of the extinction of humans – or, if not that, then a risk of humanity surviving ‘in some highly suboptimal state or in which a large portion of our potential for desirable development is irreversibly squandered’ (Bostrom, 2014, p. 281, note 1; see also Ford, 2015). Our contributors have not yet recommended that we should follow the example of Samuel Butler's Erewhonians who, fearful for their liberty, destroyed their machines (Butler, 1872) – and who also, of course, inverted conventional wisdom by punishing those who fell ill while, by contrast, treating in hospital and sympathising with those who committed crimes such as forging cheques, setting property on fire or robbing with violence. Yet, the beauty of Erewhon is that, to some present-day readers – particularly readers who are familiar with, say, Harari's Homo Deus (2016) 15 or Häggerström's Here be Dragons (2016) – the practices of the Erewhonians might seem to be anything but benighted. Is it so ridiculous to think that, with the acceleration in technological development, machines might become much smaller and smarter, capable of reproducing themselves, communicating with one another and displaying various degrees of intelligence (if not consciousness as humans experience it) and agency? Most importantly, which policy would be the more crazy [imprudent]: to disregard machines as a threat to the human condition or to treat the threat as sufficiently serious to warrant at least some precautionary measures – albeit perhaps not precaution on the scale exercised by the Erewhonians, who destroyed ‘all the inventions that had been discovered for the preceding 271 years’ (Butler, 1872, p. 260)?
Such, however, are not the most explicit concerns of our contributors. Rather, the concerns expressed by Bowling and Iyer, by Lynskey and by Macdonald, Correia and Watkin relate to our agential interests and, in particular to our interests in privacy, in the fair collection and processing of our personal data and in access to (and the integrity of) the informational eco-system. Increasingly, it is being recognised that such interests are ‘contextual’ not only in the sense that their demands might vary from one context to another, but in the more fundamental sense that we have a common interest in a context that enables our self-development (Hu, 2017; Brincker, 2017). This is nicely expressed in a paper (discussing data governance) from the Royal Society and British Academy:
‘Future concerns will likely relate to the freedom and capacity to create conditions in which we can flourish as individuals; governance will determine the social, political, legal and moral infrastructure that gives each person a sphere of protection through which they can explore who they are, with whom they want to relate and how they want to understand themselves, free from intrusion or limitation of choice.’ (Royal Society and British Academy, 2016, p. 5)
With data being gathered, in both the public and the private sector, on an unprecedented scale (Vaidhyanathan, 2011; Galloway, 2017), we might treat such dataveillance as compromising the conditions for self-development and agency (Pasquale, 2015). Moreover, we might fear that, where data are used to train smart machines that sift and sort citizens (as mooted by the Chinese social credit system) (Chen and Cheung, 2017), then, in Glen Greenwald's (2014, p. 6) words, this could be the precursor to a truly dystopian ‘system of omnipresent monitoring and control’.
Finally, there is the subtle and insidious way in which smart machines might compromise the conditions for moral development. If we accept that the fundamental aspiration of any moral community is that its members should try to do the right thing, then this presupposes a process of moral reflection and action that accords with one's moral judgment. Of course, this does not imply that each agent will make the same moral judgment or apply the same reasons. A utilitarian community is very different to a Kantian community; but, in both cases, these are moral communities and it is their shared aspiration to do the right thing that is the lowest common denominator (Brownsword, 2013; 2018a). Arguably, liberty – in the sense of having the practical option of doing both the right thing and the wrong thing – is critical to moral community. On the East coast, where crime is rife and where prudential reasoning dominates, the moral project is poorly realised; but it is at least a community with moral possibilities and with room for moral improvement. By contrast, in the well-ordered technologically managed West coast, if the possibility of moral community is lost, then, as Beyleveld and Brownsword emphasise, this should certainly give us pause about the direction of travel in the criminal justice system.
The ability to do the right thing also hinges not only on individual deliberation, but also on public moral deliberation. The automated processes designed to disable crime also typically mute and disable public moral deliberation. If behaviour that previously was condemned and prohibited has become impossible to engage in (due to technological innovations), we are less likely to debate its justifiability. We will never know whether speed limits are justified unless some people violate them; we can never know whether certain restrictions on movement promote the public interest if such restrictions are enforced perfectly by using technological innovations. In other words, automated processes do not only erode individual moral sensibilities; they also erode public moral deliberation.