Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Remaining wedded to growth imperatives ensures that the version of artificial intelligence we inevitably produce will be the bad stuff that wipes us out

Salvador Pueyo 18 - Department of Evolutionary Biology, Ecology, and Environmental Sciences, Universitat de Barcelona. 10/01/2018. “Growth, Degrowth, and the Challenge of Artificial Superintelligence.” Journal of Cleaner Production, vol. 197, pp. 1731–1736. //Penelope/

The challenges of sustainability and of superintelligence are not independent. The changing fluxes of energy, matter, and information can be interpreted as different faces of a general acceleration.2 More directly, it is argued below that superintelligence would deeply affect production technologies and also economic decisions, and could in turn be affected by the socioeconomic and ecological context in which it develops. Along the lines of Pueyo (2014, Sec. 5), this paper presents an approach that integrates these topics. It employs insights from a variety of sources, such as ecological theory and several schools of economic theory. The next section presents a thought experiment, in which superintelligence emerges after the technical aspects of goal alignment have been resolved, and this occurs specifically in a neoliberal scenario. Neoliberalism is a major force shaping current policies on a global level, which urges governments to assume as their main role the creation and support of capitalist markets, and to avoid interfering in their functioning (Mirowski, 2009). Neoliberal policies stand in sharp contrast to degrowth views: the first are largely rationalized as a way to enhance efficiency and production (Plehwe, 2009), and represent the maximum expression of capitalist values. The thought experiment illustrates how superintelligence perfectly aligned with capitalist markets could have very undesirable consequences for humanity and the whole biosphere. It also suggests that there is little reason to expect that the wealthiest and most powerful people would be exempt from these consequences, which, as argued below, gives reason for hope. Section 3 raises the possibility of a broad social consensus to respond to this challenge along the lines of degrowth, thus tackling major technological, environmental, and social problems simultaneously. The uncertainty involved in these scenarios is vast, but, if a non-negligible probability is assigned to these two futures, little room is left for either complacency or resignation. 2. Thought experiment: superintelligence in a neoliberal scenario Neoliberalism is creating a very special breeding ground for superintelligence, because it strives to reduce the role of human agency in collective affairs.

The neoliberal pioneer Friedrich Hayek argued that the spontaneous order of markets was preferable over conscious plans, because markets, he thought, have more capacity than humans to process information (Mirowski, 2009). Neoliberal policies are actively transferring decisions to markets (Mirowski, 2009), while firms' automated decision systems become an integral part of the market's information processing machinery (Davenport and Harris, 2005).

Neoliberal globalization is locking governments in the role of mere players competing in the global market (Swank, 2016). Furthermore, automated governance is a foundational tenet of neoliberal ideology (Plehwe, 2009, p. 23). In the neoliberal scenario, most technological development can be expected to take place either in the context of firms or in support of firms.3 A number of institutionalist (Galbraith, 1985), postKeynesian (Lavoie, 2014; and references therein) and evolutionary (Metcalfe, 2008) economists concur that, in capitalist markets, firms tend to maximize their growth rates (this principle is related but not identical to the neoclassical assumption that firms maximize profits; Lavoie, 2014). Growth maximization might be interpreted as expressing the goals of people in key positions, but, from an evolutionary perspective, it is thought to result from a mechanism akin to natural selection (Metcalfe, 2008). The first interpretation is insufficient if we accept that: (1) in big corporations, the managerial bureaucracy is a coherent social-psychological system with motives and preferences of its own (Gordon, 1968, p. 639; for an insider view, see Nace, 2005, pp. 1e10), (2) this system is becoming techno-social-psychological with the progressive incorporation of decision-making algorithms and the increasing opacity of such algorithms (Danaher, 2016), and (3) human mentality and goals are partly shaped by firms themselves (Galbraith, 1985).

The type of AI best suited to participate in firms' decisions in this context is described in a recent review in Science: AI researchers aim to construct a synthetic homo economicus, the mythical perfectly rational agent of neoclassical economics. We review progress toward creating this new species of machine, machina economicus (Parkes and Wellman, 2015, p. 267; a more orthodox denomination would be Machina oeconomica).

Firm growth is thought to rely critically on retained earnings (Galbraith, 1985; Lavoie, 2014, p. 134e141). Therefore, economic selection can be generally expected to favor firms in which these are greater. The aggregate retained earnings4 RE of all firms in an economy can be expressed as:

Bold symbols represent vectors (to indicate multidimensionality). F is an aggregate production function, relying on inputs of various types of natural resources R, labor L and capital K (including intelligent machines), and being affected by environmental factors5 E; w are wages, i are returns to capital (dividends, interests) paid to households, δ is depreciation and g are the net taxes paid to governments.

Increases in retained earnings face constraints, such as tradeoffs among different parameters of Eq. (1). The present thought experiment explores the consequences of economic selection in a scenario in which two sets of constraints are nearly absent: sociopolitical constraints on market dynamics are averted by a neoliberal institutional setting, while technical constraints are overcome by asymptotically advanced technology (with extreme AI allowing for extreme technological development also in other fields). The environmental and the social implications are discussed in turn. Note that this scenario is not defined by some contingent choice of AIs' goals by their programmers: The goals of maximizing each firm's growth and retained earnings are assumed to emerge from the collective dynamics of large sets of entities subject to capitalistic rules of interaction and, therefore, to economic selection.

Sign in to recut, update, or save this card.