Search the archive.

(Helena, LSE Marxists, https://socialist.net/the-green-new-deal-system-change-or-saving-the-system/ , 5-17)

On a socialist basis, however, we could achieve the GND demands and much more. Clearly the resources for such ambitious environmental transformation do exist. The fact is that there is already an incredible amount of wealth in existence. For example, it is estimated that over £700 billion – the equivalent of one third of UK GDP – lies uninvested in the vaults of the UK’s big banks. In the US, the figure stands closer to $2 trillion ($2000 billion) – and rising. The tech giant Apple, alone, sits on a cash mountain of over $200 billion. This money sits idly in the bank accounts of big business because the bankers and bosses cannot find a profitable avenue for investment in the context of a saturated world market and a crisis-ridden global economy. Instead of trying to entice the capitalists to invest against their interests, we need to take this money out of their hands. The only way to do this is through the nationalisation of the commanding heights of the economy – the key economic levers of the banks and major monopolies. Momentum has called for banks to divest from fossil fuels, and shadow chancellor John McDonnell has proposed to delist companies from the London Stock Exchange that are not doing enough on climate change. But these kinds of proposals are insufficient. In order to guarantee that the necessary steps are taken to stop climate change, the banks and big corporations must be expropriated and put under democratic control. Only in this way can we avoid the blackmail and threats of the capitalist class, and begin to plan the economy on the basis of need. The demands put forward by Jeremy Corbyn and the Labour GND campaign for public ownership are correct. But this cannot be isolated simply to the transport and energy sectors. Instead of partial nationalisations – which leave the profit system intact (and prop it up) – what is required is a massive overhaul of all sectors of the economy, as part of a socialist plan, in order to rationalise production and utilise society’s resources in a harmonious and sustainable way. The call for expropriation and a socialist plan of production go well beyond the scope of the Green New Deal as is currently being proposed in either the US or the UK. However, every day it is becoming clearer and clearer that capitalism cannot provide a way out of this environmental or economic crisis – and so the need for bold socialist change becomes clearer also.

(HLT, Justice/SocialINquiry@ASU, Futures of Black Radicalism, ed: Johnson, Lubin)

That singular obsession with how we shall be governed should be a cause of great concern, especially for those whom Sara Ahmed calls "willful subjects"6— those who do not look to be ruled, as well as those who actively refuse to be ruled, including making themselves unavailable for governing. Indeed, democratic living, as a way of collectivizing, concerns itself not with how we should be governed, but with how we should live and relate to one another.7 In an era characterized by neoliberalism, wars, and empires, not to mention extreme wealth inequality, mass incarceration, police homicides, and deportations, livability may be predicated on democratic living, a praxis that can provide an alternative mode of being and of conducting critical social inquiries, especially about the future of Black radical thought and praxis.8 This essay builds on Cedric Robinson's critique in The Terms of Order of the political and political authority and revises James Scott's treatment of "the art of not being governed"9 to draw attention to the multiple ways in which ordinary people and communities resist governing by state and non-state rule-making projects. Historically runaway slaves looked to the North Star not to be ruled by it, but to use its illumination as a guide to freedom. The practice of marronage is embodied democratic living (however momentarily) and the willful attempt to resist being governed. Through community building, where the terror and violence of racial capitalism and white supremacy were temporarily suspended, free men and women negotiated their own terms of living, and in the process, negated the terms of order.10 Following Avery F. Gordon's suggestion that we need to "combine complex and acute social analysis with a vision of how some people have lived and do live today that is a model for how all of us could live," and Cedric Robinson's insistence that marronage (flight and fugitivity) proves the existence of Black radical consciousness and praxis, I look to the willful refusal to be governed as confirmation of democratic sensibility.11 This sensibility, I argue, is an antidote to the chronic state addiction that seems to affect many social and especially political theorists, and provides an imperative for thinking about and learning from Black radical thought and praxis. A DETOUR: STATE ADDICTION AND UNGOVERNABILITY Democratic living constitutes life forms that actively seek independence from rule making. This ungovernability from below constantly threatens the unjust peace of dominant orders, including settler colonialism, racial capitalism, white supremacy, and hetero-patriarchy.12 While contestations among the elites are frequent, they do not represent ungoverning and are quite different from the active refusal to be governed from below. The federal government shutdown, out-of-control police and prison guards, and unregulated hyper-speculative finance, for instance, are not ungoverning at the top. Instead, they are a form of governance that incites instability as a consequence of its incompetence and delinquency. In contrast, ungovernability from below represents the inability of state and other dominions to assert control over subjects; the ungovernable, therefore, are those individuals and communities that render themselves unavailable for governing. In this way, the refusal to be governed is at the core of popular resistance to all forms of dominions.13 The famed, British-trained, American political scientist Richard Rose famously stated, "Analytically, the concept of ungovernability is a nonsense."14 Rose argued that, first, "the question is not whether we shall be governed but how," second, "ungovernability can only be a temporary phase of anarchy and of civil war,"15 and, third, "for better or worse," being governed is a natural state, and being ungoverned is an aberration that needs to be corrected. This view naturalizes the idea of order and norms political authority as rational. Governability is thus ontological: we are the way we are governed, and being ungoverned is nonbeing. State evasion and other forms of avoidance of being governed are deviant behavior that necessitates disciplining, from reconditioning to total annihilation. Ideationally then, the ungovernable has been conscripted as a problem-subject, and one that needs reform. Sociologists, for instance, often focus on problem-people and communities, those who have been normed and scaled as the gendered and/or racialized other. At one time or another, indigenous people, Arabs, Mexicans, Muslims, Africans, single women, and LBGTQ people have been among the "other," who have been thought of as problem people in need of being governed, tamed, subdued, and, in certain circumstances, annihilated altogether. It is presumed and argued that these ungovernable cannot be reasoned with; they are evil and will only understand the language of force.16 Similarly, certain spaces, especially urban spaces, are seen as ungovernable. The West Side of Phoenix, Harlem, Cabrini Green, Watts, Detroit, large swaths of the Global South, and the entirety of the Third World are problem peoples and spaces because they are ungovernable, appearing together as a an entire cosmology, a fearsome Black planet. The poor, Black people of Ferguson, Chicago, East Los Angeles, and Harlem—and anywhere there are ungovernable lives—are problem subjects, especially in communities where Black lives matter. The rational choice that follow entails everything the capacious apparatuses and technologies of violence have to offer, including "evacuation," mass incarceration, deportation, tanks, guns, tear gas, and surveillance. In the aftermath of the killing of Michael Brown and the ensuing protests, Ferguson became an ungovernable problem-space, with the majority of its people and even its police force deemed troublesome. Former attorney general Eric Holder was regularly dispatched to troubled/ungovernable regions, enforcing coherence on the progressively implausible imperial narrative about American exceptionalism.17 Then there is the frequent and increasingly amplified speechifying by the first Black president in defense of liberal democracies and their attendant ideologies, justifying the perpetual wars on the troublesome Middle East or South Asia or North Africa. All the while, in political science, ungovernability scandalously continues to be dismissed as analytical nonsense. While the framing of the ungovernable is used, with great frequency and utility, to cast and organize people and lands, to justify and wage permanent wars and even genocide, in political science —the field tasked with helping us understand how we come by these policies—its mandarins persist in their views that ungovernability is conceptually nonsensical. This scandalous intent suggests an ideography that owes its pathos to and possessive investments on an epistemological ordering that relies on state addiction. This affliction is a pathology that renders its sufferers entirely dependent on the state for sensibility and intelligence. The symptoms are many and obvious: most prominent is the tendency to conflate government with governing, authority with leadership, and rule with submission.18 State-centric analytics commit addicts to a certain notion of the state, even to the extent of the state becoming the very fiction that we fear.19 Take for instance the eminent architect of deconstructionism, Jacques Derrida, who conflated sovereignty with transcendence. According to Derrida, "sovereignty is in a certain manner un-historical, it is a contract made with a history contracting itself into the punctiform event of an exceptional decision without temporal and historical expansion. Thus sovereignty also withdraws itself from language."20 So it is: timeless and outside of history; moreover, there is, "no sovereignty without violence, without the force of the stronger, the justification of which—as the right of the strongest—consists in its power over everything."21 The sovereign has power over everything, including life itself. Rather than imagining the future in terms of democractic living, sensibilities, and formations of justice, state addicts leave us dependent on the state for means of expressions and terms of engagement. Their imaginary domains22 are thus entirely dependent on the state's projections. The state's memory becomes our own memory. Just as Orlando Patterson's social death thesis fallaciously endows racial capitalism and white supremacy as cultural progenitors of Black life, so state analytics such as necropolitics23 render life sans state, more often than not, chaotic, miserable and inauthentic.24 Within the same frame of social death, as David Brion Davis has argued, Black enslaved people have "no legitimate, independent being, no place in the cosmos except as an instrument of [their] master's will."25 Similarly, life sans state, as Hobbes declared in the Leviathan, is "evil, brutish and short." Yet, empirically and intuitively, we know better. If rules are not about norms but about discovering new forms of life, as Giorgio Agamben argues in The Highest Poverty,26 then, in thinking about the modern state and rule making, the ungovernable and ungovernability are theoretical spaces that can help us think about life and the politics of living wherein ordinary people and communities assert their own renderings of life and living rather than those of the state, capital and other dominions' terms of order. As suggested by the Ethiopian proverb, it is prudent to recognize those instances when subversion disguises itself as submission or obedience because those who are oppressed cannot always exchange "a slap for a slap, an insult for an insult," tear gas for tear gas.27 Just as sounds are not always heard and fights are not always open, in the shadow of the real and Active narrations of governmentality and the awesome powers of the state, there have always lurked individuals and communities embodying governing's unsuccessful inscriptions and conscriptions. Individuals and communities remain frequently unscripted and unimpressed by the state, even as they live under constant surveillance and suppression. Life goes on, sometimes independently, all the while circumscribed by the so-called technologies of governing.28 In short, the analytical problem with ungovernability is not about empirical verification, but about the epistemological investments in political order and a modern ontological commitment to governability.29 This commitment to ruling conceals the fear of the ungovernable and ungovernablity from below. So even as we now understand that power is diffuse and relational (a la Foucault),30 state addiction ensures that; the state remains at the fulcrum of our understanding of political power31 and state reifification persists32—its promiscuous genealogy, its prodigious technologies, its capacious fantasies, and its monstrous realities.33 Within the context of an ever-growing global war on terror, the resurrection of works by the Nazi political theorist Carl Schmitt has granted little relief from our state addiction.34 If anything, his intervention further ensures that the specter of the elusive state is finally vanquished. In its place is a state more capable than the technologies it possesses, more intelligible than the apparatuses it readied, and more autonomous than the subjects it has claimed.35 Students of the Black Radical Tradition, however, are familiar with this phenomenon. Just as "every slave holder seeks to impress his slave with a belief in the boundlessness of slave territory, and of his own almost [limitless] power,"36 so, too, it seems, is the case with other dominions, including discursive ones. It is also the case that slaves did run away, and many ran away successfully! Because the state presupposes the ungovernable, it points to ungovernability as the reason for its existence and its capacious fantasies, justifying both, even as its very existence produces insecurity.37 But life as we know it is quite complicated.38 That we can speak of "governmentality"—as the conduct of statehood and a logic of governing—is precisely because complex personhood throws up ungovernability every time the state and other dominions mess with it. Population management is critical to the conduct of government precisely because the people have to be made into governable subjects. People have to be rendered governable because they are ungoverned as a precondition, and therefore are not legible to the state. It is not that our presence and complex person-hood incite governing. Rather, governing incites its prodigious technologies as it encounters life countering it or independent of it. Governing is a possessively jealous beast! So Rose, however famed he was, must be famously wrong! Throughout history, as noted, individuals and communities have always resisted modernity's state- building and rule-making projects. In addition to anti- state-building projects such as those by the hill peoples of Zomia (as documented by James Scott39), campaigns for bodily and community sovereignty, sometimes understood as forms of self-government, are well documented. These are often in reaction to or in anticipation of state-building and rule-making projects. Maroon societies are thus the embodiment of this refusal of rule making. James Scott characterizes the hill peoples as runaways and fugitives.40 In addition to these state-evaders, however, there is the motley crew of rule-evaders who elude rules and rule making that are outside of the state's perogatives. Runaways, border-crossers, gender-benders, and general nonconformists are such examples of failures of or resistance to codified practices of governing. These legacies of resistance,41 replete with women and men actively countering the many faces, structures, and technologies of violence, cruelty, and death, cannot be dismissed simply as nonsense. While we may never be able to completely escape state addiction and it would be foolish to stop interrogating the state altogether, the state's memory, however, must not be our only memory. Those who seek a more just present and future must, therefore, remain vigilantly skeptical of this affliction in order to recognize how others live, especially other genealogies and life forms that are independent of the imaginaries of the state and of capital.42 In The Terms of Order, Cedric Robinson calls for a social philosophy that rejects the political as an "ordering principle," maintaining that social order is folklore of the state, political order is an alien concept, and social leaders, more often than not, are "capricious, incompetent and mischievous."43 To demystify social order is also to interrogate how in the West, the very idea of freedom is singularly wedded to the idea of an autonomous and rational individual, yet the collectivity always requires governing44 By pivoting to ungovernability instead of the naturalness of the need to be governed, we expose the unnaturalness and illegitimacy of order. People rendering themselves unavailable for governing trips up the system, more often than not, leading to crises of authority and further exposing elite incompetence and delinquencies 45 Be it in Ferguson or Syria, the ungovernable withhold both consent and legitimation, and, in the process, render the state and its allies more transparently incompetent, brutal, and imperial. How many years has it been since the United States commenced bombing the Middle East? How long has patriarchy desired to tame the proverbial shrew? The ungovernable is thus an empirical verification of the ever-present democratic sensibility, belying the state's and its enablers' claims of intelligence, overwhelming force, and the power of exception,46 not to mention the various economic, political, racial, or biological fantasies and life forgeries. As midwives of civil strife, the ungovernable are intent on impeding and negating the unjust peace of the organized practices that render subjects governable. What Toni Morrison calls "rememory,"47 particularly fugitive rememory, is a necessary tool in the evaders' and rebels' quest for wholeness, or what Robinson calls "the preservation of [an] ontological totality."48 As Robinson insisted in Black Movements in America, running away and community building in the form of maroon societies are powerful counterexamples of the conceit of slavery—the claims of naturalness of people as property and the arrogant assumption that a people, any people, can be controlled, dominated totally. Totalitarianism frequently fails for a reason.

(Peter https://www.thephilosopher1923.org/post/the-weight-of-forever)

The political programme of longtermism is founded upon a singular speculative thesis: that we are currently at a crux point in history, where our actions may have an outsized influence upon the shape of everything that comes after. We’re in the middle of a period of rapid social and technological change, yet at the same time we’re more interconnected as a species than we’ve ever been before. MacAskill argues that this means our society is in a state like molten glass: easily shaped into new configurations that once they settle might persist for an extremely long time, influencing the lives of untold trillions across a vast swathe of space. We thus have a responsibility to ensure that it ends up in the right configuration, which first and foremost means avoiding the wrong ones. The worst outcome would be our complete extinction. This would not only eliminate whatever good we currently bring to the world, but would foreclose the possibility of those far greater goods of which we’re capable. Once we account for this loss of potential, extinction events are so much worse than even near-extinction events that tiny probabilities demand decisive action to mitigate against them. MacAskill admits that there might technically be even worse states than extinction, such as societies designed to perpetuate endless torture, but he thinks that these are vanishingly unlikely, because they require unsustainably malign intent.

(Thomas is based in the San Francisco Bay Area, and covers software development, DevOps, computer security, and more for The Register. https://www.theregister.com/2023/02/27/openai_ceo_agi/)

Emily Bender, a professor in the Department of Linguistics and the director of the Computational Linguistics Laboratory at the University of Washington in the US, analyzed Altman's post thus on Twitter: "From the get-go this is just gross," she wrote. "They think they are really in the business of developing/shaping 'AGI.' And they think they are positioned to decide what 'benefits all of humanity.'" Here's a thought experiment: imagine an AGI system that advises taxing billionaires at a rate of 95 percent and redistributing their wealth for the benefit of humanity. Will it ever be hooked into the banking system to effect its recommended changes? No, it will not. Will those minding the AGI actually carry out those orders? Again, no. No one with wealth and power is going to cede authority to software, or allow it to take away even some of their wealth and power, no matter how "smart" it is. No VIP wants AGI dictating their diminishment. And any AGI that gives primarily the powerful and wealthy more power and wealth, or maintains the status quo, is not quite what we'd describe as a technology that, as OpenAI puts it, benefits all of humanity.

(Tony, https://www.cbsnews.com/news/ai-coming-for-your-job-tech-experts-weigh-in-they-dont-replace-human-labor/)

As artificial intelligence gets better, some expect job security will get worse. In reports like one in Gizmodo earlier this month, titled, "Here Are the Jobs Our New AI Overlords Plan to Kill," coding or computer programming is often on the list. But it seems none of the AI coding software currently out there can replace the talent of professional coders. In fact, it may help them. Sam Zanca, a coder and instructor at General Assembly, a coding school in New York, said AI will "definitely automate the more tedious parts" of his job, "but it's still a long ways away before having a true impact." He's not very worried about AI right now. But "the problem is that ... I'm not entirely sure if the AI that would replace me is 10 years from now, 20 years from now, or five years from now," he said. Microsoft CEO Satya Nadella is a big believer in AI — but not the idea that AI is coming for our jobs. "I believe it creates more, I will call it both satisfaction in current jobs and net new jobs," Nadella told CBS News earlier this month. Ahead of the launch of the new AI-powered Bing, Nadella argued that artificial intelligence could help the economy across the board. "My biggest worry is we need some new technology ... that starts driving real productivity," Nadella said. "It's time for some real innovation." He believes AI will drive up wages, "because productivity and wages are related." But while that may be true in the long run, MIT economist David Autor believes the rise of AI does indeed mean millions of jobs are going to change in our lifetimes. And what's scary is: we're not sure how. "We are very good at forecasting what we will not be doing and very poor at forecasting what we will be doing," Autor said. He points out, for example, that more than 60% of the types of jobs people are doing today didn't even exist in the 1940s, while many of the jobs that did exist have been replaced. "What we've seen over the last four decades in the U.S. and many industrialized economies is what economists call labor market polarization, which means the hollowing out of the middle set of jobs ... kind of middle class jobs involving blue collar production and operative work, but also ... administrative support clerical tasks," he said. "We had banks and banks of people who answered phones, who did typing and filing, who reproduced documents, who kept tables and books. Many, many of these things are now done by machines." The "hollowing out" of the middle has led to some in the labor market moving up and making more money, while others are now making less — and "that's especially where the pain happens," Autor said. That also troubles Meredith Whittaker, who co-founded the AI Now Institute at New York University to study how all these new systems of artificial intelligence will affect our lives. "They don't replace human labor. They just require different forms of labor to sort of babysit them, to train them, to make sure they're working well," Whittaker said.

(Darren, https://www.popularmechanics.com/technology/robots/a42958546/artificial-intelligence-theory-of-mind-chatgpt/)

The AI revolution is upon us as super-advanced machines continue to master the subtle art of being human at a stunning (concerning?) speed. It’s old news that AI have bested humans at their own games, specifically things like Chess and Go, but there’s more to our brains than checking a king. There are subtler skills like inference and intuition—squishier, almost subconscious concepts that help us understand and predict the actions of others. But with the advent of advanced AI platforms like Open AI’s Generative Pre-training Transformer (GPT), even those boundaries between man and machine are beginning to fade. A new study conducted by Michal Kosinski, a computational psychologist from Stanford University, used several iterations of OpenAI’s GPT neural network—from GPT-1 to the latest GPT-3.5—to perform “Theory of Mind” (ToM) tests, a series of experiments first developed in 1978 to measure the complexity of a chimpanzee’s mind to predict the behavior of others. These tests involve solving normal, mundane scenarios that humans could easily deduce the outcome. For example, one scenario involves mislabeling a bag of popcorn as “chocolate” and then the test asks the AI to infer what the human’s response will be once the bag is opened. Kosinski’s team used “sanity checks” to analyze how well GPT networks understood the scenario and the human’s predicted response. The results were published online on arXiv, the pre-print server. This content is imported from twitter. You may be able to find the same content in another format, or you may be able to find more information, at their web site. While early versions of GPT-1, first released in 2018, scored poorly on the test, the neural network showed stunning improvement stretched across different iterations and spontaneously developed “Theory of Mind” capability of a 9-year-old human by November 2022 (the release of the latest GPT-3.5). Kosinski says this could be a “watershed moment” for AI, as the ability to understand and predict human behavior would make these engines much more useful.

(Gary Grossman is the senior VP of technology practice at Edelman and global lead of the Edelman AI Center of Excellence. https://venturebeat.com/ai/better-than-humans-ai-barrels-towards-agi/)

By these criteria, at least, ChatGPT is not AGI, and neither are DeepNash or Cicero. What they all have in common, however, is a clear advance in this direction. As Stuart Russell, professor of computer science at the University of California and a leading researcher in artificial intelligence, notes: “The actual date of arrival of general-purpose AI, you’re not going to be able to pinpoint; it isn’t just a single day. It’s also not the case that it’s all or nothing. The impact [of AI] is going to be increasing. So with every advance of AI, it significantly expands the range of tasks.” With each passing year, we can expect to see much greater capabilities on the march to AGI as these models become more sophisticated and new systems appear. Given the pace and scope of recent AI breakthroughs and the huge growth in the number of research papers, we can expect developments to come ever faster with profound implications for work and life. For example, within several years, ChatGPT or a similar system could become an app that resembles Samantha in the movie Her. ChatGPT already does some of what Samantha did: an AI that remembers prior conversations, develops insights based on those discussions, provides useful guidance and therapy and can do that simultaneously with thousands of users. Or imagine NATO using tools like DeepNash or Cicero with its members or in negotiations with rivals. We are witnessing a gathering momentum towards AGI, though experts’ estimate of its time of appearance is 2045. AGI or not, AI technology is becoming much more sophisticated and becoming deeply engrained in the fabric of our lives.

(Charles, https://venturebeat.com/ai/the-gradual-march-to-agi/)

The coming of artificial general intelligence (AGI) — the ability of an artificial intelligence to understand or learn any intellectual task that a human can — is inevitable. Despite the predictions of many experts that AGI might never be achieved or will take hundreds of years to emerge, I believe it will be here within the next decade. Why artificial general intelligence is coming How can I be so certain? We already have the know-how to produce massive programs with the capacity for processing and analyzing reams of data faster and more accurately than a human ever could. And in truth, massive programs may not be necessary anyway. Given the structure of the neocortex (the part of the human brain we use to think) and the amount of DNA needed to define it, we may be able to create a complete AGI in a program as small as 7.5 megabytes. We also have seen robots that display the kind of fluid motion controlled by 56 billion neurons in the cerebellum (the part of the human brain responsible for muscular coordination). Again, it doesn’t take a supercomputer, but a few microprocessors along with the insight as to how coordination, balance and reactions must work.

(Heather Mongilio is a reporter with USNI News , 9-13)

A grim future awaits the United States if it loses the competition with China on developing key technologies like artificial intelligence in the near future, the authors of a special government-backed study told reporters on Monday. If China wins the technological competition, it can use its advancements in artificial intelligence and biological technology to enhance its own country’s economy, military and society to the determent of others, said Robert Work, former deputy defense secretary and co-chair of the Special Competitive Studies Project, which examined international artificial intelligence and technological competition. Work is the chair of the U.S. Naval Institute Board of Directors. Losing, in Work’s opinion, means that U.S. security will be threatened as China is able to establish global surveillance, companies will lose trillions of dollars and America will be reliant on China or other countries under Chinese influence for core technologies. “If that world happens, it’s going to be very bleak for democracy … China’s sphere of influence will grow as its technological platforms proliferate throughout the world, and they will be able to establish surveillance on a global scale,” he said. “So that’s what losing looks like.”

(Peter, , 7-4)

Artificial intelligence (AI) can devise methods of wealth distribution that are more popular than systems designed by people, new research suggests. The findings, made by a team of researchers at UK-based AI company DeepMind, show that machine learning systems aren't just good at solving complex physics and biology problems, but may also help deliver on more open-ended social objectives, such as the goal of realizing a fair, prosperous society. Of course, that's not an easy task. Building a machine that can deliver beneficial results humans actually want – called "value alignment" in AI research – is complicated by the fact that people often disagree on the best method to resolve all kinds of things, and especially social, economic, and political issues. "One key hurdle for value alignment is that human society admits a plurality of views, making it unclear to whose preferences AI should align," researchers explain in a new paper, led by first author and DeepMind research scientist Raphael Koster. "For example, political scientists and economists are often at loggerheads over which mechanisms will make our societies function most fairly or efficiently." To help bridge the gap, the researchers developed an agent for wealth distribution that had people's interactions (both real and virtual) built into its training data – in effect, guiding the AI towards human-preferred (and hypothetically fairer overall) outcomes. While AIs can produce truly amazing results, they can also arrive at far-from-desirable social conclusions when left to their own devices; human feedback can help to steer neural networks in a better direction. "In AI research, there is a growing realization that to build human-compatible systems, we need new research methods in which humans and agents interact, and an increased effort to learn values directly from humans to build value-aligned AI," the researchers write. In experiments involving thousands of human participants in total, the team's AI agent – called 'Democratic AI' – studied an investment exercise called the public goods game, in which players receive varying amounts of money, and can contribute their money to a public fund, and then draw a return from the fund corresponding to their level of investment. In a series of different game styles, wealth was redistributed to players via three traditional redistribution paradigms – strict egalitarian, libertarian, and liberal egalitarian – each of which rewards player investments differently. A fourth method was also tested, called the Human Centered Redistribution Mechanism (HCRM), developed using deep reinforcement learning, using feedback data from both human players and virtual agents designed to imitate human behavior. Subsequent experiments showed that the HCRM system for paying out money in the game was more popular with players than any of the traditional redistribution standards, and also more popular than new redistribution systems designed by human referees who were incentivized to create popular systems by receiving small per-vote payments. "The AI discovered a mechanism that redressed initial wealth imbalance, sanctioned free riders, and successfully won the majority vote," the researchers explain. "We show that it is possible to harness for value alignment the same democratic tools for achieving consensus that are used in the wider human society to elect representatives, decide public policy or make legal judgements." It's worth noting that the researchers acknowledge their system raises a number of questions – chiefly, that value alignment in their AI revolves around democratic determinations, meaning the agent could actually exacerbate inequalities or biases in society (provided they are popular enough to be voted for my a majority of people). There's also the issue of trust. In the experiments, players didn't know the identity behind the wealth redistribution model they were paying for. Would they have voted the same way, knowing they'd be picking an AI over a person? For now, it's unclear. Lastly, the team says its research should not be construed as a radical technocratic proposal to overthrow how wealth is actually redistributed in society – but it is a research tool that could help humans to engineer potentially better solutions than what we have now. "Our results do not imply support for a form of 'AI government', whereby autonomous agents make policy decisions without human intervention," the authors write. "We see Democratic AI as a research methodology for designing potentially beneficial mechanisms, not a recipe for deploying AI in the public sphere."

(Annette Zimmermann is Lecturer in Philosophy at the University of York and Technology & Human Rights Fellow at the Carr Center for Human Rights, Elena Di Rosa is a PhD student in Philosophy at Princeton University., Hochan "Sonny" Kim is a PhD student in Politics at Princeton University, , 1-9)

The first thing we must do is carefully scrutinize the arguments underpinning the present use of AI. Some are optimistic that weak AI systems can contribute positively to social justice. Unlike humans, the argument goes, algorithms can avoid biased decision making, thereby achieving a level of neutrality and objectivity that is not humanly possible. A great deal of recent work has critiqued this presumption, including Safiya Noble’s Algorithms of Oppression (2018), Ruha Benjamin’s Race After Technology (2019), Meredith Broussard’s Artificial Unintelligence (2018), Hannah Fry’s Hello World (2018), Virginia Eubanks’s Automating Inequality (2018), Sara Wachter-Boettcher’s Technically Wrong (2017), and Cathy O’Neil’s Weapons of Math Destruction (2016). As these authors emphasize, there is a wealth of empirical evidence showing that the use of AI systems can often replicate historical and contemporary conditions of injustice, rather than alleviate them. In response to these criticisms, practitioners have focused on optimizing the accuracy of AI systems in order to achieve ostensibly objective, neutral decision outcomes. Such optimists concede that algorithmic systems are not neutral at present, but argue that they can be made neutral in the future, ultimately rendering their deployment morally and politically unobjectionable. As IBM Research, one of the many corporate research hubs focused on AI technologies, proclaims: AI bias will explode. But only the unbiased AI will survive. Within five years, the number of biased AI systems and algorithms will increase. But we will deal with them accordingly—coming up with new solutions to control bias in AI and champion AI systems free of it. The emerging computer science subfield of FAT ML (“fairness, accountability and transparency in machine learning”) includes a number of important contributions in this direction—described in an accessible way in two new books: Michael Kearns and Aaron Roth’s The Ethical Algorithm (2019) and Gary Marcus and Ernest Davis’s Rebooting AI: Building Artificial Intelligence We Can Trust (2019). Kearns and Roth, for example, write: We . . . . believe that curtailing algorithmic misbehavior will itself require more and better algorithms—algorithms that can assist regulators, watchdog groups, and other human organizations to monitor and measure the undesirable and unintended effects of machine learning. There are serious limitations, however, to what we might call this quality control approach to algorithmic bias. Algorithmic fairness, as the term is currently used in computer science, often describes a rather limited value or goal, which political philosophers might call “procedural fairness”—that is, the application of the same impartial decision rules and the use of the same kind of data for each individual subject to algorithmic assessments, as opposed to a more “substantive” approach to fairness, which would involve interventions into decision outcomes and their impact on society (rather than decision processes only) in order to render the former more just. Even if code is modified with the aim of securing procedural fairness, however, we are left with the deeper philosophical and political issue of whether neutrality constitutes fairness in background conditions of pervasive inequality and structural injustice. Purportedly neutral solutions in the context of widespread injustice risk further entrenching existing injustices. As many critics have pointed out, even if algorithms themselves achieve some sort of neutrality in themselves, the data that these algorithms learn from is still riddled with prejudice. In short, the data we have—and thus the data that gets fed into the algorithm—is neither the data we need nor the data we deserve. Thus, the cure for algorithmic bias may not be more, or better, algorithms. There may be some machine learning systems that should not be deployed in the first place, no matter how much we can optimize them. For a concrete example, consider the machine learning systems used in predictive policing, whereby historical crime rate data is fed into algorithms in order to predict future geographic distributions of crime. The algorithms flag certain neighborhoods as prone to violent crime. On that basis, police departments make decisions about where to send their officers and how to allocate resources. While the concept of predictive policing is worrisome for a number of reasons, one common defense of the practice is that AI systems are uniquely “neutral” and “objective,” compared to their human counterparts. On the face of it, it might seem preferable to take decision making power out of the hands of biased police departments and police officers. But what if the data itself is biased, so that even the “best” algorithm would yield biased results? This is not a hypothetical scenario: predictive policing algorithms are fed historical crime rate data that we know is biased. We know that marginalized communities—in particular black, indigenous, and Latinx communities—have been overpoliced. Given that more crimes are discovered and more arrests are made under conditions of disproportionately high police presence, the associated data is skewed. The problem is one of overrepresentation: particular communities feature disproportionately highly in crime activity data in part because of how (unfairly) closely they have been surveilled, and how inequitably laws have been enforced. It should come as no surprise, then, that these algorithms make predictions that mirror past patterns. This new data is then fed back into the technological model, creating a pernicious feedback loop in which social injustice is not only replicated, but in fact further entrenched. It is also worth noting that the same communities that have been overpoliced have been severely neglected, both intentionally and unintentionally, in many other areas of social and political life. While they are overrepresented in crime rate data sets, they are underrepresented in many other data sets (e.g. those concerning educational achievement). Structural injustice thus yields biased data through a variety of mechanisms—prominently including under- and overrepresentation—and worrisome feedback loops result. Even if the quality control problems associated with an algorithm’s decision rules were resolved, we would be left with a more fundamental problem: these systems would still be learning from and relying on data born out of conditions of pervasive and long-standing injustice.

(Alf, Innovation@Odense, The Future(s) of Innovation In Debating Innovation: Perspectives and Paradoxes of an Idealized Concept ed. Rehn and Ortenblad)

Any story about innovation is, at heart, a story about the future of innovation. This as the story of how innovation becomes always involves a forward push, an opening up to the novel, the new, the never-before-seen. Innovation is future-directed activity, focused on making the future real today, if by ever so little. Innovation, in whatever way we try to do it, is an attempt to open up today to the future, if by ever so little. At the same time, the future of anything, including innovation, is unknowable. Had this book been written in the 1980s, it is exceedingly unlikely that it would have predicted the many ways in which the internet affected things such as business model innovation, social innovation, or open innovation. The step from networking technologies to what the internet became would simply have been too enormous to contemplate. Thus, any attempt to write about the future of innovation is bound to fail and to be an exercise in futility. That said, if this book has debated what innovation is right now—for and against, from different perspectives—it would seem a dereliction of duty not to at least attempt a comment on what innovation might become. For even though some of the chapters here have been quite critical of innovation (not least my own previous chapter on image and ideology in the innovation industry—see Rehn, Chap. 5 in this volume), I believe even the more critical commentators (see, e.g., Godin 2015; Godin and Vinck 2017) would agree that we will need innovation in the future. In fact, as we look to how the future seems more and more likely to be defined by a number of “wicked problems” (Rittel and Webber 1973), it becomes clear that there will be a more and more pronounced need for new technologies, new solutions, and new ways of working. This, however, doesn’t mean that innovation won’t be forced to change, even in painful ways. Looking to wicked problems such as the ecological crisis, a phenomenon like innovation looks very much like a double-edged sword. On the one hand, innovation has wreaked havoc during the consumption-driven era of late capitalism (cf. Jameson 1991). We have created more and more ways to entertain ourselves, and in so doing also created mountains of waste, and wasted mountains of energy. Bitcoin and other innovations in the field of crypto-currency have been interest ing in the way they open up for new ways of understanding the economy, yet at the same time it has been estimated that bitcoin alone uses as much energy as Argentina (with its 45 million inhabitants) and that it thus generates emissions on about the same levels as Greece. Innovative e-commerce firms, in conjunction with advances in manufacturing, have ensured that an astounding amount of products are available to buy for consumers globally, with all of the attendant waste and logistics externali ties. In a very real way, innovation created the problem it is now asked to solve, and no one knows where the flywheel stops. On the other, we can view the current state of the world as a fait accompli and have little choice but to trust in the capacity of innovation to bring forth novel ways of both keeping the world going and at the same time lessen the impact of contemporary capitalism. Either we start consuming far less, aiming for the politically very challenging project of global degrowth, or we innovate our way(s) out of the problem. Even in the latter case, it would seem that more and more of our innovation engagements will need to take into account something greater than innovation for the sake of innovation. Instead, many commentators (see, e.g., Suchek et al. 2021) have noted that one of the core issues for innovation now is how to engage with issues such as the circular economy, and in general addressing the issue of over-exploitation of global raw materials and other resources. Whereas innovators as recently as a decade or so ago could go about creating the new new thing with casual abandon regarding any other issue of sustain ability than whether raw materials could be had at a price point where the innovation could be sold at a profit in the market, today innovation is already a different ball-game. Issues such as repairability, recyclability, design for disassembly, material re-use, and sustainable production sys tems are no longer just an issue for sustainability experts and environ mental engineers, they need to be heeded by anyone whose innovation has a material form, and most of those whose do not. The innovation calculation has changed, and while we certainly need to keep innovating, the task may well become one of not innovating more, but rather innovating better.

(Annette Zimmermann is Lecturer in Philosophy at the University of York and Technology & Human Rights Fellow at the Carr Center for Human Rights, Elena Di Rosa is a PhD student in Philosophy at Princeton University., Hochan "Sonny" Kim is a PhD student in Politics at Princeton University, , 1-9)

Conceding that these issues pose genuine problems for the possibility of a truly neutral algorithm, some might advocate for implementing countermeasures to correct for the bias in the data—a purported equalizer at the algorithmic level. While this may well be an important step in the right direction, it does not amount to a satisfactory solution on its own. Countermeasures might be able to help account for the over- and underrepresentation issues in the data, but they cannot correct for the problem what kind of data has been collected in the first place. Consider, for instance, another controversial application of weak AI: algorithmic risk scoring in the criminal justice process, which has been shown to lead to racially biased outcomes. As a well-known study by ProPublica showed in 2016, one such algorithm classified black defendants as having a “high recidivism risk” at disproportionately higher rates in comparison to white defendants even after controlling for variables such as the type and severity of the crime committed. As ProPublica put it, “prediction fails differently for black defendants”—in other words, algorithmic predictions did a much worse job of accurately predicting recidivism rates for black defendants, compared to white defendants, given the crimes that individual defendants had previously committed; black defendants who were in fact low risk were much more likely to receive a high-risk score than similar white defendants. Often, such algorithmic systems rely on socio-demographic data, such as age, gender, educational background, residential stability, and familial arrest record. Even though the algorithm in this case does not explicitly rely on race as a variable, these other socio-demographic features can function as proxies for race. The result is a digital form of redlining, or, as computer scientists call it, “redundant encoding.” In response to this problem, some states—including New Jersey—recently implemented more minimalist algorithmic risk scoring systems, relying purely on behavioral data, such as arrest records. The goal of such systems is to prevent redundant encoding by reducing the amount of socio-demographic information fed into the algorithm. However, given that communities of color are policed disproportionately heavily, and, in turn, arrested at disproportionately high rates, “purely behavioral” data about arrest history, for example, is still heavily raced (and classed, and gendered). Thus, the redundant encoding problem is not in fact solved. New Jersey’s example, among many others, shows that abstracting away from the social circumstances of defendants does not lead to true impartiality.

(Daron Acemoglu, Institute Professor at MIT , , 5-20)

One objection to the argument I have developed is that it is unwise to mess with the market. Who are we to interfere with the innovations and technological breakthroughs the market is generating? Wouldn’t intervening sacrifice productivity growth and even risk our technological vibrancy? Aren’t we better off just letting the market mechanism deliver the best technologies and then use other tools, such as tax-based redistribution or universal basic income, to make sure that everybody benefits? The answer is no, for several reasons. First, it isn’t clear that the market is doing a great job of selecting the right technologies. It is true that we are in the midst of a period of prodigious technological creativity, with new breakthroughs and applications invented every day. Yet Robert Solow’s thirty-year-old quip about computers—that they are “everywhere but in the productivity statistics”—is even more true today. Despite these mind-boggling inventions, current productivity growth is strikingly slow compared to the decades that followed World War II. This sluggishness is clear from the standard statistic that economists use for measuring how much the technological capability of the economy is expanding—the growth of total factor productivity (TFP). TFP growth answers a simple question: If we kept the amount of labor and capital resources we are using constant from one year to the next, and only our technological capabilities changed, how much would aggregate income grow? TFP growth in much of the industrialized world was rapid during the decades that followed World War II, and has fallen sharply since then. In the US, for example, the average TFP growth was close to 2 percent a year between 1920 and 1970, and has averaged only a little above 0.5 percent a year over the last three decades. So the case that the market is doing a fantastic job of expanding our productive capacity isn’t ironclad. The argument that we should rely on the market for setting the direction of technological change is weak as well. In the terminology of economics, innovation creates significant positive “externalities”: when a company or a researcher innovates, much of the benefits accrue to others. This is doubly so for technologies that create new tasks. The beneficiaries are often workers whose wages increase (and new firms that later find the right organizational structures and come up with creative products to make use of these new tasks). But these benefits are not part of the calculus of innovating firms and researchers. Ordinary market forces—which fail to take account of externalities—may therefore deter the types of technologies that have the greatest social value. This same reasoning is even more compelling when new products produce non-economic costs and benefits. Consider surveillance technologies. The demand for surveillance from repressive (and even some democratic-looking) governments may be great, generating plenty of financial incentives for firms and researchers to invest in facial recognition and snooping technologies. But the erosion of liberties is a notable non-economic cost that it is often not taken into account. A similar point holds for automation technologies: it is easy to ignore the vital role that good, secure, and high-paying jobs play in making people feel fulfilled. With all of these externalities, how can we assume that the market will get things right? Market troubles multiply further still when there are competing technological paradigms, as in the field of AI. When one paradigm is ahead of the others, both researchers and companies are tempted to herd on that leading paradigm, even if another one is more productive. Consequently, when the wrong paradigm surges ahead, it becomes very difficult to switch to more promising alternatives. Last but certainly not least, innovation responds not just to economic incentives but also to norms. What researchers find acceptable, exciting, and promising is not purely a function of economic reward. Social norms play a key role by shaping researchers’ aims as well as their moral compasses. And if the norms within the research area do not reflect our social objectives, the resulting technological change will not serve society’s best interests. All of these reasons cast doubt on the wisdom of leaving the market to itself. What’s more, the measures that might be thought to compensate for a market left to itself—redistribution via taxes and the social safety net—are both insufficient and unlikely to work. We certainly need a better safety net. (The COVID-19 pandemic has made that even clearer.) But if we do not generate meaningful employment opportunities—and thus a viable social purpose—for most people in society, how can democracy work? And if democracy doesn’t work, how can we enact such redistributive measures—and how can we be sure that they will remain in place in the future?

(Daron Acemoglu, Institute Professor at MIT , , 5-20)

Artificial Intelligence (AI) is not likely to make humans redundant. Nor will it create superintelligence anytime soon. But like it or not, AI technologies and intelligent systems will make huge advances in the next two decades—revolutionizing medicine, entertainment, and transport; transforming jobs and markets; enabling many new products and tools; and vastly increasing the amount of information that governments and companies have about individuals. Should we cherish and look forward to these developments, or fear them? There are reasons to be concerned. Current AI research is too narrowly focused on making advances in a limited set of domains and pays insufficient attention to its disruptive effects on the very fabric of society. If AI technology continues to develop along its current path, it is likely to create social upheaval for at least two reasons. For one, AI will affect the future of jobs. Our current trajectory automates work to an excessive degree while refusing to invest in human productivity; further advances will displace workers and fail to create new opportunities (and, in the process, miss out on AI’s full potential to enhance productivity). For another, AI may undermine democracy and individual freedoms. Each of these directions is alarming, and the two together are ominous. Shared prosperity and democratic political participation do not just critically reinforce each other: they are the two backbones of our modern society. Worse still, the weakening of democracy makes formulating solutions to the adverse labor market and distributional effects of AI much more difficult. These dangers have only multiplied during the COVID-19 crisis. Lockdowns, social distancing, and workers’ vulnerability to the virus have given an additional boost to the drive for automation, with the majority of U.S. businesses reporting plans for more automation. None of this is inevitable, however. The direction of AI development is not preordained. It can be altered to increase human productivity, create jobs and shared prosperity, and protect and bolster democratic freedoms—if we modify our approach. In order to redirect AI research toward a more productive path, we need to look at AI funding and regulation, the norms and priorities of AI researchers, and the societal oversight guiding these technologies and their applications.

(Daron Acemoglu, Institute Professor at MIT , , 5-20)

Much of the diagnosis I have presented thus far is not new. Many decry the disruption that automation has already produced and is likely to cause in the future. Many are also concerned about the deleterious effects new technologies might have on individual liberty and democratic procedure. But the majority of these commentators view such concerns with a sense of inevitability—they believe that it is in the very nature of AI to accelerate automation and to enable governments and companies to control individuals’ behavior. Yet society’s march toward joblessness and surveillance is not inevitable. The future of AI is still open and can take us in many different directions. If we end up with powerful tools of surveillance and ubiquitous automation (with not enough tasks left for humans to perform), it will be because we chose that path. Where else could we go? Even though the majority of AI research has been targeted toward automation in the production domain, there are plenty of new pastures where AI could complement humans. It can increase human productivity most powerfully by creating new tasks and activities for workers. Let me give a few examples. The first is education, an area where AI has penetrated surprisingly little thus far. Current developments, such as they are, go in the direction of automating teachers —for example, by implementing automated grading or online resources to replace core teaching tasks. But AI could also revolutionize education by empowering teachers to adapt their material to the needs and attitudes of diverse students in real time. We already know that what works for one individual in the classroom may not work for another; different students find different elements of learning challenging. AI in the classroom can make teaching more adaptive and student-centered, generate distinct new teaching tasks, and, in the process, increase the productivity of—and the demand for—teachers. The situation is very similar in health care, although this field has already witnessed significant AI investment. Up to this point, however, there have been few attempts to use AI to provide new, real-time, adaptive services to patients by nurses, technicians, and doctors. Similarly, AI in the entertainment sector can go a long way toward creating new, productive tasks for workers. Intelligent systems can greatly facilitate human learning and training in most occupations and fields by making adaptive technical and contextual information available on demand. Finally, AI can be combined with augmented and virtual reality to provide new productive opportunities to workers in blue-collar and technical occupations. For example, it can enable them to achieve a higher degree of precision so that they can collaborate with robotics technology and perform integrated design tasks. In all of these areas, AI can be a powerful tool for deploying the creativity, judgment, and flexibility of humans rather than simply automating their jobs. It can help us protect their privacy and freedom, too. Plenty of academic research shows how emerging technologies— differential privacy, adversarial neural cryptography, secure multi-party computation, and homomorphic encryption, to name a few—can protect privacy and detect security threats and snooping, but this research is still marginal to commercial products and services. There is also growing awareness among both the public and the AI community that new technologies can harm public discourse, freedom, and democracy. In this climate many are demanding a concerted effort to use AI for good. Nevertheless, it is remarkable how much of AI research still focuses on applications that automate jobs and increase the ability of governments and companies to monitor and manipulate individuals. This can and needs to change.

(Erik Brynjolfsson is Jerry Yang and Akiko Yamazaki Professor and Senior Fellow at the Stanford Institute for Human-Centered AI, , 5-20)

Acemoglu is right on three key points. First, core AI technologies are indeed advancing rapidly and becoming increasingly powerful. Improvements in machine learning, in particular—such as the deep learning techniques that made such rapid progress on the Imagenet dataset—are affecting more and more of the economy. In the right applications, the payoff can be large. Adoption is still in its early days—only 1.3 percent of firms in the United States have adopted robotics, for instance—but the numbers are growing rapidly. Second, the societal implications of AI are profound, especially regarding the future of work and our individual freedoms and democracy. Third, and most important, outcomes are not preordained. This last point is not to be taken for granted. The most common question I get from audiences when I speak about AI is some version of: What will AI do to society? But this is not the right question to ask; it erases our agency. Acemoglu deserves special credit not simply for diagnosing the challenges created by these technologies, but also for suggesting a set of specific solutions. (In fact Acemoglu has long been one of the most powerful and rigorous advocates of the idea that we can and should direct the course of technical change. He published an influential article making this argument nearly twenty years ago.) Regarding Acemoglu’s diagnosis, I would emphasize the following details. When it comes to AI’s effect on the workforce, the real challenge is wages, not jobs. While employment has grown over the past forty years, real wages for Americans with a high school education or less have fallen. Tyler Cowen and others have argued that this is evidence of a lack of technological progress, but overall GDP and GDP per capita have also grown, and 2019 saw a record number of billionaires. Drawing on the work of Acemoglu as well as David Autor, Lawrence Katz, Melissa Kearney, Frank Levy, and Richard Murnane, Andrew McAfee and I have made the case that advances in technology were not inconsistent with falling wages for some or even large part of the workforce. The scale of these changes, and the bigger ones yet to come, is massive. The value of all the human capital in the United States—the sum of American workers’ skills, experience, education, and knowhow—is likely around $240 trillion. That implies that if our decisions change the trajectory of technology’s effects on the U.S. economy enough to cause even a 10 percent change in that value, it would be worth more than an entire year’s GDP (currently $21 trillion). As for AI’s effect on democracy, we ought to be concerned about increasing polarization and enabling Orwellian levels of surveillance. As Marshall van Alstyne and I have written, it is precisely because digital technologies better enable us to find content and people we like that they can also separate and polarize us. These technologies can also massively amplify the power of the state to monitor the words and actions of its citizens, giving it the power to not only silence critics but even shape their thoughts. The implications are increasingly recognized by world leaders; as even Vladimir Putin has put it, “Whoever becomes the leader in this sphere will become the ruler of the world.” In the wrong hands, the result may be what Jean Tirole calls a digital dystopia. But no outcome is inevitable. We have the ability to direct AI, just as we can direct other types of technical change. Let me focus on three groups that can and should play a role in shaping AI for good: technologists, managers, and policymakers.

(Rodrigo, , 12-20)

At the Media Lab, I learned that the discourse of “ethical AI,” championed substantially by Ito, was aligned strategically with a Silicon Valley effort seeking to avoid legally enforceable restrictions of controversial technologies. A key group behind this effort, with the lab as a member, made policy recommendations in California that contradicted the conclusions of research I conducted with several lab colleagues, research that led us to oppose the use of computer algorithms in deciding whether to jail people pending trial. Ito himself would eventually complain, in private meetings with financial and tech executives, that the group’s recommendations amounted to “whitewashing” a thorny ethical issue. “They water down stuff we try to say to prevent the use of algorithms that don’t seem to work well” in detention decisions, he confided to one billionaire. I also watched MIT help the U.S. military brush aside the moral complexities of drone warfare, hosting a superficial talk on AI and ethics by Henry Kissinger, the former secretary of state and notorious war criminal, and giving input on the U.S. Department of Defense’s “AI Ethics Principles” for warfare, which embraced “permissibly biased” algorithms and which avoided using the word “fairness” because the Pentagon believes “that fights should not be fair.” Ito did not respond to requests for comment. MIT LENT CREDIBILITY to the idea that big tech could police its own use of artificial intelligence at a time when the industry faced increasing criticism and calls for legal regulation. Just in 2018, there were several controversies: Facebook’s breach of private data on more than 50 million users to a political marketing firm hired by Donald Trump’s presidential campaign, revealed in March 2018; Google’s contract with the Pentagon for computer vision software to be used in combat zones, revealed that same month; Amazon’s sale of facial recognition technology to police departments, revealed in May; Microsoft’s contract with the U.S. Immigration and Customs Enforcement revealed in June; and IBM’s secret collaboration with the New York Police Department for facial recognition and racial classification in video surveillance footage, revealed in September. Under the slogan #TechWontBuildIt, thousands of workers at these firms have organized protests and circulated petitions against such contracts. From #NoTechForICE to #Data4BlackLives, several grassroots campaigns have demanded legal restrictions of some uses of computational technologies (e.g., forbidding the use of facial recognition by police). Meanwhile, corporations have tried to shift the discussion to focus on voluntary “ethical principles,” “responsible practices,” and technical adjustments or “safeguards” framed in terms of “bias” and “fairness” (e.g., requiring or encouraging police to adopt “unbiased” or “fair” facial recognition). In January 2018, Microsoft published its “ethical principles” for AI, starting with “fairness.” In May, Facebook announced its “commitment to the ethical development and deployment of AI” and a tool to “search for bias” called “Fairness Flow.” In June, Google published its “responsible practices” for AI research and development. In September, IBM announced a tool called “AI Fairness 360,” designed to “check for unwanted bias in datasets and machine learning models.” In January 2019, Facebook granted $7.5 million for the creation of an AI ethics center in Munich, Germany. In March, Amazon co-sponsored a $20 million program on “fairness in AI” with the U.S. National Science Foundation. In April, Google canceled its AI ethics council after backlash over the selection of Kay Coles James, the vocally anti-trans president of the right-wing Heritage Foundation. These corporate initiatives frequently cited academic research that Ito had supported, at least partially, through the MIT-Harvard fund. To characterize the corporate agenda, it is helpful to distinguish between three kinds of regulatory possibilities for a given technology: (1) no legal regulation at all, leaving “ethical principles” and “responsible practices” as merely voluntary; (2) moderate legal regulation encouraging or requiring technical adjustments that do not conflict significantly with profits; or (3) restrictive legal regulation curbing or banning deployment of the technology. Unsurprisingly, the tech industry tends to support the first two and oppose the last. The corporate-sponsored discourse of “ethical AI” enables precisely this position. Consider the case of facial recognition. This year, the municipal legislatures of San Francisco, Oakland, and Berkeley — all in California — plus Somerville, Massachusetts, have passed strict bans on facial recognition technology. Meanwhile, Microsoft has lobbied in favor of less restrictive legislation, requiring technical adjustments such as tests for “bias,” most notably in Washington state. Some big firms may even prefer this kind of mild legal regulation over a complete lack thereof, since larger firms can more easily invest in specialized teams to develop systems that comply with regulatory requirements. Thus, Silicon Valley’s vigorous promotion of “ethical AI” has constituted a strategic lobbying effort, one that has enrolled academia to legitimize itself. Ito played a key role in this corporate-academic fraternizing, meeting regularly with tech executives. The MIT-Harvard fund’s initial director was the former “global public policy lead” for AI at Google. Through the fund, Ito and his associates sponsored many projects, including the creation of a prominent conference on “Fairness, Accountability, and Transparency” in computer science; other sponsors of the conference included Google, Facebook, and Microsoft. Although the Silicon Valley lobbying effort has consolidated academic interest in “ethical AI” and “fair algorithms” since 2016, a handful of papers on these topics had appeared in earlier years, even if framed differently. For example, Microsoft computer scientists published the paper that arguably inaugurated the field of “algorithmic fairness” in 2012. In 2016, the paper’s lead author, Cynthia Dwork, became a professor of computer science at Harvard, with simultaneous positions at its law school and at Microsoft. When I took her Harvard course on the mathematical foundations of cryptography and statistics in 2017, I interviewed her and asked how she became interested in researching algorithmic definitions of fairness. In her account, she had long been personally concerned with the issue of discriminatory advertising, but Microsoft managers encouraged her to pursue this line of work because the firm was developing a new system of online advertising, and it would be economically advantageous to provide a service “free of regulatory problems.” (To be fair, I believe that Dwork’s personal intentions were honest despite the corporate capture of her ideas. Microsoft declined to comment for this article.) After the initial steps by MIT and Harvard, many other universities and new institutes received money from the tech industry to work on AI ethics. Most such organizations are also headed by current or former executives of tech firms. For example, the Data & Society Research Institute is directed by a Microsoft researcher and initially funded by a Microsoft grant; New York University’s AI Now Institute was co-founded by another Microsoft researcher and partially funded by Microsoft, Google, and DeepMind; the Stanford Institute for Human-Centered AI is co-directed by a former vice president of Google; University of California, Berkeley’s Division of Data Sciences is headed by a Microsoft veteran; and the MIT Schwarzman College of Computing is headed by a board member of Amazon. During my time at the Media Lab, Ito maintained frequent contact with the executives and planners of all these organizations.

(Annette Zimmermann is Lecturer in Philosophy at the University of York and Technology & Human Rights Fellow at the Carr Center for Human Rights, Elena Di Rosa is a PhD student in Philosophy at Princeton University., Hochan "Sonny" Kim is a PhD student in Politics at Princeton University, , 1-9)

San Francisco, for example, recently banned the use of facial recognition tools in policing due to the increasing amount of empirical evidence of algorithmic bias in law enforcement technology; the city’s board of supervisors passed the “Stop Secret Surveillance” ordinance in an 8-1 vote. The ordinance, which states that “the technology will exacerbate racial injustice and threaten our ability to live free of continuous government monitoring,” also establishes more wide-ranging accountability mechanisms beyond the use of facial recognition technology in policing, such as a requirement that city agencies obtain approval before purchasing and implementing other types of surveillance technology. Broaching questions of algorithmic justice via the democratic process would give members of communities most impacted by algorithmic bias more direct democratic power over crucial decisions concerning weak AI—not merely after its deployment, but also at the design stage. San Francisco’s new ordinance exemplifies the importance of providing meaningful opportunities for bottom-up democratic deliberation about, and democratic contestation of, algorithmic tools ideally before they are deployed: “Decisions regarding if and how surveillance technologies should be funded, acquired, or used, and whether data from such technologies should be shared, should be made only after meaningful public input has been solicited and given significant weight.” Similar local, bottom-up procedures akin to San Francisco’s model can and should be implemented in other communities, rather than simply waiting for the creation of more comprehensive, top-down regulatory institutions. Significant public oversight over AI development and deployment is already possible. Rather than allowing tech practitioners to navigate the ethics of AI by themselves, we the public should be included in decisions about whether and how AI will be deployed and to what ends. Furthermore, even once we do create new, larger-scale democratic institutions empowered to legally regulate AI in a meaningful way, bottom-up democratic procedures will still be essential: they play a crucial role in identifying which agendas such institutions ought to pursue, and they can shine a light on whose interests are most affected by emerging technologies. Moving to this agenda-setting approach means incorporating an ex ante perspective when we think about algorithmic accountability, rather than resigning ourselves to a callous, ex post, “wait and see” attitude. When we wait and see, we let corporations and practitioners take the lead and set the terms. In the meantime, we expect those who are already experiencing significant social injustice to continue bearing its burden. Of course, shifting democratic agendas toward decisions about algorithmic tools will not entirely resolve the problem of algorithmic bias. To take full responsibility for how technology shapes our lives going forward, we will have to make the deployment of weak AI democratically contestable by putting it on our democratic agendas. That being said, we will eventually have to combine that strategy with an effort to establish institutions that can enforce the just and equitable use of technology after it has been deployed. In other words, a democratic critique of algorithmic injustice requires both an ex ante and an ex post perspective. In order for us to start thinking about ex post accountability in a meaningful way—that is, in a way that actually reflects the concerns and lived experiences of those most affected by algorithmic tools—we need to first make it possible for society as a whole, not just tech industry employees, to ask the deeper ex ante questions (e.g. “Should we even use weak AI in this domain at all?”). Changing the democratic agenda is a prerequisite to tackling algorithmic injustice, not just one policy goal among many. Democratic agenda setting can be a powerful mechanism for exercising popular control over state and corporate use of technology, and for contesting technology’s threats to our rights. Effective agenda-setting, of course, will mean coupling the public’s agenda setting power with tangible bottom-up decision making power, rather than merely exercising our rights of deliberation and consultation. This is where we can learn from other recent democratic innovations, such as participatory budgeting, in which local and municipal decisions about how to allocate resources for infrastructure, energy, healthcare, and environmental policy are being made directly by residents themselves after several rounds of collective deliberation. Enabling more robust democratic participation from the outset helps us identify the kinds of concerns and problems that we ought to prioritize. Rather than rushing to quick, top-down solutions aimed at quality control, optimization, and neutrality, we must first clarify what particular kind of problem we are trying to solve in the first place. Until we do so, algorithmic decision making will continue to entrench social injustice, even as tech optimists herald it as the cure for the very ills it exacerbates.

(Annette Zimmermann is Lecturer in Philosophy at the University of York and Technology & Human Rights Fellow at the Carr Center for Human Rights, Elena Di Rosa is a PhD student in Philosophy at Princeton University., Hochan "Sonny" Kim is a PhD student in Politics at Princeton University, , 1-9)

This also means that we need, in part, to think critically about government decisions to procure machine learning tools from private corporations—especially because these tools are subsequently used to partially automate decisions that were previously made by democratically authorized, if not directly elected, public officials. But we will also have to ask uncomfortable questions about our own role as a public in authorizing and contesting the use of AI technologies by corporations and the state. Citizens must come to view issues surrounding AI as a collective problem for all of us rather than a technical problem just for them. Our proposal is aligned with an emerging “second wave” of thinking about algorithmic accountability, as legal scholar Frank Pasquale puts it: a perspective which critically questions “whether [certain algorithmic systems] should be used at all—and, if so, who gets to govern them,” rather than asking how such systems might be improved in order to make them more fair. This “second” perspective has immediate implications for how we ought to think about the relationship between democratic power and AI. As Julia Powles and Helen Nissenbaum emphasize, “Any AI system that is integrated into people’s lives must be capable of contest, account, and redress to citizens and representatives of the public interest.” If using algorithmic decision making means making deliberate choices for which we are all on the hook, how exactly should we as a democratic society respond? What is our role, as citizens, in shaping technology for a more just society? One might be tempted to think that the unprecedented novelty of machine learning can be regulated effectively only if we manage to create entirely new democratic procedures and institutions. A range of such measures have been proposed: creating new governmental departments (such as the new Department of Technology proposed by Democratic presidential candidate Andrew Yang), new laws (such as consumer protection laws, enforced by a kind of “FDA for algorithms”), as well as (rather weak) new measures for voluntary individual self-regulation, such as a Hippocratic oath for developers. One problem with these proposals is that they reinforce learned helplessness about algorithmic bias; they suggest that until we implement large-scale, complex institutional and social change, we will be unable to address algorithmic bias in any meaningful way. But there is another way to think about algorithmic accountability. Rather than advocating for entirely new democratic institutions and procedures, why not first try to shift existing democratic agendas? Democratic agenda-setting means enabling citizens to contest and control the concepts that underpin algorithmic decision rules and to deliberate about whether algorithmic decision-making ought to be used in a particular domain in the first place.