Outweighs all other risks

Yet another serious threat comes from artificial intelligence, or AI. In the near-term, AI systems like those sold by IBM, Microsoft, Amazon and other tech giants could exacerbate inequality due to gender and racial biases. According to a paper co-authored by Timnit Gebru, the former Google employee who was fired "after criticizing its approach to minority hiring and the biases built into today's artificial intelligence systems," facial recognition software is "less accurate at identifying women and people of color, which means its use can end up discriminating against them." These are very real problems affecting large groups of people that require urgent attention.

But there are also longer-term risks, as well, arising from the possibility of algorithms that exceed human levels of general intelligence. An artificial superintelligence, or ASI, would by definition be smarter than any possible human being in every cognitive domain of interest, such as abstract reasoning, working memory, processing speed and so on. Although there is no obvious leap from current "deep-learning" algorithms to ASI, there is a good case to make that the creation of an ASI is not a matter of if but when: Sooner or later, scientists will figure out how to build an ASI, or figure out how to build an AI system that can build an ASI, perhaps by modifying its own code.

When we do this, it will be the most significant event in human history: Suddenly, for the first time, humanity will be joined by a problem-solving agent more clever than itself. What would happen? Would paradise ensue? Or would the ASI promptly destroy us?

I believe we should take the arguments for why "a plausible default outcome of the creation of machine superintelligence is existential catastrophe" very seriously. Even if the probability of such arguments being correct is low, a risk is standardly defined as the probability of an event multiplied by its consequences. And since the consequences of total annihilation would be enormous, even a low probability (multiplied by this consequence) would yield a sky-high risk.

Even more, the very same arguments for why an ASI could cause the extinction of our species also lead to the conclusion that it could obliterate the entire biosphere. Fundamentally, the risk posed by artificial superintelligence is an environmental risk. It is not just an issue of whether humanity survives or not, but an environmental issue that concerns all earthly life, which is why I have been calling for an Extinction Rebellion-like movement to form around the dangers of ASI — a threat that, like climate change, could potentially harm every creature on the planet.