Search the archive.

In short: we respond to a recent article arguing that ensuring that “superintelligent” AI systems are safe is an impossible task. We clarify that their argument does not preclude practical work on the foundations of safety in future AI systems.

The paper  by Alfonseca et al argues that ensuring that a sufficiently complex Artificial Intelligence does not perform an unsafe action is mathematically impossible.

It seems that either deletion, escape, or deployment of the AGI is almost certain to happen eventually. Escaping from the box is likely to be an AGI’s instrumental goal. Also, the researchers will likely be eventually forced by overconfident management to remove the failsafes (i.e., deploy the AI), even if the AGI is not yet aligned.

Finally, if there is a perceived race towards AGI deployment by multiple AI organisations, this would further exacerbate their motivations to deploy prematurely. A superintelligent AGI, once on the Internet, will likely change the world in an irreversible manner, in the direction of whatever mysterious goals it evolved during its training. We can, however, guess that one of its instrumental goals will be preventing the emergence of competing AGIs with different goals. AGI creators who are overconfident about their alignment ability could perceive this as a huge first-mover advantage, which would motivate them to deploy prematurely. This in turn forces the non-overconfident AGI creators to choose between also deploying prematurely or being too late: the Unilateralist’s Curse.

How could an AI system seize control? There is a major misconception (driven by Hollywood and the media) that this requires robots. After all, how else would AI be able to act in the physical world? Without robotic manipulators, the system can only produce words, pictures and sounds. But a moment’s reflection shows that these are exactly what is needed to take control. For the most damaging people in history have not been the strongest. Hitler, Stali and Genghis Khan achieved their absolute control over large parts of the world by using words to convince millions of others to win the requisite physical contests. So long as an AI system can entice or coerce people to do its physical bidding, it wouldn’t need robots at all.98

We can’t know exactly how a system might seize control. The most realistic scenarios may involve subtle and non-human behaviors which we can neither predict, nor truly grasp. And these behaviors may be aimed at weak points in our civilization to which we are presently blind [unaware]. But it is useful to consider an illustrative pathway we can actually understand as a lower bound for what is possible.

Both the new industrial revolution and the technical features of super-intelligent AI analytically imply that super-intelligent AI and the related liability discussion should be seen as a potential judgment-proof problem. Namely, as already shown, the classic law and economics concept of the judgment-poof problem informs that if injurers lack sufficient assets to pay for the damage they cause their incentives to reduce risk will be inadequate.72 Yet, the judgment-proof problem could also be defined much more broadly to include the problem of the dilution of incentives to lower risk that emerge when a person is completely indifferent to both the ex ante possibility of being found legally liable for harm done to others and potential accident liability (given that the value of the expected sanction equals zero). This paper employs a broad definition of the judgment-proof problem which encompasses all potential sources of the dilution of incentives to mitigate risk and not merely the narrower inability of a tortfeasor’s to pay damages. In other words, super-intelligent AI might be completely indifferent to the ex ante possibility of being found liable by the human-imposed legal system for harm caused, and hence its incentives to engage in risky activities will be inadequate. Thus, the judgment-proof characteristics of superintelligent AI could in fact wholly undermine the deterrence and insurance goals of tort law. Namely, the evolution of super-intelligent and super-human AI and its capacity to develop characteristics and even personhood73 in a manner never envisaged by its designer or producer undermines the effectiveness of the traditional strict liability and other tort law instruments.

The judgment-proof characteristic of a super-intelligent AI system also implies that AI’s activity levels will tend to be socially excessive and will contribute to excessive risk taking by the super-intelligent AI.74 It will have no liability-related incentive to mitigate risk and its incentives to reduce the risk and harm will be completely diluted. The deterrence goal might be corrupted irrespective of the liability rule since the judgment-proof super-intelligent AI (possessing autonomous ‘personhood’ and in control of its own decision-making processes) will simply not internalise the costs of any accident it might cause. 75 Hence, as the literature suggests, no kind of tortious liability will provide adequate incentives to alleviate the risk (and there will be no incentives or a technical basis for AI to purchase insurance). In other words, the insurance goal will be undermined to the extent that the judgment-proof tortfeasor (autonomous AI of any kind) proves unable to fully compensate its victims. Moreover, as shown by Logue, first-party insurance markets will also not provide an adequate response/remedy.76

As the law currently stands in the United States and, as far as I know, everywhere else,  the law treats all robots of every type as chattel. That is, in the words of Neil Richards and William Smart, “Robots are, and for many years will remain, tools. They are sophisticated tools that use complex software, to be sure, but no different in essence than a hammer, a power drill, a word processor, a web browser, or the braking system in your car.” It follows that robot personhood (or AI personhood) under law remains a remote prospect, and that some lesser form of increased legal protections for robots, beyond those normally accorded to chattels in order to protect their owners’ rights, also remain quite unlikely. Indeed, barring some game-changing breakthrough in neural networks or some other unforeseen technology, there seems little prospect that in the next decades machines of any sort will achieve the sort of self-awareness and sentience that we commonly associate with a legitimate claim to the bundle of rights and respect we organize under the rubric of personhood.

<<FOOTNOTE 1 BEGINS>>

This question of AI legal personhood is mostly theoretical today. In article we try to generalize some common ways that existing in legal theory and practice. We analyze some cases of recognition of untypical legal persons as well enacted statements in Europe and USA. Readers will not find a detailed methodology in the paper, but rather a list of criteria that is helpful to make a decision on granting legal personhood.

Practices of European Union and the United States indicate that common approaches to the legal personality of some kinds of AI are already developed. Both countries are strongly against legal personhood of intellectual war machines. Liability for any damage of misbehavior of military AI is still on competence of military officers. In case of civil application of AI there are two options. AI could be as legal person or as an agent of business relations with other legal persons. Every legal person has to be recognized as such by society. All untypical legal persons have wide recognition of society. When considering the issue of introducing a new legal person into the legal system, legislators must take into account the rights of already existing subjects. Policy makers have to analyze how such legal innovation will comply with previous legal order, first of all how it will affect the fundamental rights and freedoms of the human beings. The legal personhood of androgenic robots that can imitate human behavior regarded in paper as a good solution to minimize illegal and immoral acts committed with their involvement. It would be a factor that keep people from taking action against robots very similar to people. Authors conclude that key factors would be how society will react to a new legal person, how changing of legal rules will affect legal system and why it is necessary. At least all new untypical legal persons are recognized by society, affects of the legal system in manageable way and brings definite benefits to state and society.

If regulatory questions about robot rights arise before the courts in the context of litigation a coherentist approach is likely.129 Given that existing legal categories are likely to remain (largely) unchanged with a coherentist approach, recognition of robot rights is unlikely to emerge within coherentist legal analysis. For example, existing intellectual property laws are likely to be amended rather than new forms of intellectual property being articulated and recognised.130 In contrast, legislatures are likely to engage with regulatory questions about new technologies through a regulatory-instrumentalist lens131 which may contemplate whether existing legal categories are adequate for the risks and challenges (potentially) posed by new technologies and whether new categories or new regulatory frameworks are required to meet those challenges. In this environment, compared to a coherentist approach, a regulatory-instrumentalist approach is more likely to recognise (the potential for) robot rights provided such recognition would address regulatory goals, and thus debate over the potential for robots to be granted rights is more likely to arise within an instrumentalist-regulatory approach.

While the Spaeth and Segal study cast serious doubt on the influence of stare decisis, it nevertheless demonstrated that it exerts some force, even in those instances where it is least likely to – that is, where a justice has directly considered and previously rejected a ruling. Studies coming at precedent from different angles have contributed additional evidence that precedent matters.

Hansford and Spriggs (2006), for example, found that the “vitality” of precedent, important to federal district and circuit judges, also counts with the justices themselves: they are more likely to accord positive treatment to precedents that have previously been treated positively by the Court. Bailey and Maltzman (2011), employing highly sophisticated measures of justices’ ideologies and the ideological implications of the cases before them, tested whether justices were responsive to precedents in cases where those precedents were seriously contested. They found that the precedents exerted significant effects on the decisions of some, though not all, justices.

According to Yampolskiy and Spellchecker (2016), the probability and seriousness of AI failures will increase with time. We estimate that they will reach their peak between the appearance of the frst self-improving AI and the moment that an AI or group of AIs reach global power, and will later diminish, as late-stage AI halting seems to be a low-probability event.

AI is an extremely powerful and completely unpredictable technology, millions of times more powerful than nuclear weapons. Its existence could create multiple individual global risks, most of which we can not currently imagine. We present several dozen separate global risk scenarios connected with AI in this article, but it is likely that some of the most serious are not included. The sheer number of possible failure modes suggests that there are more to come.

Even so, any sort of legal personhood for AIs would be a dramatic legal change that could prove problematic.225 As discussed earlier, providing legal personality to AI could result in increased anthropomorphisms. People anthropomorphizing AI expect it to adhere to social norms and have higher expectations regarding AI capabilities.226 This is problematic where such expectations are inaccurate and the AI is operating in a position of trust. Especially for vulnerable users, such anthropomorphisms could result in “cognitive and psychological damages to manipulability and reduced quality of life.”227 These outcomes may be more likely if AI were held accountable by the state in ways normally reserved for human members of society. Strengthening questionable anthropomorphic tendencies regarding AI could also lead to more violent or destructive behavior directed at AI, such as vandalism or attacks.228 Further, punishing AI could also affect human well-being in less direct ways, such as by producing anxiety about one’s own status within society due to the perception that AIs are given a legal status on a par with human beings.

Finally, and perhaps most worryingly, conferring legal personality on AI may lead to rights creep, or the tendency for an increasing number of rights to arise over time.229 Even if AIs are given few or no rights initially when they are first granted legal personhood, they may gradually acquire rights as time progresses. Granting legal personhood to AI may thus be an important step down a slippery slope. In a 1933 Supreme Court opinion, for instance, Justice Brandeis warned about rights creep, and argued that granting corporations an excess of rights could allow them to dominate the State.230 Eighty years after that decision, Justice Brandeis’ concerns were prescient in light of recent Supreme Court jurisprudence such as Citizens United v. Federal Election Commission and Burwell v. Hobby Lobby Stores, which significantly expanded the rights extended to corporations.231 Such rights, for corporations and AI, can restrict valuable human activities and freedoms.

Notably, while raising the questions of how patent law and policy should respond to ‘autonomously generated’ inventions, none of the reviewed policy documents provides a technical definition of such inventions. For instance, the WIPO draft issues paper states that ‘it would now seem clear that inventions can be autonomously generated by AI’.24 While no explicit reference is provided in support,25 it is worth noting that, only recently, this scenario was considered by WIPO to be ‘a science fiction’.26 The World Economic Forum white paper assumes that ‘AI is no longer “just crunching numbers” but is generating works of a sort that have historically been protected as “creative” or as requiring human ingenuity’.27 However, no technical literature but only legal sources are referenced.28

Somewhat puzzlingly, the request for comments initiated by the USPTO uses the term ‘AI inventions’ to refer to both inventions that utilise AI and inventions developed by AI.29 It is assumed that both types can comprise elements such as ‘the application of AI, the structure of the database on which the AI will be trained and will act; the training of the algorithm on the data; the algorithm itself; the results of the AI invention through an automated process’.30 (This view, however, requires further precision.31 For instance, if machine learning is applied in the process of drug discovery and development, the AI technique involved in that process would not be part of the resulting drug claimed as an invention.) Further, the document deliberates that, in both cases, a natural person can contribute to the conception of an invention, including by ‘designing the algorithm and/or weighting adaptations, structuring the data on which the algorithm runs, running the AI algorithm on the data and obtaining the results’.32 This suggests that, in the USPTO’s view, the development of an invention ‘by AI’ can still involve human input, which raises a critical question as to where to draw the line between situations where AI ‘develops’ an invention and where it is used as a tool (i.e. as a problem-solving technique).

For now, Thaler’s impact may be limited. The Federal Circuit stated that its decision does not address patent protection for inventions made with the assistance of AI, which is likely the far more common scenario at the present time.

If a researcher uses AI as a tool, the use can be analogized to using a computer to conduct complex calculations, data analysis, or simulations in which case the researcher directing or using the AI is likely to be the appropriate inventor.

In advising the World Intellectual Property Organization, the ABA Section of Intellectual Property Law resolved (1) that a patent should always name a human as an inventor and (2) that Al may not qualify as an inventor. 161 The conception requirement under U.S. patent law provides the foundation for these conclusions. 6 2 Joint inventors may perform only a part of the effort to produce the invention, and need not make the same type or amount of contribution.1 63 Where the inventive process involves AI, the patent application may attribute conception to a human who puts in motion and provides any input that the Al uses to reduce the invention to practice,16" for example, a person recognizing an inventive intermediate or final result for patenting.1 65 The ABA Section further maintains that determination of whether the human's efforts amount to the conception or mere use of software should continue to be made by fact-finding administratively and judicially under existing law.1" Therefore, the issue of determining the inventor does not introduce a new issue nor require a new means of determination.

The Multiplayer Model seems to overstate the difficulty of determining who conceived the invention.1 67 The model emphasizes the difficulty in identifying the inventor because of the number of individuals who contribute to even a minute portion of the inventive process. However, inventorship need not depend on a single person nor include those that reduce the invention to practice." Inventors benefit from the use of computers, the Internet, and research assistants in the reduction to practice, but it is in the conception of the invention that matters for inventorship. This model effectively illustrates the problem linked with inventive AI but bars the patentability of AI-assisted inventions without giving the inventor a chance of identifying the contributing human inventors.

In view of the foregoing, it appears clear that ML systems do not replace human problem-solving skills, and the assumption that an ML system can solve a given problem by itself is unfounded.

As a general principle, a normative analysis should be undertaken if there is genuine uncertainty about the appropriateness of the existing legal order. Hence, the currently applicable definitional criteria for inventorship should serve as a starting point. In this regard, the study by Shemtov50 on the concept of inventorship in the context of AI concludes that ‘[c]reative or intelligent conception of the invention, or contribution thereto, is a feature that runs either explicitly or implicitly throughout the definition of inventorship in all of the [examined] jurisdictions’.51 To meet this requirement, the ‘engagement in the conception phase [should go] beyond the provision of abstract ideas on the one hand, and mere execution of those provided by others on the other hand’,52 while such engagement should be made ‘on an intelligent and creative level rather than financial, material or mere administrative level’.53 In the subsequent analysis, this paper will rely on this standard as a general guiding principle, while it is recognised that jurisdictions might differ in fleshing out the concept of the inventor in further detail.54

The article has shown that AI techniques represent computational methods of problem solving enabling partial automation of inventive activity. As such, the application of such techniques cannot be prejudicial for the allocation of the inventor entitlement to a natural person. As long as a human specifies instructions that determine how the input-output relation is derived through computation, and as long as computers are bound by such instructions, there is seemingly no reason why AI-aided ‒ allegedly ‘AI-generated’ ‒ inventions should be treated under patent law differently than inventions assisted by other types of problem-solving tools and methods as far as inventorship is concerned. Instead, the use of such techniques should be a matter of the assessment of inventive step.

As we do not personify the laws of physics or chemistry, neither should we attribute a mystic personality to computational processes carried out according to the laws of mathematics and statistics. Even though it became common to use the language that anthropomorphises algorithms, such tendency was viewed as ‘an obstacle to properly conceptualizing’ legal and societal challenges posed by AI techniques, as well as misguiding the policy priorities. If computers only execute the problem-solving mechanism ‒ defined in this paper as instructions as to how the input-output relation should be derived through computation ‒ the distinction between human and ‘non-human’ (algorithmic) ingenuity is simply pointless.

The European Patent Office has refused two European patent applications that designated an artificial intelligence called DABUS as the inventor, following a non-public hearing on November 25, 2019.

The applications are for a “food container” (number EP3564144) and “devices and methods for attracting enhanced attention” (number EP3563896). They were filed by the Artificial Inventor Project, which has so far filed patent applications for the inventions via the Patent Cooperation Treaty (PCT) in the United States, United Kingdom, Germany, Israel, China, Korea and Taiwan.

The state of patent law in America is this: You might as well appeal because if you get lucky and draw the right panel you will win. And like it or not, that is precisely what our patent justice system has become under the Federal Circuit. A crapshoot. And we all know it to be true.

The current state of utter disarray at the Federal Circuit, with panels doing whatever they want, judges not agreeing on anything, and ignoring en banc decisions as if they never happened isn’t what the Federal Circuit is meant to have become. The Federal Circuit is a disaster and the collective unwillingness of the judges to come together is making a mockery of an institution that is a critical piece in the U.S. innovation system. Indeed, the fact that the Federal Circuit is absent and unwilling to provide predictability and certainty, which literally was their only job, is why so many people are turning to Congress to solve the problems of the patent system.

1. Good: The Sky Isn't Falling.-First, the good news: we are unlikely to break the patent system, whether by passing patent reform legislation or by failing to pass it. Much of the academic and policy debate over patent law in the past twenty years has focused on the relative dangers of overprotection and underprotection. Both sides have worried that changes to the patent system will kill the goose that laid the golden egg, retarding, rather than promoting, innovation. Those who worry about overprotection fear that patent trolls will impose a tax on true innovators and that too many strong patent rights will make cumulative innovation and bringing products to market harder.23 8 Those who (more recently) worry about underprotection tout the U.S. patent system as the primary reason for our national lead in innovation, and fear that weakening that system will discourage invention and prevent good ideas from getting to market.239

The evidence, however, suggests that both of these concerns are overblown. Radical changes in both patent substance and procedure that strengthened the hand of patent owners during the 1980s and 1990s and brought us a deluge of patent trolls didn't break the patent system, worries about the patent crisis notwithstanding. Indeed, they didn't seem to have a significant causal effect on patent applications, patent grants, patent lawsuits, or patent judgments.

But unlike some utilitarian approaches, virtue-based ethics does not pretend to offer mathematical answers to complex value questions, nor does it issue prescriptions that hang in the middle of an ethical vacuum. Consider, for example, the consequentialist work on AI authorship and copyright discussed in Section II.B.3. It is unlikely that any empirical study could demonstrate, with any degree of confidence, that affording copyright authorship to narrow AI as a de jure legal entity would result in a net increase in valuable copyrightable works. The variables are impossible to specify and are notoriously slippery. What would an ex ante "incentive" mean to a narrow AI? Would the system be programmed to want to make money from copyright royalties? Would the AI have other competing incentives like a human author? Many copyright scholars believe the incentive of copyright royalties only motivates creative work in certain fields where big money is to be had - such as pop music and commercial movies - to the detriment of other artists. Would programming an AI with a bias toward maximizing copyright royalties produce negative spillover effects, such as AI copyright hogs and AI copyright trolls? Would a proliferation of AI-authored copyrighted works, and the enforcement of those rights, chill fair use and free expression?

Empirical legal scholars can try to study and model such questions, but the notion that any studies could provide global answers is overly idealistic and Quixotic at best. Perhaps one day an AI system will possess the horsepower to create more reliable models and predictions about complex normative value questions. Perhaps that system will also be capable of tweaking legal rules applicable to other AI systems on the fly, so the entire mechanism is continually optimized. Presently, that kind of societal utilitarian computer is only science fiction. In my view, it is dystopian science fiction in which human agency is destroyed.

AI Patents will only result in stifling progress, as the culture is still developing. This will result in other countries getting ahead of the US competitively, and companies in the US needing to spend unnessary time and money to make sure they are safe from patent trolls.