Rights ensure snowball—the link is unique

Both the new industrial revolution and the technical features of super-intelligent AI analytically imply that super-intelligent AI and the related liability discussion should be seen as a potential judgment-proof problem. Namely, as already shown, the classic law and economics concept of the judgment-poof problem informs that if injurers lack sufficient assets to pay for the damage they cause their incentives to reduce risk will be inadequate.72 Yet, the judgment-proof problem could also be defined much more broadly to include the problem of the dilution of incentives to lower risk that emerge when a person is completely indifferent to both the ex ante possibility of being found legally liable for harm done to others and potential accident liability (given that the value of the expected sanction equals zero). This paper employs a broad definition of the judgment-proof problem which encompasses all potential sources of the dilution of incentives to mitigate risk and not merely the narrower inability of a tortfeasor’s to pay damages. In other words, super-intelligent AI might be completely indifferent to the ex ante possibility of being found liable by the human-imposed legal system for harm caused, and hence its incentives to engage in risky activities will be inadequate. Thus, the judgment-proof characteristics of superintelligent AI could in fact wholly undermine the deterrence and insurance goals of tort law. Namely, the evolution of super-intelligent and super-human AI and its capacity to develop characteristics and even personhood73 in a manner never envisaged by its designer or producer undermines the effectiveness of the traditional strict liability and other tort law instruments.

The judgment-proof characteristic of a super-intelligent AI system also implies that AI’s activity levels will tend to be socially excessive and will contribute to excessive risk taking by the super-intelligent AI.74 It will have no liability-related incentive to mitigate risk and its incentives to reduce the risk and harm will be completely diluted. The deterrence goal might be corrupted irrespective of the liability rule since the judgment-proof super-intelligent AI (possessing autonomous ‘personhood’ and in control of its own decision-making processes) will simply not internalise the costs of any accident it might cause. 75 Hence, as the literature suggests, no kind of tortious liability will provide adequate incentives to alleviate the risk (and there will be no incentives or a technical basis for AI to purchase insurance). In other words, the insurance goal will be undermined to the extent that the judgment-proof tortfeasor (autonomous AI of any kind) proves unable to fully compensate its victims. Moreover, as shown by Logue, first-party insurance markets will also not provide an adequate response/remedy.76

4.3 Policy suggestions regarding judgment-proof super-intelligent AI

Law and economics literatures offer several potential types of policy responses to mitigate the identified judgment-proof characteristics of autonomous AI and its continuous, unpredictable change once it leaves the production line and the potential resulting hazards.

The first instrument is vicarious liability.77 Shavell, for example, suggests that if another party (principal) has some control over the behaviour of the party whose assets are limited (agent), the principal can be held vicariously liable for the losses caused by the agent.78 Hence, vicarious liability (indirect reduction of risk) and a specific principal-agent relationship between the owner (a human who uses AI) and their autonomous AI agent should be introduced. The principal (owner) should be held vicariously liable for the losses the agent (autonomous AI) causes. If the principal can observe the agent’s level of care the imposition of vicarious liability will induce the principal to compel the agent to exercise optimal care. In other words, an extension of liability should indirectly lead to a reduction of risk.

Yet, if the principal is unable to observe and control the level of care exercised by the agent (AI) the lead he will generally be unable to compel the agent.79 Still, if the principal can control the AI’s level of activity (but has no observation capacity) then such vicarious liability will induce the principal to reduce the AI’s participation in risky activity.

However, what if AI is truly autonomous, capable of self-learning, developing emergent properties and adapting its behaviour/actions to the environment while even being self-aware? In these circumstances, the imposition of vicarious liability might prove completely inadequate due to the extreme judgment-poof problem. Here, law and economics suggests the introduction of a whole arsenal of economic/legal institutions that might address such an extreme superintelligent AI-judgment-proof problem.

Law and economics literature proposes the following instruments: a) requiring the principal to have a certain asset minimum to engage in an activity; b) the compulsory purchase of liability insurance coverage (for the principal); c) direct ex ante regulation of AI’s risk-creating behaviour – a regulatory agency setting standards for behaviour/employment/operating/functioning; 80 d) safety standards (which alone are inadequate81) and registration (of both the principal and the agent); e) the principal’s criminal liability;82 f) extending liability from the actual injurer (AI) to the company – by piercing the veil of incorporation; g) corrective taxes equal to the expected harm; and h) establishing a wide insurance fund that is publicly and privately financed.

In addition, one should consider introducing the AI manufacturer’s strict liability supplemented by the requirement that an unexcused violation of a statutory safety standard is negligence per se. Moreover, compliance with the regulation standard should not relieve the injurer’s principal from tort liability. Thus, the rule per se (violation of a regulatory standard implies tort liability – including strict liability) should also be applied to AI-related torts and the compliance defence of an AI manufacturer or its principal should not be accepted as an excuse.

This paper also suggests that regulation and tort law should be applied at the same time. Ex post liability and ex ante regulation (safety standards) are generally viewed as substitutes for correcting externalities where the usual recommendation is to employ that policy which produces lower administrative costs. However, Schmitz shows that the joint use of liability and regulation can enhance social wealth.83 Namely, regulation removes problems that affect liability while liability limits the cost of regulation.84 General regulatory standards should settle at a lower level of care (lower than optimal) and be combined with tort law instruments. 85 That is, by introducing an ex ante regulatory standard the principal and the agent (AI) might be prevented from taking low levels of precaution and might find it convenient with the regulatory standard despite the judgment-proof problem.

Moreover, the informed lawmaker should combine strict liability and vicarious liability – the strict liability of the manufacturer and vicarious liability of the principal (principal as either a legal or physical person). Yet, since the product liability regime assumes that a product does not continue to change in any unpredictable way once it leaves the production line, such a combination might not be sufficient. Furthermore, the producers (companies) and designers of super-intelligent AI devices might be judgment proof due to their size and lack of assets.86 The classic debate on the two different approaches to controlling hazardous activities, namely ex post liability for harm done and ex ante safety regulation may, given the mentioned shortcomings, boil down to a question of efficient regulatory timing and ex ante regulation. The identified judgment-proofness of super-intelligent AI is seeing society effectively shift back to regulation. Thus, a specific EU regulatory intervention is suggested that encompasses the following: a) the requirement that a principal has certain minimum assets to engage in an activity; b) the compulsory purchase of liability insurance coverage (for the principal); c) direct ex ante regulation of AI’s risk-creating behaviour – an EU regulatory agency setting standards for behaviour/employment/operating/functioning; d) safety standards; e) compulsory registration in super-intelligent-AI- related registries; f) the principal’s criminal liability; g) extending liability from the actual injurer (AI) to the company that either uses or produces the AI – piercing the veil of incorporation; h) corrective taxes that equal the expected harm; and i) the establishment of an EU-wide publicly/privately-financed insurance fund.

Such ex ante regulatory activity is also in line with the theory of optimal legislative timing. In a ground-breaking paper, Gersen and Posner investigated the optimal timing of legislative action and noted that decisions on the timing of legal intervention are often as important as decisions on the content of a new law.87 They argue that lawmakers cannot know with certainty what the appropriate law will be in the future and therefore cannot know with the certainty what the future stream of benefits from the law will be.88 In addition, the costs of introducing the new law are largely sunk and irreversible (e.g. the outlay of resources to formulate and enforce the law) where the costs of implementing the new law cannot be recovered if the law turns out to be inappropriate. Timing of the investment becomes a critical issue for such irreversible investments.89 In economic terms, the lawmaker’s decision to invest in the new law represents ‘opportunity costs’ of abandoning the option to implement the law in the future.

However, there is also an ‘opportunity benefit’ of investing today. Parisi and Ghei offer a formal model that determines three attributes that law-making shares with investment: a) the costs of law-making cannot typically be recovered if the rule proves to be ineffective or undesirable at a later point in time; b) the future benefits of the legislation are uncertain; and c) lawmakers have the option to postpone changing the current legal rules.90 The literature also offers the concept of the optimal time to legislate whereby a benevolent and rational lawmaker should enact a new rule (or modify an existing one) when the present value of the expected benefits from the legal innovation is at least equivalent to its costs.91 Obviously, the optimal timing of law-making is affected by the presence of uncertainty since, once uncertainty arises, the desirability of the law changes accordingly. Gersen and Posner investigate several potential timing rules and emphasise the significance of the ‘anticipatory legislation’ rule.92 Thus, the legislation will only become effective if its effects are positive and such a timing rule offers the option of exit through repeal.93

Moreover, such anticipatory legislation entails lower adjustment costs since stakeholders can more confidently rely on the public good being created – i.e. such legislation increases the likelihood that the public good will be created.94 The optimal timing of the legal intervention is thus crucial because delays in law-making decisions may come at a cost. The exponential rise of super-intelligent AI technology over time is adding to the costs of law-making and in the future such costs may, taking the precautionary principle into account, become even higher.

Thus, specific Europe-wide regulatory activity should employ the ‘anticipatory legislation’ approach due to the specific features of super-intelligent AI in line with the theory on the optimal timing of legal intervention. Hence, as stressed, a specific EU-wide regulatory anticipatory regulatory intervention is suggested that encompasses the direct ex ante regulation of AI’s risk-creating behaviour (an EU regulatory agency setting standards for behaviour/employment/operating/functioning), requires principals to hold certain minimum assets if they are to engage in an activity, the compulsory purchase of liability insurance coverage, safety standards and Turing Registries, the principal’s criminal liability, extended liability from the actual injurer (AI) to the company by piercing the veil of incorporation, corrective taxes that equal the expected harm, and the establishment of an EU-wide insurance fund with public/-private backing.

4.4 A new special electronic legal person should not be created

Regarding the specific legal status, in paragraph 59 of its Resolution on Civil Law Rules in Robotics95 the EU Parliament suggests that the EU create a specific legal status for robots so that at least the most sophisticated autonomous robots can be established as having the status of electronic persons responsible for making good any damage they may cause, and possibly applying an electronic personality to cases where robots (AI) make autonomous decisions or otherwise interact with third parties independently. Moreover, Solum, 96 Wright, 97 Teubner98 and Koops et al. also argue that AI should be given a legal personality and that there is no compelling reason to restrict the attribution of action exclusively to humans and social systems.99 Furthermore, Allen and Widdison state that when an item of AI is capable of developing its own strategy it makes sense that the AI should be held responsible for its independent actions.100 Yet, it must be emphasised that Teubner, for example, suggests that software agents should be given a carefully calibrated legal status.101 The answer to the risk brought by autonomy would, according to Teubner, be their status as actants, as actors with partial legal personhood whose autonomous decisions are made legally binding and in case they trigger liability for damages. 102 Are such suggestions supported by the insights from law and economics?

Obviously, from a law and economics perspective the establishment of a special status of electronic person for super-intelligent AI that would have its own legal personality and responsibility for potential damages should be avoided. 103 Namely, under such proposal the super-intelligent AI itself would be legally responsible for damage, rather than its principal, owner or manufacturer. This would then also imply that super-intelligent AI agents own financial assets and are subject to sanctions if they do not comply. As this paper argues, at best this does not make any sense and, at worst, it might bring dire consequences.

Namely, the identified judgment-proofness of super-intelligent AI suggests that the establishment of such a legal personality would, while amplifying the judgment-proof problem, have unexpected adverse effects. Why would super-intelligent AI with an electronic legal personality care at all if it were imprisoned or sanctioned for non-payment or causing harm? The judgment-proof problem would actually dilute the incentives of the super-intelligent AI (even though – or even because – it has a legal electronic personality) to reduce risk that arises due to the AI’s complete indifference to both the ex ante possibility of being found legally liable for harm done to others and potential accident liability (where the value of the expected sanction equals zero). Moreover, establishing a specific electronic legal personality for superintelligent AI would imply that policymakers then rely on the existing tort law system to control activities that create risks of harm and induce the optimal amount of precaution. Yet, as shown, the judgment-proofness of super-intelligent AI would undermine the essential deterrence and insurance goals of tort law, leaving potential risks unaddressed. In other words, the establishment of such a specific electronic person might institutionalise the judgment-proofness of super-intelligent AI.

Accordingly, the establishment of an unregulated human-like electronic personality is not an effective or adequate response to the identified super-intelligent-AI- judgment-proofness, but might be seen as an amplifier making the problem even more persistent. Namely, granting a special electronic legal personality to super-intelligent AI would make liability-related legal regimes completely ineffective. Consequently, granting legal personality to super-intelligent AI might open Pandora’s box of moral hazard and create perverse incentives on the side of human principals, the AI industry, designers, users, owners and will exacerbate the judgmentproof problem of a super-intelligent AI device.

6. Conclusions

This paper sought to address the role of public regulatory policy in regulating superintelligent AI, the related risk and civil liability for damage caused by such super-human AI. The main findings are: a) the current law-making anticipatory activities and initiatives are in line with the theory of the optimal timing of legislative action; b) generally speaking, the existing laws and legal mechanisms adequately address the responsibility for non-autonomous AI; b) if super-intelligent AI evolves unpredictably once it leaves the creator’s lab then ex ante regulatory intervention is suggested; c) the classic debate on the two different approaches to controlling hazardous activities, namely ex post liability for harm done and ex ante safety regulation, boils down to a question of efficient ex ante regulation; d) super-intelligent AI and associated companies might be judgment proof; e) a specific regulatory intervention is suggested to tackle the potential judgment-proof characteristic of super-intelligent AI which encompasses the requirement for the principal to have certain minimum assets to engage in an activity, the compulsory purchase of liability insurance coverage (for the principal), anticipatory regulation of AI’s risk-creating behaviour – a regulatory agency setting standards for behaviour/employment/operating/functioning, safety standards and registries, the principal’s criminal liability, extending liability from the actual injurer (super-intelligent AI) to the company by piercing the veil of incorporation, corrective taxes that equal the expected harm, and the setting up of a wide insurance fund that relies on public/private financing. Moreover, the contemplated new special electronic legal personality for super-intelligent AI should not be introduced. As this paper attempts to show, the judgment-proofness of superintelligent AI implies that any establishment of a legal personality would, while exacerbating the judgment-proof problem, bring unexpected adverse effects. It would dilute the incentives of super-intelligent AI to reduce the risk that materialises due its complete indifference to both the ex ante possibility of being found liable by the legal system for harm done to others and potential accident liability. The paper also shows that, due to identified shortcomings, the debate on the different ways of controlling hazardous activities may be reduced to a question of efficient ex ante safety regulation. In other words, regulatory intervention is from the law and economics perspective, due to the identified shortcomings of the current tort law institutions the best option to govern the super-intelligent AI systems.