Following onto the EU AI Act
Townsend 21 [Bev Townsend, Postdoctoral Researcher at University of York, “Decoding the Proposed European Union Artificial Intelligence Act”, Insights, https://eprints.whiterose.ac.uk/178738/1/Townsend_Decoding_the_Proposed_EU_AI_Act_ASIL_Insights.pdf, JMH]
The Proposed Regulation Laying Down Harmonised Rules on Artificial Intelligence On April 21, 2021, the European Commission published the highly anticipated proposal for a Regulation Laying Down Harmonised Rules on Artificial Intelligence (AI Act or the Act).11 This followed the Commission’s ongoing commitment to establish “legislation for a coordinated European approach on the human and ethical implications of AI.”12 At the outset it is worth noting that, rather than focusing on the properties or outcomes of AI, the Act defines an “AI system” as software that is developed using certain techniques and approaches (detailed in an annex) that can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with.13 Any AI system developed not using one of the enumerated techniques or approaches falls outside of the definition—a definition that, because of its imprecision, creates the opportunity to skilfully circumvent the Act. The intention is for the new legal framework to provide a set of harmonised rules that align (and are consistent with) existing EU human rights instruments and EU law regulating data protection, data governance, consumer protection, non-discrimination, and gender equality. Accordingly, a cautious yet balanced and proportionate regulatory approach is proposed, one that is primarily risk-based. It sets out the minimum necessary requirements to address risks to values, fundamental rights, and principles associated with AI development and deployment without unnecessarily constraining technological development or trade.14 What the Act does—and does well—is to differentiate requirements by risk level and to prohibit more objectionable AI systems that carry what is considered “undesirable risk” of fundamental rights infringement. The Act also introduces new legal obligations (such as monitoring, reporting, and transparency obligations) to manage those systems that, although not prohibited, are considered high risk. Amongst others, the Act establishes a robust governance, monitoring, and enforcement regime, sets up a European Artificial Intelligence Board, and seeks to impose sanctions and penalties for non-compliance. Adopting a Risk-Based Approach: The Level of Risk Determines the Applicable Rules. The central tenet of the AI Act is to introduce a set of binding rules based upon the intensity and scope of the risk generated by the AI system. The impetus for such a riskdriven approach is that persons at risk and vulnerable to health, safety, and rights infringement by new AI technologies require a higher level of protection. This is premised on the understanding that AI systems have certain characteristics (inter alia, an opacity, complexity, dependency on data, and capacity for autonomous behaviour) that can adversely and significantly affect fundamental human rights—rights to data privacy, transparency, autonomy, and the like.15 The Act addresses three categories of risk: 1. Prohibited Systems. Prohibited systems include AI systems that manipulate human behavior and/or exploit persons’ vulnerabilities; social scoring systems; and, save for certain exceptions, “real-time” and “remote” biometric identification (or facial recognition) systems. 2. High-Risk Systems. While not clearly defined, a “high-risk” system is understood to be one that poses significant risk to health, safety, and fundamental rights. Although the AI Act applies generally to all AI systems, certain provisions contained within the Act (and provided for in Title III) apply specifically to those considered high-risk. High-risk systems are either those products (or safety components of products) already covered by EU health and safety harmonisation legislation (such as medical devices, toys, and machinery, for instance) or those AI systems used in specified areas and contained in an annex to the Act (such as educational training, employment, and law enforcement).16 Rather than relying on certain criteria to position a system as high-risk, the Act designates as high-risk all AI systems used within a specified enlisted and pre-determined domain. There are also no gradations of high-risk systems—either a system is high-risk, in which case compliance with a comprehensive list of requirements and obligations is prescribed, or it is not. Once it has been established that an AI system is high-risk, it is mandated that the system’s provider (the person, agency, or body that develops an AI system or that has an AI system developed with a view to placing it on the market or putting it into service under its own name or trademark17) must fulfill certain requirements and obligations. These requirements include that quality and risk management systems be implemented; that training, validation, and testing datasets be subject to appropriate data governance and management practices and meet data quality criteria; that technical documentation be drawn up and proper records be kept; and that transparency obligations be fulfilled.18 These obligations are set out in detail within the Act. An important further requirement is that high-risk AI systems be designed and developed to allow for human oversight so that natural persons can oversee their functioning. Providers are required to introduce “human-machine interface tools” and measures to guarantee that a system is subject to built-in operational constraints that cannot be overridden by the system itself and is responsive only to a human operator. 3. Low- or Minimal-Risk Systems. All other AI systems that are without risk, or are of low or minimal risk, can be developed, sold, and deployed without additional legal obligations subject, of course, to compliance with any existing relevant legislation (including data protection legislation, such as the GDPR). Moreover, those that design and deploy low- or minimal-risk systems are encouraged to adhere to voluntary codes of conduct. Disclosing Bots, Detecting Emotions, and Deep Fakes Although there are no special requirements for low-risk systems, transparency obligations apply to all risk-levels, and three categories of disclosable activities are distinguished. First, providers of AI systems must design systems so that natural persons are informed that they are interacting with an AI system (so-called robot or “bot” disclosures). This disclosure is to avoid any potential confusion by a natural person when interacting with an AI system. It is not necessary to make such a disclosure in instances where it is contextually obvious that persons are interacting with an AI system. Second, users of systems that detect emotions or determine association with (social) categories based on biometric data must be informed of the operation of the system except in instances permitted by law or in crime prevention. And lastly, although certain exemptions apply, it is required that creators of artificial images, video, or audio content disclose that they have been generated synthetically or manipulated, such as in the case of “deep fakes.”19 Extraterritorial Reach Crucially, if adopted in its proposed form, the AI Act, much like the GDPR, will be farreaching and have significant consequences beyond the EU. The Act has extraterritorial effect and, subject to certain specific exceptions, applies to: (i) providers that place on the market or deploy AI systems in the EU, regardless of where such providers are located; (ii) users of AI systems located within the EU; and (iii) providers and users of AI systems that are located outside the EU to the extent that the output produced by the system is used within the EU. Conclusion: A Step in the Right Direction If the AI Act is strikingly similar to the GDPR, it is perhaps no coincidence. Is the AI Act likely to become the new global “gold standard” for the adoption of AI regulation? Will the EU be the sole and dominant crafters of the laws governing technology? Certainly, many low- and medium-resource countries look to regulations such as those promulgated by the EU to guide and inform their new regulatory policy development. To this end, and despite various shortcomings, the Act is a valuable start in helping to shape global norms and standards and promote trustworthy AI—AI systems that are, at least to some degree, more consistent with human values and interests. The Act also promotes innovation, including regulatory sandboxes and specific measures to support small-scale users and providers. Some may say that in preserving rights it does not go far enough, while innovators and developers may argue that it goes too far. But promoting ethical innovation and fair competition while balancing rights is not easily done— compromise is often needed—and the right regulatory framework should (prudently) do both. But by covering too much ground—fundamental rights, health and safety, data protection, and consumer law—there is a risk that the practical operationalisation and implementation of the law may be untenable. Further, a plausible framework should also be nuanced and flexible enough to adapt to, and keep abreast of, the rapidly evolving landscape that is AI development and deployment. Whether the AI Act does this sufficiently in its present form remains to be seen.