Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Testing and Interoperability are key concerns for stakeholders

Meredith Broadbent 21, Meredith serves as a senior adviser with the Scholl Chair in International Business at the Center for Strategic and International Studies and is a former chair of the U.S. International Trade Commission. “What’s Ahead for a Cooperative Regulatory Agenda on Artificial Intelligence?” March 17. https://www.csis.org/analysis/whats-ahead-cooperative-regulatory-agenda-artificial-intelligence //barn

Responding to Commission plans to enact stricter rules for high-risk AI technologies, such as compliance tests and controls, 14 EU member states published a paper outlining their position urging the Commission to adopt a “soft law approach” consisting of “self-regulation, voluntary labeling and other voluntary practices, as well as a robust standardization process as a supplement to existing legislation that ensures that essential safety and security standards are met.”1 The countries recommend a European approach to AI regulation in which innovation and trustworthiness work together in a “coherent and borderless single AI market.”

For several reasons, AI giants such as Google and Facebook, as well as startups, tech companies, and associations, have raised concerns that meeting regulatory requirements considered in the AI white paper will stifle innovation and may require actions that breach EU data privacy rules.

First, technology corporations are waiting anxiously to see how the Commission will define “high-risk” AI, as that will have a foundational impact on how AI regulation is implemented. A chief concern is that the Commission’s definition of “high-risk” AI and its approach to the concept is too broad and would result in one-size-fits-all regulation for all AI applications in “high-risk” sectors, regardless of differences between the sectors and the type of AI application. Instead, Google, Facebook, and Digital Future for Europe, which represents associations, tech companies, and startups from the Digital Nine,2 propose a sector, technology, or application-specific regulatory approach that takes into account unique risks posed by individual sectors as well as the application itself. For example, risks posed by AI applications in the healthcare sector are much different than those in the transportation sector. One-size-fits-all regulation for AI applications within sectors would also be suboptimal. An AI chatbot developed for a healthcare company to better understand patient needs presents a different level and type of risk than an AI application involved in medical decision-making.

Although expressing agreement with several aspects of the white paper, IBM, one of the largest technology employers in the European Union, argues that existing sector-specific governance structures would be better suited to implement regulation to avoid unnecessarily impeding AI development. For example, the Commission’s propositions on human oversight requirements may be inappropriately restrictive in the context of automated driving, since “it is impossible to oversee every single decision taken by an automated car, due to most decisions being taken in real time.”

Stakeholders have also raised concerns over vague exceptions included in the white paper that could make it difficult to determine whether an AI application is subject to “high-risk” rules. The inclusion of AI applications that could cause “immaterial damage” as a factor in determining risk was characterized as nebulous by many stakeholders. The white paper also suggests that there may be “exceptional circumstances” where an AI application could be considered “high-risk” regardless of its sector. Booking.com, an online travel agency headquartered in the Netherlands, has questioned the white paper’s unspecific notion that “applications affecting consumer rights” could also be required to apply “high-risk” rules. These exceptions and ambiguities inject an uncertainty into which AI applications will be considered “high-risk” and increases expectations that the Commission will adopt an overly broad view of “high-risk” AI applications, subjecting applications to onerous regulations that are not justified. In comments to the white paper, the European Digital SME Alliance, a network of associations representing more than 20,000 European small and medium ICT enterprises, added, “[s]mall companies are the first to suffer from rules that are up to interpretation.”

Digital Future for Europe maintains that the “risk assessment process would stifle many AI innovations before they have developed,” and through ex-ante regulation “the EU threatens to skew the whole development of European AI.” Additionally, IBM has cast doubt upon the white paper’s proposal of an explicit and exhaustive list of sectors in which “high-risk” applications could emerge, due to the rapid evolution of AI technology and its diverse applications.

Second, companies have raised concerns regarding the requirement that AI developers would need to retrain AI systems in the European Union to meet data set requirements. Facebook and Google note that restricting data available to train AI to just EU data is likely to reduce the quality of the AI product and contribute to biases in the systems. Digital Future for Europe suggests the Commission focus on increasing the pool of data acceptable to regulators and available to AI developers in order to improve AI applications and encourage innovation. IBM opposes input-specific requirements, arguing instead that “high-risk” applications should be required to ensure a specific outcome, such as the absence of discrimination. The Commission’s approach to training data either wrongly assumes AI applications employ a static data set or raises the prospect that AI systems would need to be re-audited on a regular basis. That would drastically slow the ability to update and improve AI tools. This is particularly concerning if an AI application is producing undesirable or suboptimal outcomes based on a previously audited data set. Facebook has pointed out that it is unclear what data needs to be “EU data” and how the service being provided by an AI application changes that consideration. Google, as well as the European Association of Automotive Suppliers (CLEPA), worry that it is difficult to separate out the provenance of some parts of training datasets in certain fields, which often rely on third-party and open-source data. Leaving out these data sets in training could seriously hurt the quality of AI systems subsequently released in the European Union.

Third, Google and Facebook have raised concerns that compliance with rules for “high-risk” could run counter to EU privacy rules, including GDPR. Facebook worries that the proposed requirement to store training data sets could “create a direct tension” with policies to protect users’ data, such as those related to data minimization and data retention. The social media platform also worries that the proposed requirement could prohibit federated learning and other AI approaches intended to protect privacy.3 In Facebook’s view, this requirement could undermine the potential effectiveness of federated learning in protecting users’ privacy.

Requirements around algorithmic fairness and the need to share source code also appear to be in tension with EU privacy rules. Google believes that the requirement to ensure that data sets are “sufficiently representative” contrasts with GDPR obligations: “developers should not be able to access attributes such as ethnicity and therefore could not test for ethic [sic] representation in a dataset.” Google stresses that “sufficiently representative” should be more clearly defined.

As an alternative to ex-ante conformity assessments that can be burdensome, unpredictable, and could expose companies to breaches in privacy rules, Facebook and Google propose benchmark tests tailored to specific high-risk applications to determine whether or not AI will behave in a way that is expected. This approach would hold industry players accountable for mitigating bias in AI, allow internal testing of AI systems, and support a self-certification process. Digital Future for Europe suggests an overall regulatory approach that encourages responsible adoption of AI before introducing onerous regulation. To do so, European governments should open public data sets, provide for data interoperability between government data sets, refit existing European legislation to account for AI, and make sure that any new AI rules are agile and flexible. Google proposes a “large central dataset” that developers could access to protect against developers attempting to overfit models to accord with EU rules

Sign in to recut, update, or save this card.