Opening the “Black Box” of AI is key to creating proper regulations & risk assessment
Buiten 19 (Miriam, CERRE Research Fellow and Assistant Professor of Law and Economics at the University of St.Gallen, Switzerland, “Towards Intelligent Regulation of AI, Published 4/29/2019, Accessed 7/15/2022, https://www.cambridge.org/core/journals/european-journal-of-risk-regulation/article/towards-intelligent-regulation-of-artificial-intelligence/AF1AD1940B70DB88D2B24202EE933F1B, EA)
The concern of AI is that it presents new and unknown risks with which current laws may not be able to cope. We may thus need new rules to mitigate the risks of AI. One proposal is to introduce transparency requirements for AI.29 Such a transparency requirement would be difficult to implement in practice, if we are not sure what its scope is. The lack of a clearly defined scope would leave the industry uncertain whether its activities are covered by the regulation, and would leave the definition of AI to the courts. The definitions of AI discussed above are, however, context-sensitive and time-varying in character. Take the approach of defining AI in terms of a system’s autonomy. This raises the question of what it means for an application to act autonomously. Scherer uses the example of autonomous cars and automated financial advisers that can perform complex 24 Oxford Dictionaries, “Artificial Intelligence” accessed 22 October 2018. 25 P Stone et al, “Artificial Intelligence and Life in 2030: One Hundred Year Study on Artificial Intelligence” (2016) accessed 20 September 2018. 26 C Reed, “How should we regulate artificial intelligence?” (2018) Phil Trans R Soc A 1. 27 R Calo, “Robotics and the Lessons of Cyberlaw” (2015) 103 Cal L Rev 513, 529. 28 JM Balkin, “The Path of Robotics Law” (2015) 6 Cal L Rev 45, 51. 29 See eg European Parliament, supra, note 11, and White House Report, supra, note 8. 2019 Towards Intelligent Regulation of Artificial Intelligence 45 https://doi.org/10.1017/err.2019.8 Published online by Cambridge University Press tasks “without active human control or even supervision”. 30 Let us consider the example of cars. Most would agree that a regular car is not autonomous, but some would say, like Scherer, that a self-driving car is. What about cars that have some autonomous-driving features, such as automatic distance keeping? Do they have sufficient autonomy to be considered AI, and should this distinction have any legal implications? If we were to require more transparency from AI, what would be the implications for producers of regular cars, partly autonomous and fully autonomous cars? Who would be subject to the requirement, and what would the requirement entail concretely? The problem, of course, would not be limited to cars, but to all systems that may be called AI. Considerable uncertainty would ensue on which systems and applications fall within the scope of such a rule. Using intelligence to define AI presents similar problems. Capabilities we consider intelligent today may not be seen as such in the future. The AI literature from the 1980s is illustrative, referring to a system’s ability to learn from experience as intelligent.31 Systems capable of playing chess, for instance, were seen as AI systems.32 Today, chess is considered as a relatively easy problem for a computer, given that the rules are known and the number of moves is finite. In 1996, IBM chess computer Deep Blue defeated the then world champion Gary Kasparov.33 In essence, while AI is sometimes defined as the capability of machines to perform tasks that require intelligence, our conception of what constitutes intelligent may change as machines acquire more capabilities. After all, how can the task require intelligence if a machine can do it?34 Even if we overcome the problem of varying definitions over time, intelligence may be viewed very differently depending on whom you ask. In 1982, John Searle essentially took the position that any technology that is understandable is not intelligent. Computers may seem to understand concepts but actually do not, as they attach no meaning or interpretation to any of it.35 On the other end of the spectrum are the views discussed above of intelligent behaviour as mimicking intelligent human behaviour, or being perceived as intelligent by humans.36 The range of applications that would be considered AI would vary greatly depending on which approach is taken. In some instances, features of AI such as autonomy or intelligence may help guide us in incorporating AI in our laws.37 When asking how laws should respond to AI, however, I propose to take a step back and demystify this concept. The narrative of AI as an inscrutable concept may reinforce the idea that AI is an uncontrollable force shaping our societies. In essence, part of the problem is that we cannot control what we do not 30 understand. I therefore propose to “open the black box” of AI and try to identify the risks and “unknowns”. This is not to say that AI does not present risks, or even that all of these risks are knowable now. In fact, the majority of the policy challenges regarding AI boil down to the need to mitigate risks that are unknown or difficult to control. It is clear that the question of how to address these risks deserves attention. A first step in doing so is to make these risks concrete, by considering the underlying computational systems. Notwithstanding AI being difficult to understand even for experts, it is a misconception that AI is completely unknowable. AI is a product of human invention, and humans continuously improve upon its underlying technology, algorithms. Algorithms are the basis of the wide array of AI applications, which may vary widely in their sophistication and purpose.38 Since the technology of algorithms is what AI applications have in common, I propose that we focus the policy debate on this technology, starting by better understanding the risks associated with algorithms. Understanding these risks may be possible without full comprehension of the technology, allowing us to grasp the risks and flaws that advanced automated systems present.