2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Solvency › Solvency - Cooperation
Scharre, 2019 - Vice President and Director of Studies at CNAS [Paul, May-June, “Killer Apps: The Real Dangers of an AI Arms Race,” https://omnilogos.com/killer-apps-real-dangers-of-ai-arms-race/6/18/22 MD]
Finally, the United States should look for ways to work with other countries, even hostile ones, to ensure AI safety. International cooperation on new technologies has a mixed record, but countries have sometimes succeeded in working together to avoid mutual harm. During the Cold War, the United States and the worked together to limit certain types of delivery systems for nuclear warheads that both sides agreed were particularly destabilizing. The United States also encouraged other countries to adopt safety measures to prevent the unauthorized use of . Today, the United States should work with both allies and adversaries to boost international funding on AI safety. It should also begin discussions with China and over whether some applications of AI pose unacceptable risks of escalation or loss of control and what countries can do jointly to improve safety. The biggest danger for the United States in an AI race is not losing but creating a world in which no one wins.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Solvency › Solvency - Cooperation
Schmidt 2021 – Chairman of the National Security Commission on Artificial Intelligence [Eric, March 12, “House Armed Services Subcommittee on Cyber, Innovative Technologies and Information Systems and House Oversight and Reform Subcommittee on National Security Hold Joint Hearing on AI and the National Security Commission”, https://congressional-proquest-com.proxy.lib.umich.edu/congressional/result/congressional/congdocumentview?accountid=14667&groupid=95663&parmId=180DE199B78&rsId=180DE197DA6]
We need to build coalitions with like-minded nations, the technology democracies, the techno democracies in my own ver--verbiage to advance the development and use of AI in emerging technologies that support our values, which is critical. We spent a lot of time on our report talking about values. And the second one, consisting with the values is responsible use. In the face of dig--digital authoritarianism, we need, we the U.S. need to present a democratic model of responsible use of AI for national security. You can imagine the opponents and how they might use or misuse these things. The trust of our nation, the trust of our citizens will hinge on justified assurance that the government's use of AI will respect privacy, cybil--civil liberties and civil rights. We have a set of recommendation along those lines. I really thank you all for giving just this opportunity. It has been a true privilege for me to be part of this, and to help lead it. Thank you very much.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Solvency › Solvency – Accidents
Artificial Intelligence/Machine Learning Risk & Security Working Group, 2020 [Wharton, ‘Artificial Intelligence Risk & Governance’, ]
Key Takeaways AIRS believes there are significant potential benefits of AI and that its adoption within financial services presents opportunities to improve both business and societal outcomes when risks are managed responsibly. This paper explores the potential risks of AI and provides a standardized practical categorization of these risks: Data Related Risks, AI/ML Attacks, Testing and Trust, and Compliance AI governance frameworks could help organizations learn, govern, monitor, and mature AI adoption. Four core components of AI governance are: definitions, inventory, policy/standards, and a governance framework, including controls. AI, in certain use cases, could lead to privacy issues, and/or potentially discriminatory or unfair outcomes, if not implemented with appropriate care. We explore, in detail, the subject of interpretability and discrimination in using AI for certain use cases. While there is no one-size-fits-all approach, practices institutions might consider adopting to mitigate AI risk include oversight and monitoring, enhancing explainability and interpretability, as well as exploring the use of evolving risk-mitigating techniques like differential privacy, and watermarking, among others.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Solvency › Solvency - TEVV
Batarsheh et al. 2021 - Professor of Electrical and Computer Engineering, Virginia Polytechnic Institute [Feras, Laura Freeman Director, Intelligent Systems Division – National Security Institute Research Associate Professor, Chih Hao Huang, Data Analyst at Turing Research. April 26 “A survey on artificial intelligence assurance” https://link-springer-com.proxy.lib.umich.edu/article/10.1186/s40537-021-00445-7#Sec8]
The need for AI assurance The emergence of complex, opaque, and invisible algorithms that learn from data motivated a variety of investigations, including: algorithm awareness, clarity, variance, and bias [94]. Algorithmic bias for instance, whether it occurs in an unintentional or intentional manner, is found to severely limit the performance of an AI model. Given AI systems provide recommendations based on data, users’ faith in that the recommended outcomes are trustworthy, fair, and not biased is another critical challenge for AI assurance.
Safety and reliability are the best balance of speed and caution
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Solvency › Solvency - TEVV
Haugh, 2018 - project leader for the Institute for Defense Analysis [Brian A., September “The Status of Test, Evaluation, Verification, and Validation (TEV&V) of Autonomous Systems,” , 6/23/22 MD]
1. Instrumenting machine thinking To diagnose the causes of incorrect behavior or inadequate performance, it will be necessary determine whether the problem lies in the Perception, Reasoning, or (course of action) Selection functions of the autonomous system—even after it has been established that the problem is not in the sensor hardware or signal processing. It will also be necessary to distinguish coding errors from inadequate algorithms or bad training data. Without the ability to instrument and monitor internal states of the autonomy, diagnosing problems will be slow at best and impossible at worst. 2. Linking system performance to autonomous behaviors In complex collaborative activities, it can be very difficult to determine what enables (or hinders) success. For example, on a soccer or basketball team it can be very difficult to pinpoint which players (and which behaviors) are leading to wins and losses. To design and improve autonomous systems, it will be necessary to understand how the system’s various autonomous capabilities interact to enable (or hinder) mission execution. The requirements specification for autonomous behavior is often a problem here due to incomplete specifications based on an analogy with human behavior. 2. Linking system performance to autonomous behaviors 3. Comparing AI models to reality Autonomous systems represent reality through stylized internal models. Perception provides inputs for these models; Reasoning allows them to be expanded and corrected. The ability of an autonomous system to do its mission will depend on the degree to which the internal modeling of reality supports accurate Perception, valid Reasoning, and effective Selection. This will not generally be a function of how detailed the models are (“high resolution”) or even of how closely the models mirror reality (“high fidelity”)—it will be a function of whether the right kind of information is incorporated into the model and that the resolution and fidelity be enough to support the mission needs. TEV&V will necessarily include prototyping and experimentation to determine 2-2 what kind of internal model, using what kind of representation, is needed to achieve both performance and dependability.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Solvency › Solvency - TEVV
Batarsheh et al. 2021 - Professor of Electrical and Computer Engineering, Virginia Polytechnic Institute [Feras, Laura Freeman Director, Intelligent Systems Division – National Security Institute Research Associate Professor, Chih Hao Huang, Data Analyst at Turing Research. April 26 “A survey on artificial intelligence assurance” https://link-springer-com.proxy.lib.umich.edu/article/10.1186/s40537-021-00445-7#Sec8]
Providing a scoring system to evaluate existing methods provides support to scholars in evaluating the field, avoiding future mistakes, and creating a system where AI scientific methods are measured and evaluated by others, a practice that is becoming increasingly rare in scientific arenas. More importantly, practitioners –in most cases– find it difficult to identify the best method for assurance relevant to their domain and subarea. We anticipate that this comprehensive review will help in that regard as well. As part of AI assurance, ethical outcomes should be evaluated, while ethical considerations might differ from one context to another, it is evident that requiring outcomes to be ethical, fair, secure, and safe necessitates the involvement of humans, and in most cases, experts from other domains. That notion qualifies AI assurance as a multidisciplinary area of investigation. In some AI subareas, there are known issues to be tackled by AI assurance, such as deep learning’s sensitivity to adversarial attacks, as well as overfitting and underfitting issues in machine learning. Based on that and on the papers reviewed in this survey, it is evident that AI assurance is a necessary pursuit, but a difficult and multi-faceted area to address. However, previous experiences, successes, and failures can point us to what would work well and what is worth pursuing. Accordingly, we suggest performing and developing AI assurance by (1) domain, by (2) AI sub area, and by (3) AI goal; as a theoretical roadmap, similar to what is shown in Fig. 3. In some cases, such as in unsupervised learning techniques, it is difficult to know what to validate or assure [86]. In such cases, the outcome is not predefined (contrary to supervised learning). Genetic algorithms and reinforcement learning have the same issue, and so in such cases, feature selection, data bias, and other data-relevant validation measures, as well as hypothesis generation and testing become more important. Additionally, different domains require different tradeoffs; trustworthiness for instance is more important when it comes to using AI in healthcare versus when its being used for revenue estimates at a private sector firm; also, AI safety is more critical in defense systems than in systems built for education or energy application.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Solvency › Solvency - TEVV
Batarsheh et al. 2021 - Professor of Electrical and Computer Engineering, Virginia Polytechnic Institute [Feras, Laura Freeman Director, Intelligent Systems Division – National Security Institute Research Associate Professor, Chih Hao Huang, Data Analyst at Turing Research. April 26 “A survey on artificial intelligence assurance” https://link-springer-com.proxy.lib.umich.edu/article/10.1186/s40537-021-00445-7#Sec8]
Artificial Intelligence (AI) algorithms are increasingly providing decision making and operational support across multiple domains. AI includes a wide (and growing) library of algorithms that could be applied for different problems. One important notion for the adoption of AI algorithms into operational decision processes is the concept of assurance. The literature on assurance, unfortunately, conceals its outcomes within a tangled landscape of conflicting approaches, driven by contradicting motivations, assumptions, and intuitions. Accordingly, albeit a rising and novel area, this manuscript provides a systematic review of research works that are relevant to AI assurance, between years 1985 and 2021, and aims to provide a structured alternative to the landscape. A new AI assurance definition is adopted and presented, and assurance methods are contrasted and tabulated. Additionally, a ten-metric scoring system is developed and introduced to evaluate and compare existing methods. Lastly, in this manuscript, we provide foundational insights, discussions, future directions, a roadmap, and applicable recommendations for the development and deployment of AI assurance. Introduction and survey structure The recent rise of big data gave birth to a new promise for AI based in statistical learning, and at this time, contrary to previous AI winters, it seems that statistical learning enabled AI has survived the hype, in that it has been able to surpass human-level performance in certain domains. Similar to any other engineering deployment, building AI systems requires evaluation, which may be called assurance, validation, verification or another name. We address this terminology debate in the next section. Defining the scope of AI assurance is worth studying, AI is currently deployed at multiple domains, it is forecasting revenue, guiding robots in the battlefield, driving cars, recommending policies to government officials, predicting pregnancies, and classifying customers. AI has multiple subareas such as machine learning, computer vision, knowledge-based systems, and many more—therefore, we pose the question: is it possible to provide a generic assurance solution across all subareas and domains? This review sheds light on existing works in AI assurance, provides a comprehensive overview of the state-of-the-science, and discusses patterns in AI assurance publishing. This section sets that stage for the manuscript by presenting the motivation, clear definitions and distinctions, as well as the inclusion/exclusion criteria of reviewed articles. Relevant terminology and definitions All AI systems require assurance; it is important to distinguish between different terms that might have been used interchangeably in literature. We acknowledge the following relevant terms: (1) validation, (2) verification, (3) testing, and (4) assurance. This paper is concerned with all of the mentioned terms. The following definitions are adopted in our manuscript, for the purposes of clarity and to avoid ambiguity in upcoming theoretical discussions:
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › AI Apocalypse
Green et al., 2019 - Director of technology ethics at the Markkula Center for Applied Ethics [Brian Patrick et al., Big Data and Cognitive Computing, “Global Solutions vs. Local Solutions for the AI Safety Problem,” https://proxy.lib.umich.edu/login?url=https://www.proquest.com/scholarly-journals/global-solutions-vs-local-ai-safety-problem/docview/2546937568/se-2, 6/18/22 MD]
4.4.2. Ways to Affect a Race to Create the First AI An AI creation race is generally regarded as bad because it encourages the creation of the least-safe AIs first. A war between AIs may also become possible if several AIs are created simultaneously [64,65]. There are many ideas on how to affect an AI race in order to make it safer, that is, to lower the probability of creating dangerous AI. As a race with many participants is a very complex game, there are not obvious ways to predict how it will react to seemingly good interventions, for example, openness. Bostrom has shown that if no one knows the capabilities of others and their own capabilities, it will slow down the race, so openness about capabilities may be dangerous [66]. Actions that may affect an AI race and make it safer may include: Changing the number of participants. Increasing or decreasing information exchange and level of openness. Reducing the level of enmity between organizations and countries, and preventing conventional arms races and military buildups. Increasing the level of cooperation, coordination, and acceptance of the idea of AI safety among AI researchers. Changing the total amount of funding available. Promoting intrinsic motivations for safety. Seth Baum discussed the weakness of monetary incentives for beneficial AI designs, and cautions: “One recurrent finding is that monetary incentives can reduce intrinsic motivation” [67]; when the money is gone, people lose motivation. Baum also noted that the mere fact that a law existed promoted obedience in some situations and that social encouragement can increase intrinsic motivation. Changing social attitudes toward the problem and increasing awareness of the idea of AI safety. Trying to affect the speed of the AI race, either slowing it down or accelerating it in just one place by concentrating research. It is interesting to note that acceleration could be done locally, but slowing it would require global cooperation, and so is less probable. Affecting the idea of the AI race as it is understood by the participants [67]: if everybody thinks that the winner takes everything, the race is more dangerous. A similar framing solution has been suggested in the field of bioweapons, that is, to stop claiming bioweapon creation is easy, as it might become attractive to potential bioterrorists. In fact, bioweapons are not as easy to develop and deploy as is shown in movies, and would probably kill the terrorists first [68].
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › AI Apocalypse
Effoduh, 2021 – PhD candidate at Osgoode Hall Law School [Jake Okechukwu June 23 World Economic Forum, “Weapons powered by artificial intelligence pose a frontier risk and need to be regulated” . BK]
What are frontier risks? Frontier risks are low-likelihood, high-impact threats that could arise as humans explore new realms, whether technological, ecological or territorial. The outcomes cannot be predicted easily (especially by a non-niche group without expert insights), but they have the capacity to alter the world as we know it. These frontier risks are best illustrated by three metaphors: “dragon kings” (rare, high-impact events that are somewhat predictable), “grey rhinos” (highly probable, high-impact events that occur only after a lengthy series of warnings) and “black swans’ (high-impact events that are virtually impossible to predict). The frontier risks that could emerge from the full militarization of autonomous weapons include catastrophic fallout from army raids and a human existential crisis in the age of machine sentience. Why are military systems becoming autonomous? Drones are already used by the military today, but they have vulnerabilities. Most rely on human command through a set of communication channels that can be infiltrated by adversaries. In addition, considerable personnel is required to operate the multiple drones dispatched on the field. By cutting out the need for human control, thousands of L.A.W.S can be dispatched at once, all able to make tactical and timely decisions on their own. This intensifies the speed and expansion of combatants’ reach and reduces the need for proximity to targets. L.A.W.S also exceed the limitations of the human mind and body (for example, they do not risk suffering post-traumatic stress disorders or make erroneous judgment in service of self-preservation). There are, however, some unanswered questions and potential unforeseen consequences. The race by world powers to become leaders in this field have led to what four-star U.S. Army General David Petraeus called “the early stages of a tech Cold War.” Similarly, the entrepreneur Elon Musk warned in 2017 that the scramble for military applications of artificial intelligence could become so intensified that it causes WWIII. Because of the popularity of robot doomsday speculation in pop culture over the decades, the fear of L.A.W.S. is already ingrained in the public imagination. But due to political sovereignty some states do not seem as moved by the fears of the potential risks from L.A.W.S. There are also insufficient global regulations on the use of militarized AI. While the United Nations and the European Parliament have fielded proposals to ban the use of lethal autonomous robots in war, there are still unanswered questions about how the weapons could be regulated and whether regulation would conflict with states’ sovereignty. Will the machines save us or kill us? Machines make mistakes, albeit less often than human soldiers do. But while humans are held responsible for their actions, machines cannot suffer legal consequences. With the complexities of international humanitarian law (IHL) regarding warfare, there could be the risk of setting loose machines that have difficulty differentiating among active combatants, injured soldiers and those surrendering, flouting the 1949 Geneva Conventions. One could argue that the idea of fully autonomous weapons already contravenes IHL. For instance, the Martens Clause prioritizes human dignity and the capacity of soldiers to show compassion toward their fellow humans. But robots powered by lines of code do not have the capacity to make such decisions yet and could end up flouting the rules. Another concern about violations of IHL is underscored by the potential for L.A.W.S. to act of their own accord. An idea that used to seem like fodder for the most imaginative of dystopian fiction is gaining more traction in recent years; Musk and theoretical physicist Professor Stephen Hawking both warned about sentient AIs establishing a takeover and threatening human civilization. The UN Convention on Certain Conventional Weapons and Article 14 of the New Delhi Rules prohibits the use of weapons that could escape from the control of those who employ them, thus endangering the civilian population. While that scenario might still be considered unlikely, this is a kind of frontier risk that could have catastrophic impact on the world.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › Operator Trust – TEVV Solves
Haugh, 2018 - project leader for the Institute for Defense Analysis [Brian A., September “The Status of Test, Evaluation, Verification, and Validation (TEV&V) of Autonomous Systems,” , 6/23/22 MD]
4. CONOPS and training as design features To date, the paradigm for designing systems has been to make a reasonable guess about how the operator(s) will use that system and what the user interface should look like, and then work out the details of the Concept of Operations (CONOPS), tactics for effective employment, and training of future operators long after the basic design has been decided. For autonomous systems, where the system operates itself to some extent and interacts autonomously with humans, the details of CONOPS and tactics (and corresponding training) are part of the system design, at least on the machine side (and probably on the human side as well), and will have to be identified, verified, and validated much earlier in the development process. This will pose organizational and personnel challenges to T&E, as well as methodological challenges. 5. Human trust In human-machine teaming (HMT) contexts, how the humans behave (and thus how well the team performs) depends in part on the humans’ psychological attitudes toward the autonomous systems. “Trust” is the term generally used to describe those attitudes, though in practice those attitudes are generally more nuanced and multi-dimensional than simply asking “how much do I trust it?” In order to design, debug, and assure performance, TEV&V will need to be able to measure the various dimensions of trust, to support understanding of how trust affects team performance.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › Operator Trust – TEVV Solves
Haugh, 2018 - project leader for the Institute for Defense Analysis [Brian A., September “The Status of Test, Evaluation, Verification, and Validation (TEV&V) of Autonomous Systems,” , 6/23/22 MD]
B. Cognitive Instrumentation and Explainable AI 1. Summary The key distinguishing feature of autonomous systems is that they make decisions, when interpreting their environments and selecting courses of action. Test and evaluation of autonomy will depend critically on the ability to assess the quality of this decision making capability. In general, observed high-level system performance in limited test scenarios will not be sufficient to validate and verify autonomous decision-making in the ways necessary for successful development and deployment, especially for systems designed to team with humans. Instead, novel instrumentation approaches will be required to diagnose and characterize: • Adequacy of architectures and algorithms (including machine learning). • Adequacy and appropriateness of training data. • Effectiveness of operational concepts for HMT. 2. Why Explanations Are Needed Explanations of system behavior support at least four distinct goals for military systems: 1. Diagnosis: Knowing why the system is exhi biting undesired behavior is the first step toward fixing it. 2. Prediction: Being able to forecast how the system will behave in given circumstances is essential to effective employment of the system. 3. Bounding: Understanding the limits of dependable performance allows formulation of tactics/techniques/procedures for how the system is to be used and identification of where run-time monitoring of system state may be the only way to avoid undesirable behaviors. 3-3 4. Trust: If humans are teaming with autonomous machines, the overall performance of the human-machine system may depend on how well the humans feel they understand the reasons behind the machine’s behaviors. For traditional software employing procedural algorithms, the logic of the algorithm traditionally serves as the explanation. Describing the flow of control (e.g., using pseudocode or flowcharts) at various levels of description could “explain” the behavior of the system. As systems have grown in complexity, the ability of humans to understand the logic of the systems in a way that counts as an “explanation” has eroded. Machine learning breaks this paradigm completely by replacing comprehensible procedural logic with a trained ability to generate outputs from inputs in a manner that looks much more like an intuition or hunch than like reasoning. The relationship between input and output is fundamentally opaque; there is no procedural logic to be traced. Producing surrogate procedural descriptions that summarize how the system is reaching its conclusions in ways that can function as explanations for purposes of diagnosis, prediction, bounding, and trust requires additional work and access to the internal states (and possibly the training algorithm) of the machine-learning module. (Note that different explanatory frameworks will generally be needed for each of these goals—the explanation appropriate for diagnosis may be very different from the explanation required for bounding or trust.) The US Defense Advanced Research Projects Agency (DARPA) has recently launched a funded research program in “Explainable Artificial Intelligence” (XAI). The four goals mentioned above (diagnosis, prediction, bounding, and trust) are all represented in the Explainable AI program. “Explainability” in the XAI program involves multiple measures of effectiveness, depending on which of the goals is being pursued. These measures can include human satisfaction with the explanation, how helpful the explanation is in choosing how to interact with the autonomous system, how well the explanation supports diagnosis of unexpected behaviors, or how well the explanation predicts future system behavior. 3. Cognitive Instrumentation The phrase cognitive instrumentation refers to any measurement of internal system states of the software modules that provide the autonomous capabilities of a system. There is an obvious analogy with physical measurement of internal system state (e.g., temperature, pressure, voltage, torque) during development, test, and evaluation of hardware systems. As with those physical analogues, the measurements may be only for TEV&V purposes, or they may be incorporated into the design of the system as run-time monitors on behaviors that cannot be adequately assured through other means. One area of explainability research involves designing autonomous systems to selfreport their reasons for their behaviors. This is a promising approach, but it does not avoid 3-4 the need for instrumenting the internal states of the system. At a minimum, verification and validation of the self-reporting system will involve the same diagnosis, prediction, and trust issues as the mission behavior of the system, with an associated need to look inside the black box and make sense of what is happening there. At some point, all system development requires knowing the truth about what is happening inside the system. In the special case of HMT, where the machines have significant autonomous capabilities, cognitive instrumentation will include measurements of both the machines and the humans, as well as the interface between them. Measurement and characterization of trust by humans is already a rich area of academic research. Measurement and characterization of machine understanding of human intent is also receiving increasing attention. There are also active research programs exploring the design of HMT protocols. Some of these focus on the optimal allocation of work between humans and machines in various contexts. Others are concerned with optimal communications protocols—under what circumstances should humans or machines volunteer information (and which information), make suggestions, or ask for guidance? Without cognitive instrumentation, those optimization programs could easily devolve into guesswork and trial-and-error. Cognitive instrumentation is a necessary condition for Explainable AI; any valid explanation of how the AI is thinking or why it is behaving a certain way must be based on accurate measurements of its internal states. However, the measurements themselves are not the explanation. A complete state description will generally be so complex that it is not understandable, whereas too small a subset of measurements will generally not be sufficient to support explanation. Additional effort will be required to identify a minimal sufficient set of measurements that can be used as inputs to explanatory models that can be understood in human terms for diagnosis, prediction, bounding, and developing appropriate operator trust. There are many tricky steps between “the weights in layer 17 of the neural network are as follows” to “the system seems to be recognizing cats by their ears and coloration.” As noted above, different explanatory models will be needed for the different goals—an explanatory model supporting useful diagnosis for developers may be much more complex (and unwieldy) than a model that produces explanations of mission behavior for operators or commanders.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › Operator Trust – TEVV Solves
Gilli, 2020 - Senior Researcher at the NATO Defense College [Andrea, NDC Research Paper No.15 – December ““NATO-Mation”: Strategies for Leading in the Age of Artificial Intelligence” https://www.ndc.nato.int/news/news.php?icode=1514 Acc 4/21/22 TA]
This is important because AI is data-intensive, unpredictable and brittle.98 While a system may work well in the context in which it was trained, it may break in an unfamiliar setting. The reliance on BD in the current second wave of ML also creates fallibilities,99 given that the algorithms can scale up harm if the data over- or under-represents certain groups.100 For neural networks in particular, it may be impossible to explain or interpret results. Some refer to this as the “black-box” problem, meaning that the outcomes of these complex AI systems are opaque to humans that either want to reproduce the good, or prevent the bad from recurring. While traditional software can be debugged to solve a performance issue, the lack of linear causality between a programmer’s inputs and the AI system’s outputs means that it is difficult to track bias and reliability. Creating organizational processes to minimize these concerns across the AI lifecycle is critical to responsible use of the technology. Considering AI ethics also means developing trust in systems. Organizations developing AI applications should think deeply about user expectations related to transparency and disclosure. Ultimately, these norms will change the distinction between human, AI-assisted and AI interactions.101 “Calibrating” trust is especially important for AI-assisted decision making,102 a concept whose value will only increase given the emphasis on human-machine teaming in Allied militaries.103 In many cases, the unethical or unacceptable outcomes of AI systems pertain not to moral dilemmas, but to the reliability and robustness of the systems at hand. As such, building trust is fundamentally tied to building safe and secure systems.104 The problem of bias in AI illustrates the overlap between AI safety and AI ethics: bias is morally problematic, because it can unfairly harm or systematically discriminate against specific groups105 – and it can also be seen as a failure mode that reduces the reliability of a given algorithm in a given context.106 For NATO, this relates to the safety of enterprise tools and weapons systems alike. In deployments, these safety measures would be critical to ensure that AI-enabled weapons systems are used in a manner consistent with the principles of international humanitarian law. As explained in greater detail below, this may also feed into the eventual standardization process.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › Operator Trust – TEVV Solves
Tarraf, Shelton, Parker 2019 - Senior Information Scientist, Senior Engineer, and physical Scientist at the RAND Corporation [ Danielle C., William, Edward, et al RAND Cooperation, “The Department of Defense Posture for Artificial Intelligence”, , Acc 6/18/22, M.A.]
As we noted in Chapter Three, our interviewees agreed that V&V of AI systems is a critical topic that DoD will need to address, particularly when it seeks to deploy and use safety-critical AI systems. 18 Interviewees both inside and outside DoD emphasized the importance of improving the understanding of human-machine interaction. 52 The Department of Defense Posture for Artificial Intelligence However, like our academic and industry interviewees (see section “Industry: Advancement and Adoption” in Appendix C), the interviewees lacked current V&V solutions and could not predict which technical avenues toward V&V would eventually prove to be effective for AI systems. Concerns about the “valley of death” and the failures in transitioning research to prototypes and products, both broadly and specifically for AI, also came up in our interviews.19 Finally, although our interviewees generally agreed that AI will have a bigger role to play, they also emphasized the need for training and experimentation to build trust and enhance adoption of these technologies, while noting that generational factors might help down the line.20
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › Operator Trust – Operator Trust essential to Military
Costello, 2020 - Chief of Staff for Test Wing Atlantic [Donald, "Certifying an Autonomous System to Complete Tasks Currently Reserved for Qualified Pilots” - ProQuest accessed on 6-21-2022 TM]
Trust is vital for certification. When a military commander certifies a pilot as fully qualified they are bestowing their trust on that pilot. Following qualification, the pilot is expected to use his judgment to make decisions based on their experiences. When dealing with autonomy, trust is not inherent and certification is not business as usual. For commanders to trust that an autonomous system will perform as a pilot would will require methods and metrics different than what they are accustomed to. 37 The Defense Science Board identified trust as an integral requirement for the use of autonomous systems by the DoD. “The decision for DoD to deploy autonomous system must be based both on trust that they will perform effectively in their intended use and that such use will not result in high-regret, unintended consequences. Without such trust, autonomous systems will not be adopted except in extreme cases such as mission that cannot otherwise be performed. Further, inappropriate calibration of trust assessments – whether over-trust or under-trust during design, development, or operations will lead to misapplication of these systems. It is therefore important for DoD to focus on critical trust issues and the assurance of appropriate levels of trust [91].” Autonomy is a new concept for the DoD. When employing military systems in the field, someone is ultimately responsible for the actions of that system. This may include the actual military member employing the technology or the individual that certified it for use. Having a system that exhibits non-deterministic behavior inherent in autonomy is a new concept for certification officials. Trust needs to be built prior to the use of autonomy within the DoD.
Operator trust is essential because AI is a black box – operators cannot examine the machine logic for themselves.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › Operator Trust – Unsafe AI Undermines Trust
National Security Commission on Artificial Intelligence 2022 [March 19, “NCSAI Final Report 2021 – Chapter 7: Establishing Justified Confidence in AI Systems”, ]
Artificial intelligence (AI) systems must be developed and fielded with justified confidence.1 If AI systems do not work as designed, or are unpredictable in ways that can have significant negative consequences, then leaders will not adopt them, operators will not use them, Congress will not fund them, and the American people will not support them. “ “If AI systems ... are unpredictable in ways that can have significant negative consequences, then leaders will not adopt them, operators will not use them, Congress will not fund them, and the American people will not support them.” Achieving acceptable AI performance often is linked to the decision to accept some level of risk. No technology works perfectly under all conditions. Risk calculus changes with circumstances. The variables and considerations that inform judgments to rely on AI will vary significantly across military, intelligence, homeland security, and law enforcement missions. In a high-threat environment like combat, in some cases it may be reasonable to employ a system offering some immediate military advantage, while recognizing that it might fail; in other cases, however, a reasonable commander might want the highest assurances of AI reliability before fielding when lives are at risk. As departments and agencies rely more heavily on machines, a central guiding principle across national security scenarios is the continued centrality of human judgment. Those charged with utilizing AI need an informed understanding of risks, opportunities, and tradeoffs. They need awareness of the possibilities and limitations in a system’s expected performance. Ultimately, they need to formulate an educated answer to this question: In the given circumstance, how much confidence in the machine is enough confidence? These issues bear on the full lifecycle of an AI system––from acquisition or system development and the thresholds for justified confidence to deploy a specific AI-intensive system to the performance of the system in the field. While there is no absolute assurance of perfection, there are policies and best practices that support making these decisions responsibly. Agencies are broadly aware of the principal challenges in employing AI systems and the necessity of incorporating best practices in the engineering and management of AI systems.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Operator Trust Advantage › Operator Trust – Unsafe AI Undermines Trust
Costello, 2020 - Chief of Staff for Test Wing Atlantic [Donald, "Certifying an Autonomous System to Complete Tasks Currently Reserved for Qualified Pilots” - ProQuest accessed on 6-21-2022 TM]
While some level of trust is required for certification officials to accept risk, there is not a clearly defined method for how we build trust in military systems. The current V&V techniques are designed for systems with a qualified pilot or operator. To achieve a level of trust, V&V techniques used for certifying autonomy must be sufficient that the certification officials will have trust in the actions of the decision engines controlling the vehicle, and trust that the system will not pose an unnecessary risk to mission completion. For a system to operate autonomously, it will need to sense the environment it is currently operating in. Upon sensing the environment, it will then need to properly classify the input its onboard sensors give it. Once it has interpreted the environment, and built its own SA, it will need to perform appropriate actions based on the unpredictable environments it will find itself in. This is a similar process that a fully qualified pilot is expected to perform once leadership has bestowed trust on their actions. One issue with using sensors to build the SA for a UAV 41 is the vast amount of data the needs to be filtered through. Using automation to process onboard imagery is not a new concept. From December 2013 to January 2015 a CubeSat was flown with multiple onboard sensors. An autonomous onboard processing was used to determine what images would be passed back to the ground station for further processing [64]. While trust is required for certification officials to accept risk, there is not a clearly defined method for how we build that trust in autonomous military systems. The V&V techniques used for certifying autonomy must be sufficient that the certification officials will have trust in the actions of the decision engines controlling military aircraft.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Cyber Attacks Advantage › Cyber – DOD Key
Nurkin and Konaev, 2022 - senior fellows at the Center for Strategy and Security at the Atlantic Council [Tate and Margarita, 5-25-2022, “Eye to eye in AI: Developing artificial intelligence for national security and defense” Atlantic Council ARD]
Use Case 3: The Limits of AI Adoption in the Information Domain Intensifying competition with China and Russia is increasingly playing out in the information and cyber domains with real, enduring, and disruptive 53 Ibid. implications for US security, as well as the US economy, society, and polity. For cyber and information operations, AI technologies and techniques are central to the future of both offensive and defensive operations, highlighting both the peril and promise of AI in the information domain. Concern is growing about the threat of smart bots, synthetic media such as deepfakes—realistic video or audio productions that depict events or statements that did not take place—and large language models that can create convincing prose and text.56 And, these are just the emerging AI-enabled disinformation weapons that can be conceived of today. While disinformation is a challenge that requires a societal and whole-of government response, DoD will undoubtably play a key role in managing and responding to this threat— due to its prominence in US politics and society, the nature of its functional role, and the impact of its ongoing activities. AI is at the forefront of Pentagon and other US government efforts to detect bots and synthetic media. DARPA’s MediaForensics (MediFor) program is using AI algorithms to “automatically quantify the integrity of an image or video,” for example.57 Still, there is concern about the pace at which this detection happens, given the speed of diffusion of synthetic media via social media. As Lt. General Dennis Crall, the Joint Staff’s chief information officer, observed, “the speed at which machines and AI won some of these information campaigns changes the game for us…digital transformation, predictive analytics, ML, AI, they are changing the game…and if we don’t match that speed, we will make it to the right answer and that the right answer will be completely irrelevant.”58
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Cyber Attacks Advantage › Cyber – NATO Key
Taddeo and Floridi, 2018 - Senior research fellow at the Oxford Internet Institute and Professor of Philosophy and Ethics of Information at the University of Oxford [Mariarosaria, Luciano, Nature.com, “Regulate artificial intelligence to avert cyber arms race,” 6/18/22 MD]
No rules Right now the UN process is in deadlock. In 2004, the United Nations set up a Group of Governmental Experts (UN GGE) to agree on a set of voluntary rules for how states should behave in cyberspace. Their 5th meeting in 2017 ended in a stand-off. The Group could not reach consensus on whether existing international laws on self-defence, humanitarian laws, and state responsibility should apply in cyberspace. The US argued cyber-defence regulations should build on these foundations. Other 3 nations, including Cuba, Russia, and China, firmly disagreed, arguing that this would ‘militarize’ cyberspace,sending the wrong message about peaceful conflict resolution. The Group failed to deliver its report. It is unclear whether it will meet again, or what will happen next. International dialogue and action must resume. NATO could pave the way through its forthcoming guidelines. Although it is currently unclear what their scope will be.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Cyber Attacks Advantage › Cyber – NATO Key
Taddeo and Floridi, 2018 - Senior research fellow at the Oxford Internet Institute and Professor of Philosophy and Ethics of Information at the University of Oxford [Mariarosaria, Luciano, Nature.com, “Regulate artificial intelligence to avert cyber arms race,” 6/18/22 MD]
The EU is stepping-up too. In 2017, it reassessed cyber security and defence policies and launched the European Centre of Excellence for Countering Hybrid Threats. EU directives present the most comprehensive regulatory framework for state conduct in cyberspace to date. Yet they do not go far enough. The EU treats cyber defence as a case of cyber security, to be improved passively by making member states’ information systems more resilient. It disregards active uses of cyber defence and does not include AI. This is a missed opportunity. The EU could have begun defining ‘red lines’ and proportionate responses. For example, the EU Directive on Security of Network and Information Systems [10] provides criteria for identifying critical national infrastructures, such as health systems or key energy and water supplies that should be protected. The same criteria could be used to define illegitimate targets of state-sponsored cyber-attacks. Regional forums, like NATO and the EU, must take the following three steps to avoid serious imminent attacks on state infrastructures and to maintain international stability. Three steps Define legal boundaries. The international community needs to agree urgently upon ‘red lines’ distinguishing legitimate and illegitimate targets and definitions of proportionate responses for cyber defence strategies. International consensus at the UN level will ultimately be required. Until then guidelines from regional multilateral bodies, like NATO and the EU, must cover these issues and lead by example. Test strategies with allies. ‘Sparring’ exercises should be organised between friendly countries to test AI-based defence tactics. These tests should be mandatory before any system is deployed. They could be in the form of DARPA’s Grand Challenge or the cyber simulation exercises routinely run by NATO and the EU. Because AI learns by experience, these matches will improve the strategies of the alliance, while finding and healing weaknesses. Fatal vulnerabilities of key systems and crucial infrastructures should be shared with allies; policy frameworks should demand disclosure. Agreements and regulations with similar sharing and disclosure requirements include the EU eIDAS (electronic IDentification, Authentication and trust Services) Regulation and the IPA (Industry Partnership Agreement) within NATO. Monitor and enforce rules. The international community needs to agree on how to audit and oversee AI-based state cyber defence operations. Alert and remedy mechanisms are needed to address 5 mistakes and unintended consequences. A third-party authority with teeth, such as the UN Security Council, should rule on whether red lines, proportionality, responsible deployment or disclosure norms have been breached. As in the case with other international norms, economic or political sanctions should be imposed on states violating rules. NATO and the EU should enforce the norms within their remits.
2022
MNDI
Affirmative
TEVV AFF – MNDI HPA › Cyber Attacks Advantage › Cyber - Solvency
Fitzpatrick, 2019 - former the IISS Non-Proliferation and Nuclear Policy Programme [Mark, 21 May, “Artificial Intelligence and Nuclear Command and Control,” MD]
Finally, while disinformation is a long-standing intelligence and strategic problem that pre-dates the cyber age, the integrity of AI systems is especially vulnerable to it. The combination of reliance on AI and its appropriation by malevolent actors could seriously amplify the disruptive effects of disinformation campaigns. Governments could play a key role in establishing regulatory frameworks that shape how new technology interfaces with the current suite of disinformation tools, such as social media. Although time limitations precluded further exploration of an AI system’s vulnerability, hints and suspicions of AI-linked espionage permeated the tabletop exercise. They are likely to persist in the real world as well.