Trustworthy AI values relate to safety and security and reflects of human essence ---although they are not trustworthy based on environmental awareness
Schmid et al ’22 (Schmid, S-- human-computer interaction cscw crisis informatics
., Riebe, T-- is a research associate and doctoral student at the Chair of Science and Technology for Peace and Security (PEASEC) in the Department of Computer Science at the TU Darmstadt.
., & Reuter, C.-- Christian Reuter is Full Professor at Technical University of Darmstadt. His chair Science and Technology for Peace and Security (PEASEC) in the Department of Computer Science with secondary appointment in the Department of History and Social Sciences combines computer science with peace and security research
(2022). “Dual-Use and Trustworthy? A Mixed Methods Analysis of AI Diffusion Between Civilian and Defense R&D.” Science and Engineering Ethics, 28(2). https://doi.org/10.1007/s11948-022-00364-7)//Marzz
Robustness, accuracy, and information quality seem to be apparent values which support Trustworthy AI, when considering military purposes. This does not mean that these norms are entirely absent when it comes to civilian AI applications. Instead, our analysis indicates that they are relatively more prevalent in the military context (see supplementary material Table A). Thus, as a value, robustness is comparatively more significant in the context of military (D11) applications, including resilience as an important standard (D10). Further, accuracy is particularly important in the context of military applications, including transparency on problems of inaccuracy: Although the RMS [root mean square; author’s note] errors for building reconstruction […] indicate that our method provides reasonable geometrical accuracies (height error is the same as for single points if the parallax accuracy is about one image pixel), the results in building detection are less precise. (D12.; own emp.) Similarly, the EU guideline stresses the importance of the technical values of robustness and accuracy. These relate to both safety and security, which are crucial in warfare scenarios. Military AI applications may support standards of Trustworthy AI, paying special attention to robustness and accuracy (European Commission, 2019) in more critical contexts. This reflects the potential to ensure security as proposed by the EU guideline (European Commission, 2019), while also indicating the technology’s possible normative ambiguity regarding general human and environmental well-being. Information quality has also been relatively more important for military (D1) applications. Given the high stake of a military operation, errors due to low information quality may have a greater impact on people, e.g., by mistaking civilian infrastructure for military bases or by falsely engaging civilians as combatants. Trustworthiness of Civilian AI Applications At the same time, there is a comparatively stronger interest in civilian projects in awareness, indicating the importance of capturing the environment in all its complexity. For example, SPARC, a project on autonomous driving in urban trafc, relies heavily on orientation in the context of moving and directing surrounding objects, opting for a “holistic representation” (D3), while at the same time training data is focused on “eventful […] and […] unique situations” (D13). Whether in terms of space, time, or speed, there is a strong reference to environmental information. This is surprising, as situational awareness is not only stressed by the EU (European Commission, 2019) but is mostly apparent in military contexts. Overall, civilian applications emphasize the relevance of explainability, which is referred to as “retaining many of the advantages of variational trajectory optimization methods, in particular expressiveness” (D11; own emphasis). Others underline that “[t]he ability for humans to understand the reasoning process is essential to the presented case study” (D13). This highlights the ambivalence of explainability as a normative concept. While it may be defined as the ability to explain, interpretability, namely the ability to provide (grounds for) an interpretation, is often associated with the concept of explainability, as it is also the case in the Trustworthy AI guide (European Commission, 2019). This requirement for civilian applications may be plausible, should special attention be paid to a broader and more diverse group of end-users. This becomes particularly apparent considering that the project on autonomous driving in cities (D11) stresses explainability (or expressiveness) the most. It should be noted that both security and safety were also qualitatively deduced regarding military applications, indicating human-centric approaches albeit in different terms. Human dignity, implying human-centric approaches, represents one of the core values of Trustworthy AI (European Commission, 2019). Such statements are more common in the context of civilian applications; as they apply AI applications that put focus on human reasoning, hand gestures, or the human body (see supplementary material Table A). Military applications accordingly refect less interest in a precise analysis of the social or intimate environment. Yet, a strong focus on people’s movements or behavior does not necessarily imply the implementation of a human-centric AI in terms of human dignity or personal rights. Difused Values Across Civilian and Military Applications Regardless of the feld of application, the authors of scientifc publications were transparent about procedural problems. In contrast, AI was depicted relatively fawless in online presentations of projects or product fyers. This may be due to the nature of scholarly debates, supporting values such as transparency (of problems). Problematic issues were not made transparent in shorter, more easily accessible online contributions, while such documents contained more direct references to economic merits. The European expert group’s guide would suggest presenting complex, inconvenient facts to a broader audience and allow for understandability independent from personal background (European Commission, 2019). Furthermore, the fgurative alignment of AI and animal behavior became visible. AI projects were oriented towards phenomena in nature, for example in the development of “swarms” of UAVs or processing as in an “ant colony” (D10). AI was also designed to imitate the human essence. This is refected in notions about the AI’s self and its abilities (see supplementary material Table A). Trustworthy AI refers to approaches such as values-by-design, implying a certain degree of technological agency (European Commission, 2019). However, Fraunhofer projects do not refect the awareness of such interactional approaches or non-human agency. While projects indicate anthropomorphization of AI as well as bionic models, they do not guarantee trustworthiness based on environmental awareness.