Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Centralized implementation causes uninformed decisions – fails to control for accidents, biases, data poisoning AND weakness – turns the case AND causes unintended technological outcomes

Michael Horowitz & Kahn 20, Lauren Kahn, Michael C. Horowitz is professor of political science and the interim director of Perry World House at the University of Pennsylvania. Lauren Kahn is a research fellow at Perry World House at the University of Pennsylvania.1-14-2020, "The AI Literacy Gap Hobbling American Officialdom," War on the Rocks, https://warontherocks.com/2020/01/the-ai-literacy-gap-hobbling-american-officialdom/, nihara

Because AI is a general-purpose technology, the corresponding adoption challenges may prove especially difficult. There is a great deal of emphasis at present on how the United States can more effectively recruit and retain AI talent to work for the national security community and the government as a whole. This is critical, but the people making decisions about the use of algorithms from the situation room to the battlefield will not necessarily be informed about current developments in AI, let alone be AI experts, but military leaders and policymakers. Thus, a vital challenge is familiarizing and educating government leaders and policymakers about AI. This is a different challenge than that of incentivizing those with AI expertise to work for the U.S. government. Instead, it is about AI education for the policy community.

AI education for the policy community, from military leaders to Hill staffers to senior government officials, needs to happen now, as cases outside of the national security sphere illustrate. For example, in December 2017, the New York City council unanimously passed a bill that required the Office of the Mayor to form a task force dedicated to increasing transparency and oversight of the algorithms used by the city. A local city council member characterized the bill as essential, saying of the algorithms, “I don’t know what it is. I don’t know how it works. I don’t know what factors go into it. As we advance into the 21st century, we must ensure our government is not ‘black boxed.’”

This example touches on a key dynamic: Top policymakers — who are generally not technically trained — are at an increasing risk of being “black boxed” as technological complexity increases. This is especially true given questions even at the vanguard of AI research about the “explainability” of algorithms. However, organizational decisions about AI adoption and applications that “generally shape the impact of that technology” are being made today. Section 256 of the FY20 NDAA provides a first step towards national security literacy in AI, requiring the Secretary of Defense to develop an AI education strategy for military servicemembers. But there are questions about implementation, given well-known issues with professional military education, and the national security community more broadly requires AI literacy, not just military service members, since most policymakers are not in uniform.

The effective use of algorithms in the national security sphere requires basic education as well as recognition of what algorithms can and can’t do, the risks of automation bias and cognitive offloading to machines, the dangers of accidents and data poisoning, among other challenges. These need to be demystified and made relevant to national security officials, especially those senior leaders who may not consider themselves tech-savvy. It is impossible to hedge against potential dangers of a given technology if those using it do not fully understand and therefore many not be able to control it. Failure to control for biases, weaknesses, and accidents of even the most innocuous-seeming of algorithms has the potential to disrupt and create unintended policy outcomes that undermine US leadership in AI and US national security in general. Fortunately, there are a number of paths forward for effective AI education for end users, from online coursework to programs in civilian universities, to specially designed national security training programs.

Sign in to recut, update, or save this card.