Pilot programs are key to effective regulatory implementation.
Colleen V. Chien 19, Justin D'Atri Visiting Professor of Business Law, Columbia Law School; Professor, Santa Clara University School of Law; 2013-2015 White House Senior Advisor, Innovation and Intellectual Property, “Rigorous Policy Pilots: Experimentation in the Administration of the Law,” 104 Iowa L. Rev. 2313, July 2019, Lexis
In the government, fear of making a Type 1 or “false positive” error—taking a mistaken step—leads to an overabundance of Type 2 or “false negative” errors—failing to take a productive step. At root, however, is the problem of “what [the] government doesn’t know,” regarding what changes to law or policy to consider taking, what the impact of such changes might be, and which among several potential changes is most likely to achieve a policy goal, all topics about which there may be many opinions, but little relevant evidence. Uncertainty about whether a law or policy change will achieve its intended purpose, fear of making a mistake in the face of this uncertainty, and institutional inertia all contribute to status quo bias—whether in favor of preserving regulations that don’t work or failing to adopt new policies that do.
One way to address both the knowledge gap and risk aversion in policy development is by introducing a temporary change to law or policy for the purpose of learning from it, through a “policy pilot.” Implemented with rigor, through the application of, “well-designed and well-implemented methods tailored to the questions being asked”—rigorous policy pilots are a generative yet under-used tool for addressing informational deficits that stand in the way of developing effective law and policy.
The range of open questions in law and policy that can be addressed by pilots are wide-ranging. They include, for example, the question of how giving a universal basic income (“UBI”) to individuals impacts their well-being and employment. In 2017, the Finnish government began tracking the outcomes of 7,000 unemployed citizens, about a third to which it gave $600 a month, to address this question; several U.S. jurisdictions are planning their own tests. Also unknown is how the administration of patent law and issuance of quality patents best advance the Constitutional goal of “promot[ing] the progress of science and useful arts.” Over the last decade, the United States Patent and Trademark Office (“USPTO”) has run numerous pilots to address this question. As the D.C. Circuit has said, “there are some situations in which, ‘a month of experience will be worth a year of hearings.’” While experiments or pilots can address a variety of questions pertaining, e.g., to a policy’s feasibility or stakeholder reactions, when the question concerns whether the policy has caused an observed outcome, a controlled trial that provides the treatment to one part of population, withholds it from another, and then compares the difference in outcomes provides the best insight. Such rigorous experiments can be designed, in turn, with or without randomization, and to vary the rules or laws that apply or to approximate such changes. A pilot that limits to whom a rule or policy applies constrains the risk but not the learning. As Justice Brandeis famously dissented, “a single courageous state may . . . serve as a laboratory; and try novel social and economic experiments without risk to the rest of the country.”