Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Costs of experimentation are decreasing, interest is going up, but establishing a precedent for rigorous evaluation is key

Colleen V. Chien 19, Justin D'Atri Visiting Professor of Business Law, Columbia Law School; Professor, Santa Clara University School of Law; 2013-2015 White House Senior Advisor, Innovation and Intellectual Property, “Rigorous Policy Pilots: Experimentation in the Administration of the Law,” 104 Iowa L. Rev. 2313, July 2019, Lexis

It is also worth remembering that calls for rigorous evaluation are not new and their results have not necessarily been encouraging. Critics have described evaluation mandates embedded in the Government Performance and Results Act of 1993 (“GPRA”), the GPRA Modernization Act of 2010 (“GPRAMA”), and successive generations of retrospective review requirements as hampered by resource constraints, pro forma compliance, and a lack of oversight.

However, several developments make it an opportune time to persevere and attempt to answer, not abandon their call. First the costs of controlled experimentation are going down, thanks to the advance of open data, and the adoption of “agile” and “policy lab” approaches in government, which have increased the expertise and decreased measurement and assessment costs. Second, the embrace of performance- or results-based lawmaking has shifted attention away from policy processes and towards policy outcomes, discoverable through trial and error or disciplined experimentation. Third, the 2018 Evidence Act, if implemented as intended, should significantly shore up agency capacity for rigorous evaluation. The Act requires departments and agencies within the Act’s jurisdiction to submit annual plans for identifying and addressing policy relevant questions that specify “data to . . . facilitate the use of evidence in policy making.” It establishes an Interagency Council on Evaluation Policy, requires each relevant agency to designate a Chief Evaluation Officer and requires the Office of Management and Budget (“OMB”) to establish an Advisory Committee on Data for Evidence Building to advise on expanding access to and use of federal data for evidence building. These activities should inform and ease at least two of the steps of rigorous piloting described below—identifying important questions and developing the evidence to evaluate them.

Sign in to recut, update, or save this card.