Cognitive bias due to lack of expertise
In this paper, Toma and Bell, attempt to shed light into the decision-making process of policymakers by providing insights into how policymakers respond to evidence about program impact when assessing the value of government programs. They use a lab-in-the-field experiment among high-ranking U.S. policymakers to document a limited sensitivity to impact. They find:
- Policymakers’ assessments of the value of a policy program are inelastic with respect to impact; that is, their valuations don’t update one-to-one with a change in program impact.
- The use of two decision aids that aim to simplify the decision problems – i.e. the presentation of two similar programs Side-by-Side, and an Impact Calculator that translates the total cost of a program into an annual cost per person – increases policymakers’ responsiveness, or sensitivity, to evidence-based information about a program;
- The inability of mapping scientific findings to an assessment of program value plays an important role in driving the observed inelasticity. The authors find that when program impact increases by 100%, the value individuals ascribe to a program increases by 33%.
Understanding and Increasing Policymakers’ Sensitivity to Program Impact
Authors: Mattie Toma, Elizabeth Bell
From: Warwick Business School, University of Oxford
Overconfidence
In this paper, Vivalt and Coville, look at how policymakers update their beliefs when interpreting new evidence even when they have the benefit of experts’ feedback. They ran a set of experiments on three groups of people involved in the generation and translation of research into policy decisions: policy-makers, policy practitioners and researchers were invited to World Bank (WB) and Inter-American Development Bank (IDB)’s impact evaluation workshops. During the workshops policy-makers and policy practitioners were paired with researchers and tasked with designing a prospective impact evaluation for their program over the course of the week. Some of the takeaways from the experiments include:
- Policymakers believed development programs will have more positive results and were more certain about it than policy practitioners and researchers, despite reporting less familiarity with the programs.
- When participants were presented with the results of impact evaluations, the authors found evidence that policymakers and policy practitioners updated their beliefs asymmetrically on good news. They also found evidence of overconfidence.
New information affected the allocation of real resources, suggesting that these biases affect policy decisions. Our results imply that imprecisely estimated “good news” may be particularly dangerous.
How Do Policy-Makers Update Their Beliefs?
Authors: Eva Vivalt, Aidan Coville
From: University of Toronto, World Bank
