Friday, June 1, 2012

One-armed economists and Potomac Fever: Suggestions for the lay public

Or, public policy is hard.

The well-known story goes that Harry Truman asked for a one-armed economist because his economic advisers always gave "on the one hand,... but on the other hand..." answers to policy questions. The best economists still give these kinds of answers, but when I read the econ blogosphere I occasionally encounter Truman's preferred kind of experts: economists or econ enthusiasts using the discipline as if it were conducive to easy answers to policy questions. In this post, I'm going to describe the problem and offer some suggestions for non-economists consuming economic analysis. (If you don't have time to read the whole post, skip down to the bullet points).

The truth is, economics is complicated. Lacking full laboratories in which to conduct experiments, economists and other social scientists carefully apply what can only be called second-best (or worse) methods to imperfect data to tease out reliable assessments of how things work. Further, not only do policymakers face a set of difficult tradeoffs for every policy, but even when preferences over those tradeoffs can be agreed upon it is rarely obvious how policy levers should be pulled to accomplish desired outcomes. This is all complicated by the fact that social scientists “are people too” and therefore are vulnerable to the same biases which afflict everyone else (hopefully we are careful to put in place checks against those biases). In fact, my hunch is that a lot of the one-armed economists have motives other than good social science.

I like to think that we as a profession are generally doing a decent job, but it can’t be said for all of us.

Politicians don’t like two-handed economists. Politicians like clear policy recommendations, preferably ones which entail no costs or possibility of failure. They also like it when experts will lend “credibility” to their policy platforms. Therefore, economists who are willing to provide clear-cut answers to any question while making it all look easy are highly likely to acquire the attention of policymakers. The trouble is that these economists are probably drawing conclusions based on ideology or personal interests rather than careful application of economic science. When politicians hire such ideologically committed experts, they are defeating the purpose of hiring experts in the first place. Policy prescriptions delivered with complete certainty and ideological purity can be obtained on the cheap from cable news; an education in economics is not necessary.

I think this is a significant problem in policy debates. Part of the problem comes from the fact that economists typically use analytical tools which average people don’t understand. This can make it difficult to evaluate the credibility of economists’ arguments and even more difficult to draw conclusions about economic research reported in the media. Therefore, I’ve brainstormed a list of pointers for approaching the economics profession, and the media's version of it, in a policy context. I might need to revise them upon further reflection, but consider them a starting point.

  • Postpone acceptance of papers cited in isolation unless you are familiar with the relevant literature or have the analytical tools to evaluate the papers for yourself. Research on most policy questions is likely to have a history of contradictory findings before consensus is reached. This does not mean there are no policy questions for which a reasonable consensus exists; it simply means that loud voices pointing to economic research in defense of their views should be able to point to evidence of consensus, not a solitary, cherry-picked study. At the very least, they should be able to provide a discussion of competing studies and why they are unsatisfactory. Of course, this rule can be hard to follow for studies on topics with little previous coverage (e.g., studies of the recent recession) or real-time analysis.
  • Even when a consensus around the existence of some policy effect develops, it is not obvious what sort of policy prescriptions should follow. Be cautious about commentators who make the logical leap from well accepted research results to unqualified normative conclusions. That leap is impossible to make without some sort of welfare criterion. Competent, well-read researchers can agree about the findings of a strand of literature and disagree about the policy implications of those findings. As James Zuccollo explains, “few policies are unambiguously good or bad, so almost any economist’s commentary provides ammunition for both sides of the normative debate.”
  • There are precious few policy dilemmas with obvious solutions; and even for these, responsible economists are likely to deliver prescriptions with a complement of caveats and cautions. Steer clear of commentators who promise costless solutions unless they have a long record of carefully making their case (like Scott Sumner).
  • In more general terms, social scientists are most credible when they deliver analyses and recommendations with a large dose of humility. Economist Greg Mankiw writes that, among economics pundits, “certitude reflects bravado more often than true knowledge” (my italics).
  • Ideological dividing lines in the United States exist due to a variety of historical factors which are probably orthogonal to any rigorously founded worldview construction. As Brink Lindsey argues, “There’s no epistemologically sound reason why one’s opinions about, say, the effects of gun control should predict one’s opinion about whether humans have contributed to climate change or how well Mexican immigrants are assimilating.” Therefore, it’s very unlikely that a person with rigorous training in economics would arrive honestly at a set of political preferences which map seamlessly onto the platform of one or the other American political party. Be skeptical of commentators who are loud cheerleaders for mainstream political candidates.
  • Credible economists often do policy work, but they hold their noses while they are asked to be team players. Greg Mankiw is a good example of this: he has worked for several Republicans, but he hasn’t embraced their nonsense about magical tax cuts. He currently works for Mitt Romney, but you can bet that he’s not cheering for the parts of Romney’s fiscal “plan” which require fantastical supply side effects to avoid adding trillions to US debt. Be wary of economists who appear to be team players when they are not employed by candidates or politicians. Odds are they are pining for a policy job or are wearing ideological blinders.
  • Be cautious of commentators who give lip service to opposing views without actually giving air time to their strongest argument (i.e., they fail the ideological Turing test). There’s nothing objective about caricaturing the other guys’ argument then snidely dismissing it as idiotic. This is par for the course in political polemics, but most debates among academic economists are likely to be too substantive for that sort of treatment.
I am not discounting the value of economic research. Indeed, I think it is probably the best guide to policy we have. Don’t misinterpret my comments as implying that economic research is unreliable or that economists are incapable of agreeing on anything. I simply think readers should exercise caution against the possibility that economic research is being abused by pundits and even other economists for ideological or political ends.

Also, I am not suggesting that economists are only credible if they have no political preferences. Rather, I am suggesting that credible economists arrive at political preferences after careful consideration of tradeoffs, reflection on relevant theory, and evaluation of empirical evidence. Having gone through this process, these economists can make policy recommendations which adequately reflect both the process of arriving at a conclusion and the amount of uncertainty associated with the conclusion. If they display little or no uncertainty, write it off as “bravado” instead of “true knowledge,” and hire a new expert.

My personal experience has been that my training in economics has gradually made me less and less certain about what I thought I knew. I am puzzled when I see similar training having the opposite effect on others. My puzzlement is mitigated, though, when I see that those others’ surety is highly correlated with their ex ante political preferences. I think you see the point.

Hayek was right: “The curious task of economics is to demonstrate to men how little they really know about what they imagine they can design.” We are studying a subject about which perfect knowledge and, therefore, perfect policy recommendations are impossible. Economists peddling simple, obvious fixes to policy problems are more likely to be snake-oil salesmen than credible social scientists. With that in mind, beware of economists who self-assuredly cheer on the politicians with their grand designs.


See also Andrew Gelman's comments on economics journalism; Hayek's warnings.