Some of the criticisms of mathematical modeling in economics apply equally to narrative or heuristic modeling, such as this one by Arnold Kling. He says,
Macroeconomic equations are not proven and tested. They are instead tentative and speculative.
Yep. And this is also true of any other approach to macroeconomics (more below). So Kling isn't helping us sort out the relative merits of the two approaches. He's usually much more insightful. Another offering is from Noah Smith; of his grad school macro classes, he says:
We just assumed a bunch of equations and wrote them down. Then we threw them all together, got some kind of answer or result, and compared the result to some subset of real-world stuff that we had decided we were going to "explain".
This is not fair to typical macro modeling, which is not just throwing things together but rather consists of starting with a few assumptions about behavior and the structure of markets then drawing them out to a conclusion. That's no different from narrative or heuristic theorizing, except that in math everyone can clearly see the assumptions and proposed mechanisms so we can fight about them. (By the way, most of the Smith post is good; it's more about the difference between math in physics and econ rather than simply a criticism of econ math). I very much doubt that Smith's graduate macro professors were such poor teachers that the whole process looked like throwing a bunch of silly equations together.
Bryan Caplan has weighed in as well. He makes a false distinction between economic intuition and math, since each can usefully inform the other. He also focuses too much on explaining economics and not enough on developing new insights. Yes, a lot of insights provided by math can be explained without it; every good paper follows its math section with a narrative explaining the intuition derived from the math. This is part of the reason for Caplan's observation that many economists skip the theory section of papers and just read the conclusions; the conclusions typically expand the insights from the math into words. That doesn't mean the math was unnecessary. So Caplan's empirical test of this debate isn't enough; of course he can explain Krugman's math discoveries in words! That's what makes them great (though I still think that a few equations can profitably replace a lot of words when explaining economics--go look at an intertemporal first-order condition, as Claudia Sahm notes, then write it down in words, and see which is simpler to study). The other question is whether math facilitates new insights that would have required a lot more work if done in a heuristic or narrative fashion; and the answer to that question probably depends on the person doing the research (math has worked well for me). Caplan also assumes too much by calling math a cost/benefit failure; the costs of acquiring math skills probably depend on the individual, and in any case most macro papers can be understood if you have a basic grasp of differentiation and can conceptualize market clearing. The latter skill should be possessed even by economists who shun math. And is ability to take a derivative really that costly to obtain, given that it can then be used to read thousands of papers over the course of a career?
Also, as I've noted before, most people think in partial equilibrium. In macro, we often need general equilibrium, which is a concise way of saying that your theory should try to account for feedback effects and aggregate resource constraints. For me, at least, that's much easier to do with math than without it. Maybe keeping track of it all using words is a piece of cake for others, but I doubt it.
A lot of people seem to be deluding themselves into thinking that narrative/heuristic economics can be done without a lot of assumptions. You're lying to yourself if you think that. Macroeconomics is really, really difficult. You can't do it without simplifying assumptions. If you think you can, you're either arrogant or naive. Show me a narrative essay that accounts for all the heterogeneity and cognitive biases and informational issues and idiosyncratic market structures and time variation and legal framework that exists in the real world.
Using math is an act of intellectual humility: I admit that I cannot keep track of a lot of moving parts in my head, and I'm willing to subject my reasoning to the scorn of others, in a decent-sized package, so that my model can be easily criticized. That doesn't mean that discarding math is arrogant in and of itself, but it does mean that discarding math may force others to wade through a big narrative to isolate your assumptions and shortcuts. And that can be very costly.
If your main criticism of math is that it requires lots of assumptions, it's time for you to abandon economics; you can't do it without assumptions that are often unrealistic. And the thing about math is that it provides a really nice way to think about what happens when assumptions are relaxed one-by-one.
My bigger question is, why can't we all just get along? Does Caplan really think that the people who use math have Stockholm Syndrome? Is he really unwilling to consider the possibility that a lot of economists have found math to be really useful for providing the insights they need, and that those insights would have been costlier without the intellectual leverage and simplicity that math can provide? Why can't he just say that he doesn't find math useful, but that others do, so we should allow for diversity of approaches?
Yes, a lot of the math out there is just showing off; but the other side of that coin is that maybe some of the anti-math sentiment we see is driven by the complainers' unwillingness to learn the few math tools they would need to read papers [edit: I'm not suggesting that Caplan, Smith, or Kling fall into this category]. I see no reason that their laziness should be used to guilt math users into abandoning a useful tool.
If you find narrative style useful, do that. If you find math useful, do that (and explain your math afterwards). Don't assume your approach is without flaws, and don't pretend that your learning style is appropriate for everyone.
* I have to note that Caplan's observation about math for undergrads is myopic; my undergrad institution used math aplenty--here's a taste of what they're up to now.