Thursday, September 24, 2015

shortcomings of mathematical modeling

Over the course of the twentieth century, accelerating in the second half, the discipline of economics became increasingly mathematical, to the chagrin of some people.  I myself think it has gone too far in that direction in many ways, but I feel like some of the critiques aren't exactly on point.

One of the benefits — perhaps most of the benefit — of mathematical modeling is that it forces a precision that is often easier to avoid in purely verbal arguments. This precision in certain contexts allows one to make deductions about the behavior of the model that goes beyond what is intuitive, for better or worse — most of the time it will ultimately drag intuition along with it.  Certainly if you want a computer to simulate your model, it needs to be precise enough for the computer to simulate it.  Further, any model in which forces are counteracting each other in interesting ways is going to have to be quantitative to some degree to be useful; if you want to know what effect some shock will have on the price of a good, and the shock increases both supply and demand for the good, if you don't have enough detail in your model to know which effect is bigger, you can't even tell whether the price will go up or down.

I think there are basically four problems one encounters with the mathematization, or perhaps four facets of a single problem; all of them are to varying degrees potential problems with verbal arguments as well, but they seem to affect mathematical arguments differently.
It is tempting to use a model that is easy to understand rather than one that is correct.
All models are wrong, but some models are useful; if you're studying something interesting, it is probably too complex to understand in full detail, and an economic model that requires predicting who is going to buy a donut on a given day is doomed to failure on both fronts. The goal in producing a model (mathematical or otherwise) is to capture the important effects without making the model more complicated than necessary to do so.  There are sometimes cases in which models that are too complex are put forth, but the cost there tends to be obvious; a model that is too complex won't shed much light on the phenomena being studied.  The other error — leaving out details that are important but hard to include in your model — is more problematic, in that it can leave you with a model that can be understood and invites you to believe it tells you more about the real world than it really does.
It can be ambiguous how models line up with reality.
I've discussed here before the shortcomings of GDP as a welfare measure, and have elsewhere a fuller discussion of related measures of production, welfare, and economic activity; most macroeconomic models will have only a single variable that represents something like "aggregate output", and when people try to compare their models to data they almost always identify that as "GDP", which is almost always, I think, wrong; one of the proximate triggers for this post was a discussion of an inflation model that made this identification where Net Domestic Product was probably the better measure, and if you're comparing data from the seventies to data today — before and after much "fixed investment" became computers and software instead of longer duration assets — one isn't necessarily a particularly good proxy for the other. Similarly, models will tend to have "an interest rate", "an inflation rate", etc., and it's not clear whether you should use T-bills, LIBOR, or even seasoned corporate bonds for the former or whether you should use the PCE price index, the GDP deflator, or something else for the latter.
One can write models that leave out important considerations.
One of the principles of good argumentation — designed to seek the truth rather than score points — is that one should address one's opponents' main counterarguments. This is as true for mathematical arguments as for verbal ones. I occasionally see a paper on some topic that is the subject of active public policy debate in which the author says, "to answer the question, we built a model and evaluated the impact of different policies," and the model simply excludes the factor at the heart of the arguments of one of the two camps. Any useful model is a simplification of reality, but a useful model will necessarily include any factors that are important, and an argument (mathematical or verbal) that ignores a major counterargument (mathematical or verbal) should not be taken to be convincing.
Initiates and noninitiates, for different reasons, may give models excessive credence.
People who don't deeply understand models sometimes accept models that make their presenters look smart. I like to think that most of the people who produce mathematical models understand their limitations, but there is certainly a tendency in certain cases for people who have a way of understanding the world to lean too heavily on it, and there is a real tendency in academia in particular for people who have extensively studied some narrow set of phenomena to think of themselves as experts on far broader matters.
As I noted, these problems can be present to some degree even in non-mathematical arguments; certainly Keynes talked about "interest rates" without always specifying which ones, he made tacit assumptions that were crucial to his predictions and weren't well spelled out, and he seems to have been very confident about predictions that haven't always panned out, all without mathematics.  (To the extent that we learn mathematical "Keynesian" economics in introductory macroeconomics, the math was largely introduced by Hicks a few years later attempting to make Keynes's arguments clearer and more precise.)

Ultimately, it may be that the best argument against excessive math in economics is that is has sometimes crowded out other ways of thinking; having some mathematical papers that are related to economics is a good thing, but if papers that do mathematics that is far removed from economics are displacing economic arguments that are hard to put in mathematical terms, then the discipline has moved well past the optimum, which almost certainly has a diversity of approaches.

No comments: