Thursday, December 18, 2014

medical testing, economics, and the placebo effect

One of the central lessons of introductory microeconomics is that in evaluating an option, there isn't really a relevant absolute standard of "good" or "bad", or (depending on how you want to think about it) that the only baseline for that standard is the best or most likely alternative.  If you have a chance to spend the afternoon bowling or going to the zoo, these might, in a colloquial sense, both seem like "good" options, and if you're trying to decide whether to spend a lot of money getting a car fixed or having it junked, these might seem like "bad" options, but from the standpoint of rational decision-making you're only concerned with whether bowling is better than going to the zoo or going to the zoo is better than bowling — the term "cost", in its most complete sense as economists understand it, is about what opportunities you give up by making the choice you're making, such that, by construction, only one of a mutually exclusive set of actions can be "worth more than it costs".

The FDA requires, at least for certain kinds of drugs and medical devices and procedures, that it be established that they are better than previous standards of care, at least for some patients in some circumstances, and that seems to be consistent with this principle, but that kind of testing tends to come later in the human subject trials than a better known and even more common kind of medical trial, which is testing for bare "efficacy".  Some subjects are given the treatment, and typically others are given a "placebo" treatment, and the results are compared; if there is no difference, the new treatment is deemed not to be efficacious.  This is wrong for a couple related reasons[1]: it may, indeed, be efficacious because of the placebo effect. On a more practical level, and drawing from the economic principle, the resulting information only provides information about whether the treatment should be given to patients if (and to the extent that) they would otherwise be likely to be given the placebo.

I want to emphasize — this seems to me to be the sort of objection someone might raise to what I'm proposing — that anyone lie to a patient here. "This treatment is the standard of care, has been shown to have some effect, and is at least as effective as anything else we know of [with given side effects, or whatever qualifiers are appropriate]," if it is true, is all true, and is all that is relevant to the patient. Indeed, if the patient is not self-sabotaging, you can say "this seems to work, to the extent we can tell, because of the placebo effect." Indeed, I drink orange juice and consume large amounts of vitamin C when I have a cold for this reason; my understanding of the state of scientific knowledge is that 1) vitamin C, for almost all populations,[2] has no more (or less!) effect on the common cold than placebo, and that 2) the common cold is unusually susceptible to the placebo effect.[3] People with whom I have discussed this seem to, at least initially, think I am somehow "fooling" myself, but consider it from a neutral standpoint:
  • If you believe it will work, it will work.
  • If you believe it will not work, it will not work.
My belief is not "foolish"; it is vindicated. Whether the mechanism involves direct action on the immune system or some convoluted, poorly understood neural pathways is only relevant if I'm likely to sabotage the neural-pathway mechanism should someone tell me that it is heavily involved.[4]

What I am proposing, then, is that testing from the beginning be done against the best standard of care; if there is no treatment, then perhaps include three groups in your testing, with a true control group (which receives no treatment) in addition to the placebo group and the test group. If your placebo group ends up having the best balance of effects, side effects, and costs, though, you should also consider making that placebo the standard of care.[5]


[1] Even leaving aside the statistical power of the test, which is surely relevant to medical decision-making but is not the focus of my interest for this post.

[2] One of the populations for which it seems to have some effect is long-distance runners, a group that used to include me. I don't know whether a mechanism has been determined; some scientifically-informed people I know think it's a bit amazing that the idea that vitamin C boosts your immune system ever got off the ground — it mostly helps maintain connective tissue — and so my not-terribly-well justified hypothesis is that long-distance running puts enough strain on your connective tissue that it diverts resources from your immune system to heal your connective tissue, and that higher doses than are usually of use to people may help free up the immune resources of long-distance runners. As I say, the mechanism I propose here is basically something I made up, but the fact of its having an effect is not.

[3] This latter point has been offered as an explanation for the prevalence of folk remedies for the common cold; they all work for the people who believe they work.

[4] Sort of. Certainly improvements on treatments are likely to be triggered by an understanding of the mechanisms; there is also a suite of issues, in the real world, related to the finite power of statistical testing. The mechanisms may give hints to likely drug interactions or long-term effects for which testing is difficult (because there are so many things to test for or because it would risk withholding an effective drug from a generation of potential beneficiaries, respectively). There is also an issue that computer scientists in machine learning call "regularization"; it is related to things like publication bias, and in this case boils down to the idea that understanding the mechanism might help us shade the experimental results, providing a bit more of a hint as to whether a 2σ effect is more likely or less likely than usual to just be noise. This is also related to the base rate problem; essentially, if the mechanism is such that we strongly expect not to see an effect, then a 2σ effect is probably noise (though a 6σ effect perhaps still means we were wrong). These factors all run parallel to my main point, as understanding the mechanism is also useful for a drug that outperforms a placebo than one that, by all indications, is a placebo.

[5] It seems, further, that even two "drugs" that only give their desired effect through the placebo effect are not necessarily ipso facto interchangeable. I have heard that the color of a pill can have an effect on how effective the patient expects it to be; this may be a problem if you're trying to decide which placebo to test against to decide "efficacy", but if you aren't prejudiced against "placebos", the rule that you go with whatever works regardless of mechanism carries over directly: use the best placebo if that beats everything else on effectiveness/cost/side-effects etc., and use something else if it does. (If the color of the pill affects its effectiveness, that is of course something the drug designers should exploit, but red drug, blue drug, red placebo, blue placebo, and no treatment should start on the same footing.)

No comments: