Bad Science (4 page)

And for all homeopaths’ talk about the “memory of water,” we should remember that what you actually take, in general, is a little sugar pill, not a teaspoon of homeopathically diluted water, so they should start thinking about the memory of sugar too. The memory of sugar, which is remembering something that was being remembered by water (after a dilution greater than the number of atoms in the universe) but then got passed on to the sugar as it dried. I’m trying to be clear, because I don’t want any more complaints.

Once this sugar, which has remembered something the water was remembering, gets into your body, it must have some kind of effect. What would that be? Nobody knows, but you need to take the pills regularly, apparently, in a dosing regime that is suspiciously similar to that for medical drugs (which are given at intervals spaced according to how fast they are broken down and excreted by your body).

I Demand a Fair Trial

These theoretical improbabilities are interesting, but they’re not going to win you any arguments: Sir John Forbes, physician to Queen Victoria, pointed out the dilution problem in the nineteenth century, and 150 years later the discussion has not moved on. The real question with homeopathy is very simple: Does it work? In fact, how do we know if
any
given treatment is working?

Symptoms are a very subjective thing, so almost every conceivable way of establishing the benefits of any treatment must start with the individual and his or her experience, building from there. Let’s imagine we’re talking—maybe even arguing—with someone who thinks that homeopathy works, someone who feels it is a positive experience, and who feels he gets better, quicker, with homeopathy. They would say: “All I know is, I feel as if it works. I get better when I take homeopathy.” It seems obvious to them, and to an extent it is. This statement’s power, and its flaws, lie in its simplicity. Whatever happens, the statement stands as true.

But you could pop up and say: “Well, perhaps that was the placebo effect.” Because the placebo effect is far more complex and interesting than most people suspect, going way beyond a mere sugar pill; it’s about the whole cultural experience of a treatment, your expectations beforehand, the consultation process you go through while receiving the treatment, and much more.

We know that two sugar pills are a more effective treatment than one sugar pill, for example, and we know that saltwater injections are a more effective treatment for pain than sugar pills, not because saltwater injections have any biological action on the body, but because an injection feels like a more dramatic intervention. We know that the color of pills, their packaging, how much you pay for them, and even the beliefs of the people handing the pills over are all important factors. We know that placebo operations can be effective for knee pain and even for chest pain. The placebo effect works on animals and children. It is highly potent, and very sneaky, and you won’t know the half of it until you read the placebo chapter in this book.

So when our homeopathy fan says that homeopathic treatment makes them feel better, we might reply: “I accept that, but perhaps your improvement is because of the placebo effect,” and they cannot answer no, because they have
no possible way of knowing
whether they got better through the placebo effect or not. They cannot tell. The most they can do is restate, in response to your query, their original statement: “All I know is, I feel as if it works. I get better when I take homeopathy.”

Next, you might say: “OK, I accept that, but perhaps, also, you feel you’re getting better because of ‘regression to the mean.’” This is just one of the many “cognitive illusions” described in this book, the basic flaws in our reasoning apparatus that lead us to see patterns and connections in the world around us, when closer inspection reveals that in fact, there are none.

“Regression to the mean” is basically another phrase for the phenomenon whereby, as alternative therapists like to say, all things have a natural cycle. Let’s say you have back pain. It comes and goes. You have good days and bad days, good weeks and bad weeks. When it’s at its very worst, it’s going to get better, because that’s the way things are with your back pain.

Similarly, many illnesses have what is called a natural history: they are bad, and then they get better. As Voltaire said, “The art of medicine consists in amusing the patient while nature cures the disease.” Let’s say you have a cold. It’s going to get better after a few days, but at the moment you feel miserable. It’s quite natural that when your symptoms are at their very worst, you will do things to try to get better. You might take a homeopathic remedy. You might sacrifice a goat and dangle its entrails around your neck. You might bully your physician into giving you antibiotics. (I’ve listed these in order of increasing ridiculousness.)

Then, when you get better—as you surely will from a cold—you will naturally assume that whatever you did when your symptoms were at their worst must be the reason for your recovery.
Post hoc, ergo propter hoc
, and all that. Every time you get a cold from now on, you’ll be back at your physician, hassling her for antibiotics, and she’ll be saying, “Look, I don’t think this is a very good idea,” but you’ll insist, because they worked last time, and community antibiotic resistance will increase, and ultimately old ladies die from multiple-drug-resistant bacteria, because of this kind of irrationality, but that’s another story.
⁶

You can look at regression to the mean more mathematically, if you prefer. On
Card Sharks
, when the host puts a three on the board, the audience all shout, “Higher!” because they know the odds are that the next card is going to be higher than a three. “Do you want to go higher or lower than a jack? Higher? Higher?” “Lower!”

An even more extreme version of regression to the mean is what is known as the
Sports Illustrated
jinx. Whenever a sportsman appears on the cover of
Sports Illustrated
, goes the story, he is soon to fall from grace. But to get on the cover of the magazine, you have to be at the absolute top of your game, one of the best sportsmen in the world, and to be the best in that week, you’re probably also having an unusual run of luck. Luck, or “noise,” generally passes; it “regresses to the mean” by itself, as happens with throws of a die. If you fail to understand that, you start looking for another cause for that regression, and you find…the
Sports Illustrated
jinx.

Homeopaths increase the odds of a perceived success in their treatments even further by talking about aggravations, explaining that sometimes the correct remedy can make symptoms get worse before they get better, and claiming that this is part of the treatment process. Similarly, people flogging detox will often say that their remedies might make you feel worse at first, as the toxins are extruded from your body; under the terms of these promises, literally anything that happens to you after a treatment is proof of the therapist’s clinical acumen and prescribing skill.

So we could go back to our homeopathy fan and say: “You feel you get better, I accept that. But perhaps it is because of regression to the mean, or simply the natural history of the disease.” Again, he cannot say no (or at least not with any meaning—he might say it in a tantrum), because he has no possible way of knowing whether he was going to get better anyway on the occasions when he apparently got better after seeing a homeopath. Regression to the mean might well be the true explanation for his return to health. He simply cannot tell. He can only restate, again, his original statement: “All I know is, I feel as if it works. I get better when I take homeopathy.”

That may be as far as he wants to go. But when someone goes further and says, “Homeopathy works,” or mutters about “science,” then that’s a problem. We cannot simply decide such things on the basis of one individual’s experiences, for the reasons described above: they might be mistaking the placebo effect for a real effect or mistaking a chance finding for a real one. Even if we had one genuine, unambiguous, and astonishing case of a person’s getting better from terminal cancer, we’d still be careful about using that one person’s experience, because sometimes, entirely by chance, miracles really do happen. Sometimes, but not very often.

Over the course of many years, a team of Australian oncologists followed 2,337 terminal cancer patients in palliative care. They died, on average, after five months. But around 1 percent of them were still alive after five years. In January 2006 this study was reported in
The Independent
newspaper in the U.K., bafflingly, as:

But the point of the study was specifically
not
that there are miracle cures (it didn’t look at any such treatments; that was an invention by the newspaper). Instead it showed something much more interesting: that amazing things simply happen sometimes; people can survive, despite all the odds, for no apparent reason. As the researchers made clear in their own description, claims for miracle cures should be treated with caution, because “miracles” occur routinely, in 1 percent of cases by their definition, and
without
any specific intervention. The lesson of this paper is that we cannot reason from one individual’s experience or even that of a handful, selected out to make a point.

So how do we move on? The answer is that we take lots of individuals, a sample of patients who represent the people we hope to treat, with all of their individual experiences, and count them all up. This is clinical academic medical research, in a nutshell, and there’s really nothing more to it than that: no mystery, no “different paradigm,” no smoke and mirrors. It’s an entirely transparent process, and this one idea has probably saved more lives, on a more spectacular scale, than any other idea you will come across this year.

It is also not a new idea. The first trial appears in the Old Testament, and interestingly, although nutritionism has only recently become what we might call the bullshit du jour, it was about food. Daniel was arguing with King Nebuchadnezzar’s chief eunuch over the Judaean captives’ rations. Their diet was rich food and wine, but Daniel wanted his own soldiers to be given only vegetables. The eunuch was worried that they would become worse soldiers if they didn’t eat their rich meals, and that whatever could be done to a eunuch to make his life worse might be done to him. Daniel, on the other hand, was willing to compromise, so he suggested the first ever clinical trial:

To an extent, that’s all there is to it; there’s nothing particularly mysterious about a trial, and if we wanted to see whether homeopathy pills work, we could do a very similar trial. Let’s flesh it out. We would take, say, two hundred people going to a homeopathy clinic, divide them randomly into two groups, and let them go through the whole process of seeing the homeopath, being diagnosed, and getting their prescription for whatever the homeopath wants to give them. But at the last minute, without their knowledge, we would switch half of the patients’ homeopathic sugar pills, giving them dud sugar pills, that have not been magically “potentized” by homeopathy. Then, at an appropriate time later, we could measure how many in each group got better.

Speaking with homeopaths, I have encountered a great deal of angst about the idea of measuring, as if this were somehow not a transparent process, as if it were forcing a square peg into a round hole, because “measuring” sounds scientific and mathematical. We should pause for just a moment and think about this clearly. Measuring involves no mystery and no special devices. We ask people if they feel better and count up the answers.

In a trial—or sometimes routinely in outpatients’ clinic—we might ask people to measure their knee pain on a scale of one to ten every day, in a diary. Or to count up the number of pain-free days in a week. Or to measure the effect their fatigue has had on their lives that week: how many days they’ve been able to get out of the house, how far they’ve been able to walk, how much housework they’ve been able to do. You can ask about any number of very simple, transparent, and often quite subjective things, because the business of medicine is improving lives and ameliorating distress.

We might dress the process up a bit, to standardize it, and allow our results to be compared more easily with other research (which is a good thing, as it helps us get a broader understanding of a condition and its treatment). We might use the General Health Questionnaire, for example, because it’s a standardized “tool,” but for all the bluster, the GHQ-12, as it is known, is just a simple list of questions about your life and your symptoms.

If antiauthoritarian rhetoric is your thing, then bear this in mind: perpetrating a placebo-controlled trial of an accepted treatment—whether it’s an alternative therapy or any form of medicine—is an inherently subversive act. You undermine false certainty, and you deprive doctors, patients, and therapists of treatments that previously pleased them.