Gullibility and pseudoscience, bridged by headlines
Much have been made in the media recently, of a February 2013 paper, published by a German group in the Annals of Internal Medicine, claiming that acupuncture may help relieve seasonal allergies. Always interested in examining the bold claims of efficacy by various forms of pseudoscientific, wannabe-medicine modalities (such as homeopathy, naturopathy, and so forth), I elected to go to the source; the paper was behind an annoying paywall, but thankfully, I had institutional access, and dove in.
The paper describes a randomized, controlled, multicenter clinical trial of acupuncture in patients suffering from allergic rhinitis associated with the season. Rhinitis is the inflamed condition of the inner nasal passageway, leading to stuffy or runny nose, irritation of the nasal passage and back of throat which induces sneezing and coughing. Rhinitis may be secondary to an allergic response (hence "allergic rhinitis") that people may have to airborne dander or dust, tree or grass pollen, or other allergens, during particular seasons.
Individuals sensitized to these allergens produce certain antibodies to them, called IgE, which in turn binds to special types of immune cells, called Mast cells and Basophils. This leads to release of a substance called histamine, a potent mediator of inflammation and allergic responses. Apart from nasal congestion, eye, ear and throat symptoms are common.
In Western societies, about 10-25% people, many of them children, face the annual occurrence of allergic rhinitis. Ordinarily, the treatments for this condition have long been pretty standardized, geared towards prevention or reduction of symptoms. Pharmacological interventions, in form of steroids and anti-histamine agents, are highly effective - even over-the-counter ones; in extreme cases, desensitization via prolonged immunotherapy may be prescribed, which suppresses IgE formation in the body.
The bottom-line is, the existing therapies work, and people get better, with symptomatic relief. And yet...
... yet, as the authors of the study have noted in the paper, many people seek out unproven, unscientific alternative medicine modalities looking for relief from symptoms. About 1 out of 5 of these patients feel acupuncture is going to help them, regardless of the fact that evidence for actual efficacy or benefit of acupuncture in allergic rhinitis is severely lacking. Score 1 for gullibility.
Taking cognizance of the fact that some people still choose acupuncture for allergic rhinitis, the authors designed a clinical trial to assess the efficacy of this intervention upon two variables, disease-specific quality of life (in other words, whether the patients feel they're getting better), and secondly, the actual need for anti-histamine medication (indicative of the patients' physiological condition). They decided to study the effects of acupuncture along with an antihistaminic, cetirizine, to be used as a rescue medication (one that provides a quick relief when needed); their comparators were sham acupuncture (in which so-called 'real' acupuncture therapy, using 'meridians' and points prescribed by Traditional Chinese Medicine, wouldn't be applied, but needles would be applied randomly) along with cetirizine, and cetirizine alone.
This is a nice design for a controlled trial. However, when applied to Tooth Fairy science, there is a significant problem.
Coined by Harriet Hall, MD, accomplished physician and blogger, Tooth Fairy science refers to a research undertaking on a hypothesis before establishing that the hypothesis is plausible in reality, and/or the phenomenon under study actually occurs/exists. I shall leave you to follow the above link for the definition, but acupuncture is one such phenomenon - its efficacy is one such hypothesis, which has so long never been unequivocally proved empirically, except, perhaps, in the imagination of TrueBelievers™. This is why clinical trials with an unproven modality like acupuncture can never be designed for a head-to-head comparison with existing medical therapies, because of moral and ethical considerations, and obligations of conscientious physicians towards their patients. In this study, all patients took doses of cetirizine. Therefore, the assessment of the outcome would necessarily depend upon the rigorousness of observations and prodigious statistical legerdemain.
422 patients were randomly assigned to the treatment groups (212, acupuncture; 102, sham acupuncture; 108, cetirizine; the actual analysis was done with slightly less number of patients for various reasons, mentioned in figure 2). These were late 20s to late 30s folks, predominantly women, and mostly from Berlin/Brandenburg and Bavaria, all having suffered seasonal allergy-associated rhinitis in the previous year - but mostly (3 out of every 5) of a shorter duration.
There were variations in the acupuncture procedure performed on patients - number of needles used, time of needle retention. Someone who is more versed with these variations can perhaps comment on the possible effects of these variations on the outcome.
Throughout the study, patients, as well as non-physician personnel, were blinded to the treatment assignment, which means that the patients didn't know to begin with what kind of treatment they would receive. However, an important point to be noted is that - as mentioned - about 80% of participants came to take part after reading newspaper articles - placed by the study recruiters - about the use of acupuncture for allergic rhinitis. It is not unlikely that these patients were already favorably inclined towards accessing acupuncture for their treatment, a possible real source of bias. This is particularly significant if one considers that, by the study design, RM (Rescue Medicine) Scores and part of RQLQ (Rhinitis Quality of Life Questionnaire) were assessed by self-reported symptom diaries and answers to questionnaires. The mind boggles at the implication.
The remaining 20% were recruited by physicians at trial centers. It would, therefore, have been interesting to see the total study outcome stratified according to mode of recruitment also.
It is perhaps a reflexion of this fact, that at the time of recruitment, 4 out of every 5 patients started with a high expectation of acupuncture efficacy; in addition, about 20% of them had previous exposure to acupuncture. These are the same folks who ended up finding themselves in the acupuncture group in most numbers (1 in 4 patients in this group had prior acupuncture). This makes me worried about the quality of blinding of the study. According to the authors,
... recipients of real acupuncture rated the question, "How confident do you feel that acupuncture can alleviate your complaint?," higher than did recipients of sham acupuncture (P = 0.016).
This is a statistically significant difference. Since belief in acupuncture was equivalent at the onset of treatment, if the patients were indeed blinded to the intervention, why would the sham group show a lower confidence in acupuncture than those in the real group? The authors don't explain this further, beyond mentioning a possible unblinding en passim in the discussion, and indicating that "... effects of acupuncture compared with RM in this study might have heen affected by patient beliefs about acupuncture."
In the results, the authors have stated that the RQLQ and RM Scores decreased more in the acupuncture group than in the controls. However, in the longer term (end of fourth month and in the second year), scores in all the groups pretty much even out. In the cetirizine group, these later observations are, of course, confounded by the inexplicable study design of giving acupuncture sessions to cetirizine-taking patients. It was almost as if the study designers were determined to show an effect of acupuncture anywhichway.
It also doesn't explain why all the scores seem to return to the baseline or higher in the second year, per figure 3.
Image ©The American College of Physicians; Figure adapted from Brinkhaus et al., Ann Intern Med. 2013;158(4):225-234. doi:10.7326/0003-4819-158-4-201302190-00002
The problem with any of such subjective scoring system is always the assessment of the biological/clinical significance, particularly in absence of mechanistic explanations. What does a decrease of 0.3, 1.5 or 2 points mean in absolute or relative terms? In microbiology, for example, any drug or intervention that reduces the number of bacteria by 3 logs (≥1000-fold) is considered clinically significant. What do these scores mean in terms of actual effect on the physiology of the patient? How does acupuncture address the primary cause of the symptoms of allergic rhinitis, the local inflammatory response and the inflammatory mediators running amok?
Such is the nature of Tooth Fairy science.
To their credit, the authors do comment in the discussion somewhat on this.
...CIs surrounding the estimates of improvement included values that were less than predefined thresholds for clinically important differences, so the clinical significance of the findings is uncertain. There were no between-group differences in responses at 16 weeks, and acupuncture led to greater improvements than sham acupuncture (but not RM) after the 8-week follow-up phase in the second year, with CIs that again included differences of uncertain clinical significance. [All emphases, mine]
In the discussion, the authors mention a few studies, mostly from one group (one of which, Dr. Pfab, is a co-author in this paper) and a review, which have attempted mechanistic explanations for the effects of acupuncture. However, the mechanisms that these papers have expounded on relate to itch, which until recently was thought to be related to pain receptors (but may actually involve a distinct neural circuit altogether). The pathways are not the same as those used by allergic rhinitis, and unlike the latter, antihistamines don't work on most forms of itch.
So, no... Still no mechanism for the hypothesized beneficial action of acupuncture on allergic rhinitis. But that does not prevent the author from claiming benefit as such. Score 1 for pseudoscience.
Now the headlines. My attention to this study was drawn yesterday via the Twitter feed of an Indian TV News channel, Zee News; aware of the influence of several pseudoscientific modalities in Indian healthcare, I thought, perhaps in a newfound spirit of regional co-operation, India was embracing TCM as well; the more, the merrier. I was wrong. Zee News, or more precisely, its "Health" News (scarequotes are fully intended) section, was merely reporting on the German study, quoting Fox News.
In order to gauge the Media response, I decided to engage in a Google Search with the study citation as a search term. Not unexpectedly, the media headlines turning in the search were almost all uncritical statements designed to highlight an impression of benefit, and the actual reports therein were accordingly vacuous. Here is a sample of my search results, aggregated from first 3 pages:
There was a single notable exception, Medscape Medical News - much plaudits to them for the appropriately critical assessment. The question is: can this poor quality of general headlining and reporting be explained away by the much-discussed difference between scientific report and journalism?