Politics of Evidence
Curious, I read the Pain® journal article by Edzard Ernst et al: Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews (1). Their conclusion: “Although acupuncture is commonly used for pain control, doubts about its effectiveness and safety remain.” And, that, “Acupuncture remains associated with serious adverse effects. One might argue that, in view of the popularity of acupuncture, the number of serious adverse effects is minute. We would counter, however, that even one avoidable adverse event is one too many.”
The reviewed data were collected internationally and represent ten years of studies, identifying only 38 cases of infection, 42 traumas, 13 adverse effects and five deaths claimed to be related to acupuncture. Ernst et al. listed 165 references. 32 seemed to be systematic reviews. Most are anecdotal case reports of complications. There were 17 negative reviews between 1989-2000 and 57 positive later reviews.
Safety in Medicine
Ernst et al. did not examine iatrogenic deaths due to conventional drugs, chemotherapy, radiotherapy and surgery. They focused on the safety and efficacy of acupuncture. Ernst is chair of Complimentary and Alternative Medicine at Exiter, so while it would be interesting to see results an in-depth ‘review of reviews’ on the safety and efficacy of conventional medicine and surgery, that is not likely the scope and mission of the department.
Consider the 41,000 ulcers and 3,300 deaths caused by nonsteroidal anti-inflammatory drugs in one year (2). Also, the 16,000 injuries from auto crashes each year connected with psychoactive drugs such as benzodiazepines and tricyclic antidepressants (3). The magnitude of drug related deaths internationally over a 10-year period, while not their topic, would provide context. This is not tu quoque – erroneous logic, based upon the argument that: “If you did it, so can I”. Rather, I concur that avoidable injuries and death are unacceptable.
To their credit Ernst et al. recommend, “The key to making progress would be to train all acupuncturists to a high level of competency.” Ernst et al. called it malpractice rather than any real risk of acupuncture. I couldn’t agree more. Abbreviated acupuncture programs pose a risk to the patient and to society.
Rogers reached a similar conclusion 30 years earlier, “It is shown that serious complications can arise only from the improper use of acupuncture or from its gross abuse by incompetent practitioners. The safety of a technique must be judged on its results in the hands of competent practitioners who use it properly. If this criterion is accepted, acupuncture is seen as an extremely safe therapeutic system whose complications are very rare and are easily avoided or rectified” (4).
Ernst et al. do not reference previous studies showing acupuncture as safe. Take for instance, the 2003 article by Brian Berman and Lixing Lao, Is Acupuncture Safe a Systematic Review of Case Reports (5). That article clearly demonstrates that acupuncture is a safe practice. Further, Edzard doesn’t identify other complications such as fainting, vomiting or bruising, which are the more common side effects seen with this intervention. More importantly, the number of adverse events per treatment is not stated. In order to assess risk, one must know the number of times acupuncture was performed during the studies. Edzard does not provide the number of patients, treatments, or needles inserted. Without that, no conclusion can be drawn on safety.
The methods section also is weak and lacking in detail. For data on efficacy, he doesn’t provide information about how the studies were controlled. Was it acupuncture alone, skin penetrating, non-penetrating or adjacent to the point location? I don’t see any of that here. It leaves the reader wanting to know more about the studies that he used.
In systematic reviews, there is also a danger of pulling information without getting the context of the meaning. These authors seem to pull sentences, describing study findings, out of the larger context of the reviews. Further, lack of detail as to how Ernst et al. quantified the quality of the studies brings into question the validity of their conclusion.
Ernst et al. continue: “In many of the case reports, causality was uncertain… not least because of a lack of sufficient detail. Yet, most of the authors seemed confident about causality.” It is not clear how a conclusion can be drawn from conflicted propositions such as these.
The field has standards such as STRICTA (Standards for Reporting Interventions in Controlled Trials of Acupuncture) (9). Further, a new tool has been developed by Hammerschlag et al. that assesses reporting quality and combines STRICTA and CONSORT (Consolidated Standards of Reporting Trials) (10). Enrnst et al. have cast the net wide and have included an impressive number of citations. It would be useful to have them more focused in terms of time frames and qualifications.
It appears that Ernst et al. do not provide a secure theoretical or evidential basis for their argument, and have used analytical tools inadequate to achieve objective and reliable conclusions. Their argument relies heavily on preconceptions, variable definitions and opinion, a problem exacerbated by a tendency to confirmation bias on the authors’ part. In their attempt to link acupuncture to adverse events, they included hearsay and unverified data (“most of the authors seemed confident about causality”). Pain can be treated effectively with acupuncture and at a lower cost of care.
Conclusion? Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews: Are the papers conclusions valid? There remains serious question given the inconsistencies in the document. We must as the question, what is the objective for Pain®, regarded as a prestigious and impartial journal, in publishing this article.
1. Ernst E, Myeong SL, Choi TY. Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews. PAIN®. 2011;152(4).
2. Ray WA GM, Shorr RI. Adverse drug reactions and the elderly. Health Affairs 1990;9:114-22.
3. Ray WA FR, Decker MD. Psychoactive drugs and the risk of injurious motor vehicle crashes in elderly drivers. . American Journal of Epidemiology. 1992;136:873-83.
4. Rogers P. Serious Complications of Acupuncture … Or Acupuncture Abuses? (An edited version of the original in the American Journal of Acupuncture Oct-Dec 1981: 9(4); 347-351). 1998.
5. Lao L HG, Fu J, Berman BM. Is acupuncture safe? A systematic review of case reports. Altern Ther Health Med 2003. 2003;9(1):72-83.
6. H. M. Langevin PMW, H. MacPherson, R. Schnyer, R. M. Milley VN, L. Lao, J. Park, R. E. Harris,, M. Cohen KJS, A. Haramati, and R. Hammerschlag. Paradoxes in Acupuncture Research: Strategies for Moving Forward. Evidence-Based Complementary and Alternative Medicine. 2011.
7. Kaptchuk TJ, Shaw J, Kerr CE, Conboy LA, Kelley JM, Csordas TJ, et al. “Maybe I made up the whole thing”: placebos and patients’ experiences in a randomized controlled trial. Cult Med Psychiatry. 2009 Sep;33(3):382-411.
8. Janssens L, Altman S, Rogers P. Respiratory and cardiac arrest under general anaesthesia: treatment by AP of the nasal philtrum. Vet Rec Sep 22 1979;105(12):273-6.
9. Hugh MacPherson AW, Mike Cummings, Kim Jobst, Ken Rose, Richard Niemtzow, for the STRICTA Group*. Standards for Reporting Interventions in Controlled Trials of Acupuncture: The STRICTA Recommendations. Acupuncture in Medicine. 2002 March 2002;VOL 20(1):Page 22 – 5.
10. Hammerschlag R, Milley R, Colbert A, Weih J, Yohalem-Ilsley B, Mist S, et al. Randomized Controlled Trials of Acupuncture (1997–2007): An Assessment of Reporting Quality with a CONSORT- and STRICTA-Based Instrument. Evidence-Based Complementary and Alternative Medicine. 2011:25.