The aim of a new meta-analysis published today in Acupuncture in Medicine was to estimate the clinical effectiveness and safety of acupuncture for amnestic mild cognitive impairment (AMCI), the transitional stage between the normal memory loss of ageing and dementia.

Note: Readers may also be interested in the other Mental Elf blogs on mild cognitive impairment.

Methods

Randomised controlled trials (RCTs) of acupuncture versus medical treatment for AMCI were identified using six electronic databases.

Five RCTs involving a total of 568 subjects were included. The methodological quality of the RCTs was generally poor.

Results

Participants receiving acupuncture had better outcomes than those receiving nimodipine with greater:

• Clinical efficacy rates (odds ratio (OR) 1.78, 95% CI 1.19 to 2.65; p<0.01),
• Mini-mental state examination (MMSE) scores (mean difference (MD) 0.99, 95% CI 0.71 to 1.28; p<0.01), and
• Picture recognition score (MD 2.12, 95% CI 1.48 to 2.75; p<0.01).

Acupuncture used in conjunction with nimodipine significantly improved MMSE scores (MD 1.09, 95% CI 0.29 to 1.89; p<0.01) compared to nimodipine alone.

Three trials reported adverse events.

Conclusion

The authors concluded that:

Acupuncture appears effective for AMCI when used as an alternative or adjunctive treatment; however, caution must be exercised given the low methodological quality of included trials. Further, more rigorously designed studies are needed.

Commentary

Meta-analyses like this one are, in my view, perfect examples of the ‘rubbish in, rubbish out’ principle of systematic reviews. This may seem like an unfair statement, so let me justify it by explaining the shortfalls of this specific paper.

RCTs cannot tell us if acupuncture is safe

The authors try to tell us that their aim was “to estimate the clinical effectiveness and safety of acupuncture…” While it might be possible to estimate the effectiveness of a therapy by pooling the data of a few RCTs, it is never possible to estimate its safety on such a basis.

To conduct an assessment of therapeutic safety, one would need sample sizes that go two or three dimensions beyond those of RCTs. Thus safety assessments are best done by evaluating the evidence from all the available evidence, including case-reports, epidemiological investigations and observational studies.

The authors tell us that “two studies did not report whether any adverse events or side effects had occurred in the experimental or control groups.” This is a common and serious flaw of many acupuncture trials, and another important reason why RCTs cannot be used for evaluating the risks of acupuncture. Too many such studies simply don’t mention adverse effects at all. If they are then submitted to systematic reviews, they generate a false positive picture about the safety of acupuncture.

The absence of adverse effects reporting is a serious breach of research ethics. In the realm of acupuncture, it is so common, that many reviewers do not even bother to discuss this violation of medical ethics as a major issue.

Acupuncture: totally devoid of therapeutic effects

The authors conclude that acupuncture is more effective than nimodipine. This sounds impressive – unless you happen to know that nimodipine is not supported by good evidence either. A Cochrane review provided “no convincing evidence that nimodipine is a useful treatment for the symptoms of dementia, either unclassified or according to the major subtypes – Alzheimer’s disease, vascular, or mixed Alzheimer’s and vascular dementia” (Birks et al, 2002).

The authors also conclude that acupuncture used in conjunction with nimodipine is better than nimodipine alone. This too might sound impressive – unless you realise that all the RCTs in question failed to control for the effects of placebo and the added attention given to the patients. This means that the findings reported here are consistent with acupuncture itself being totally devoid of therapeutic effects.

Unfounded claims

The authors are quite open about the paucity of RCTs and their mostly dismal methodological quality. Yet they arrive at fairly definitive conclusions regarding the therapeutic value of acupuncture. This is, in my view, a serious mistake: on the basis of a few poorly designed and poorly reported RCTs, one should never arrive at even tentatively positive conclusion. Any decent journal would not have published such misleading phraseology, and it is noteworthy that the paper in question appeared in a journal that has a long history of being hopelessly biased in favour of acupuncture.

Chinese published research on acupuncture: almost always positive

Any of the above-mentioned flaws could already be fatal, but I have kept the most serious one for last. All the 5 RCTs that were included in the analyses were conducted in China by Chinese researchers and published in Chinese journals. It has been shown repeatedly that such studies hardly ever report anything other than positive results; no matter what condition is being investigated; acupuncture turns out to be effective in the hands of Chinese trialists. This means that the result of such a study is clear even before the first patient has been recruited. Little wonder then that virtually all reviews of such trials (and there are dozens of them) arrive at conclusions similar to those formulated in the paper before us.

As I already said: rubbish in, rubbish out!

Links

Primary paper

Xu-Feng W. et al (2016) Acupuncture for amnestic mild cognitive impairment: a meta-analysis of randomised controlled trials Acupuncture in Medicine doi 10.1136/acupmed-2015-010989

Other references

Birks J, López-Arrieta J. (2002) Nimodipine for primary degenerative, mixed and vascular dementia. Cochrane Database of Systematic Reviews 2002, Issue 3. Art. No.: CD000147. DOI: 10.1002/14651858.CD000147.

Photo credits

• Michael CC BY 2.0
• Blue Oxen Associates CC BY 2.0
• Feral78 CC BY 2.0
• Shaun Carpenter CC BY 2.0