Download this page in PDF formatChinese research is apparently getting better. In fact, China is taking first place, ahead of the USA and UK, in publishing scientific papers. With the amount of people China has and the emphasis on being good students, it is hard to see anything stopping China from leading the scientific world other than bad methodology. I suspect most Chinese scientists have recognized the importance of quality in research and are working to repair the poor reputation that Chinese research has developed. Having read plenty of abstracts on Chinese medicine research, I’m well aware of the low quality that has given TCM (Traditional Chinese Medicine) research a bad name. As an American acupuncturist who is determined to provide safe, effective, and responsible treatment in my clinic, I don’t feel the need to hide the fact of research problems in the TCM field. On the contrary, being honest and open about the problems and limitations in my field has won me more respect from intelligent patients, doctors, and scholarly acupuncturists than just parroting the Party line about TCM.
So how bad was Chinese research? In pouring through dozens of abstracts on external Qi Gong energy healing looking for convincing evidence that the ability to project or detect a human energy field has been verified in decent research, the results were stunningly disappointing. Often it’s hard to learn about the actual research methods used from the studies written by the researchers themselves. This is where James Randi’s trip to China with CSICOP (The Committee for Scientific Investigation of Claims of the Paranormal, now just CSI, the Committee for Skeptical Inquiry) takes the cake. I know plenty of psychics and New Agers hiss when they hear these names, but the best way to shame the skeptics would be to rigorously prove psychic abilities in a controlled setting and then take Randi’s million dollar prize (see my recent book review about _Randi’s Prize_) or expose them as closed-minded frauds. Unfortunately for paranormalists, there are far more documented frauds and closed minds on the psychic and religious side of the fence.
As I was reviewing _Randi’s Prize: What sceptics say about the paranormal, why they are wrong and why it matters_, I bought _The Hundredth Monkey and Other Paradigms of the Paranormal: A Skeptical Inquirer Collection_, edited by Kendrick Frazier.
The Hundredth Monkey and Other Paradigms of the Paranormal
Along with a nice introductory essay by science hero Carl Sagan, there are thought-provoking pieces by Martin Gardner, University of Oregon professor Ray Hyman, and other usual suspects from the skeptical movement. An unexpected gem was the chapter “Testing Psi in China: Visit by a CSICOP Delegation” which details a 1988 trip by a group including James Randi. It really hit home when they visited the Shanghai College of Traditional Chinese Medicine, as that esteemed TCM school is the publisher of several textbooks I used in my acupuncture education. I’ve recently been shaking my head over some of the acupuncture point functions given in _Acupuncture: A Comprehensive Text_ put out by the Shanghai College of TCM and translated by Dan Bensky and John O’Connor, which was a good warmup for the head shaking _The Hundredth Monkey_ produced.
Acupuncture: A Comprehensive Text
Fitting nicely within the history told in _Qigong Fever_, the CSICOP team reports the following experience from visiting the Xian Paranormal Function Application Association and working with its secretary-general, Mr. Ding Wei Xin:
Professor Fan Yu Lin, president of the group, said in a formal talk that he believed the paranormal phenomena produced by the children were genuine and he tried to explain this as a product of the evolutionary process.
We were in a crowded room with about 80 people. In the rear were several of the “psychic” children, all about 11 years old. Mr. Ding brought forth two young girls for us to test. He first claimed that they could read characters in sealed envelopes. James Randi produced a number of envelopes that had been prepared beforehand, each containing one randomly chosen Chinese character. The children were permitted to hold the envelope in only one hand and for some 20 minutes they were under constant scrutiny by those present. The results were negative and the children became very distressed. Mr. Ding was evidently surprised. The next test, he said, would be of PK [Psychokinesis--moving things with the mind]. He brought up two other young girls and provided two matchboxes. Inside one box was placed a green match, which we insisted be marked on all sides with two red stripes. Mr. Ding said one of the girls would break the match using her psychic power. A second match, similarly identified, was broken into several pieces and inserted into the second box. He asserted it would be restored to its original condition by the other girl. Again our controls were stringent. The boxes were marked on the inside and outside with the initials J.R. They were then sealed with tape. The girls could only tough the box with one hand or lay it on the floor in front of them. They could not remove it from sight. In evident distress at these strict condition, they were not able to perform, and these tests were also negative. Mr. Ding admitted that he had never before marked the matches.
Mr. Ding was himself dismayed at the negative results. He implied that they might be due to the fact that the children were nervous and under the scrutiny of too many people. He agreed that we should continue testing the next day in our hotel under quieter conditions. That evening Mr. Ding held a huge banquet at which we were able to meet and dine informally with six of his prime subjects, including two young boys. We had a wonderful time and ended the night by disco dancing with the youngsters. The atmosphere was friendly and the attitude was positive, something that Mr. Ding had insisted was necessary for proper testing the next day.
[I'm omitting two paragraphs about another strictly controlled test the next morning which similarly showed no psychic powers.]
Mr. Ding again voiced surprise. The children were usually successful. We then told him he could conduct the next experiment as he wished an we would simply observe his method of testing. So we began anew. This time there were four subjects–three girls and a boy. For this experiment Mr. Ding used cellophane tape and matches with green heads. He also wrapped the boxes in paper. The procedure was the same as before, though Mr. Ding told us he would tape the boxes even more tightly. The first box (“A”) contained a broken matchstick; the second box (“B”), a whole match; the third box (“C”), a broken matchstick; and the fourth box (“D”), three matches. He said the children would restore the broken matches or, if whole, break them by the power of their minds. Mr. Ding kept no records of any kind; nor did he mark any of the matches.
Under Mr. Ding’s supervision, bedlam broke out. After a few minutes the children darted from the room with the matchboxes in their possession. They ran up and down the stairs, in and out of the elevator, inside and outside the building. Mr. Ding saw nothing wrong with this. He did not bother to count the remaining matches, nor did he take notes as we had done of all the previous tests. The children went in and out of the room several times. At about noon, after an hour and a half of running around, the children sat quietly in their chairs for 15 minutes in an attitude of concentration. They then said they were tired, and Mr. Ding was not confident that they had been succesful. The children asked if they could leave the hotel grounds with the boxes. Mr. Ding said yes, and proposed that we meet again at eight o’clock that evening. We were shocked at this loose protocol; but it was Mr. Ding’s test, so we agreed.
That evening they all returned. We were told that one of the boxes, box “D,” had been accidentally destroyed; it was not returned. We then proceeded to examine the other three.
The outer wrapping paper and tape on boxes A and B did not appear to have been tampered with or unsealed. When we opened them, the matches were exactly as they had been before. They had not changed. Box “C” was a different matter. Although somewhat the worse for wear, the box at first appeared not to have been tampered with. But on closer inspection it was clear that the tape had been unwrapped and removed; vegetative matter (most likely grass) and a strand of hair were found under the cellophane tape. We opened the box. It had previously contained five broken pieces of a match with a green head. Now we found an entirely intact match, but it had a red head! Moreover, we discovered that the girl in the experiment had given the matchbox to the two boys who returned it, but who had not even been part of the experiment. Mr. Ding apparently saw nothing wrong with this.
We ruled that there was obvious evidence of tampering and that cheating had taken place. Although Mr. Ding now admitted to us that some of his children had cheated in the past, he maintained that many such cases were genuine. Unwilling to admit that a child had cheated in this case, he argued that there may have been vegetative matter on the table when the matchbox was wrapped. This was simply not so. Moreover, he rationalized that the green matchstick had been miraculously changed to a red one. He reached into his brief case and produced a match, carefully wrapped in paper, which appeared to have both a red head and a green heat. This, he said, had been produced by one of the young boys who had returned it; and he affirmed that this indicated an even more surprising power of psychokinesis! Later, when we confronted the two boys individually about what had happened, we got contradictory stories. One even blamed his father, who he said had told him if he could restore a broken green match he could just has easily change it into a red one!
What may we conclude from this fiasco? It was apparent that Mr. Ding was extremely naive and that he was unable to design a simple controlled experiment to detect fraud. Obviously the children were playing games and doing so with impunity. Yet Mr. Ding attributed their feats to a psychokinetic effect. (Frazier, 243-245)
That part wasn’t at the Shanghai College of TCM, but is an amazing story which lines up with other Chinese children’s “psychic abilities” to read with their ears, etc., that duped some of the highest Communist Party officials (as told in _Qigong Fever_). Directly following that passage is a brief summary of their trip to Shanghai:
The last two days of our sojourn in China were spent in Shanghai, a city of faded elegance with very little new construction other than some tourist hotels. Here we met with the faculty and staff of the Shanghai College of Traditional Chinese Medicine. This group seemed most receptive to our suggestion that rigorous tests be made of Qigong. To our surprise, they were completely unaware of the importance of double-blind trials and expressed enthusiasm about doing such tests in the future. They focused primarily on internal Qigong, a form of relaxation that they claimed could reduce hypertension and have a beneficial effect on other illnesses. They were somewhat skeptical of external Qigong, where a master seeks to induce changes in a patient’s health.
We also met at the Dong Hu Guest House with some skeptical scientists and philosophers who said that they had done tests with psychics and Qigong masters with invariably negative results. Unfortunately, they confided, the press was more interested in reporting the fantastic claims of paranormalists than in the more mundane, skeptical critiques. This, we noted, is similar to what occurs in other countries. (Frazier, 245)
Basic Theory of Traditional Chinese Medicine (II) by the Publishing House of the Shanghai College of Traditional Chinese Medicine (1990)
At least there were no major scandals at the Shanghai College of TCM! However, I’m not very surprised that in 1988 they were unaware of double-blind study design. My textbook on acupuncture from their school was translated in 1981. My version of _Acupuncture, A Comprehensive Text_ is a 1998 printing but doesn’t note any revisions. Amazon (which that last link leads to) describes this book as:
Compiled by the faculty of one of China’s leading schools of traditional medicine, Acupuncture: A Comprehensive Text is among the most authoritative textbooks and reference sources in its field. Since its translation into English in 1981, it has become a standard text used throughout the world.
There are several comments which show this is still regarded as a key book in acupuncture education:
What does the 1981 copyright have to do with anything? The book has stood the test of time. I’ve had my copy since 1984 and still refer to it regularly. Some don’t like the book because it covers the acupoints by anatomical region… If I had to have only one book on acupuncture, this would be it. Fact is, with all of its 741 pages, this book is the bible, at least in English!
I’ve become used to skipping over absurd claims in TCM books, and hope that other acupuncturists do so as well. What do I mean by absurd? I’ve been writing about the nutritional deficiency Beriberi lately. This serious and fatal deficiency of Vitamin B1 is key in the modern development of nutritional science, and in fact led to the coining of the word “vitamin” just 100 years ago! Beriberi has a cheap, natural cure. It’s called Thiamine, and is in many foods. I don’t recommend you try to treat it with acupuncture. Really.
Unfortunately, _Acupuncture, A Comprehensive Text_ doesn’t agree with me. There are several points, especially on the lower limb, which have “beriberi” as an indication. For example, in page 387, the point “Qi’s Extremity” at the tip of each toe is recommended for:
Apoplectic coma, beriberi, paralysis of toes, red swollen dorsum of foot.
The following page lists three other points with beriberi as an indication. Why are these points on the lower leg? Because beriberi leads to swelling in the feet as the capillary walls weaken and the plasma leaks out into the tissues. Next the lungs fill up, the brain misfires, and the heart fails. Many proponents of TCM state that acupuncture treats the underlying cause of a disease, not just the symptoms. Poking holes in the toes and lower legs to treat foot swelling due to severe malnutrition contradicts that claim. I’m not saying that all of TCM is like this, but it is a real example which is clearly put forth in this influential acupuncture text. Why is it there at all? Who on earth would want to treat beriberi with acupuncture, when brown rice, bean sprouts, or B-Vitamin pills will quickly reverse and cure this condition? Did the authors not know about this in 1981? I understand that the translators’ job was just to translate, but it’s important to note that there is a complete lack of a plausibility filter.
There are no footnotes in this book at all. There are no directly referenced studies (even bad studies). The bibliography is fairly short and mostly references other Chinese textbooks (which in turn probably reference older Chinese textbooks). It is fair to say that there is no supportive research for most of the claims made in this book for acupuncture points. Some might think that the Shanghai College of TCM professors knew what a double-blind test was before CSICOP’s visit, but there is no evidence of that in this text. The only good news about this is that there is still a place for good quality research on acupuncture. Unfortunately, the best research on acupuncture so far shows little benefit, mostly related to pain relief. Definitely no studies have found acupuncture to cure a nutritional deficiency.
What other implausible claims are in _Acupuncture, A Comprehensive Text_? Too many to discuss in just one post, but here’s a smattering from the same couple pages on lower limb acupoints (386-389):
- Infantile convulsions
- Seizures
- Mental illness
- Gingivitis (On the back of the heel! Try brushing and flossing first…)
- Brain disease
- Paraplegia (half of the body paralyzed)
- Lupus
- Diabetes
- Puerperal fever (an often fatal bacterial infection of newborns, usually from unclean birthing environment)
Many of the point functions are for local pain (i.e. points around the knee for knee pain). These are more plausible, depending on the cause of the pain. But throughout the book, there are point indications for both specific conditions (like malaria and myopia), and broad categories (psychosis, seizures, paralysis). None of them, from what I’ve been able to find, have been confirmed by decent research. Most of them haven’t even been the subject of bad research. China has notorious “publication bias” meaning they almost exclusively publish positive results (99%). Thus, if a study looks at points on the foot to treat “brain disease” or “mental illness” and fails to find benefit, it will likely be round-filed away instead of printed. This does a disservice to the medical and research communities, as it’s important to know when something has been disproven, too. This would hopefully lead to editing textbooks accordingly and moving on to more productive areas. When cultural pride interferes with honest presentation of scientific findings, it may look good in the short term, but over time everyone suffers.
Most acupuncture patients go for the treatment of pain (I saw the figure 88% somewhere recently, but I can’t find that reference). Most acupuncture research focuses on pain. Unfortunately, pain is very subjective and is the symptom most likely to be influenced by placebo treatments. This increases the need for carefully designed studies with credible placebos.
I suspect very few patients who get a “brain disease” or have a baby with puerperal fever think “I’m going to rely on acupuncture for this!” I also doubt many modern acupuncturists would claim with confidence that they can cure diabetes or gingivitis with acupuncture, though I’m sure many feel like they give “overall support to a healthy system.”
I remember being a beginning student with very hopeful ideas about acupuncture, including the hope that it could cure my nearsightedness. I’ll write more about that in the future (it links in to Aldous Huxley and the Bates Method). But I will say that much of my myopia has been cured by reading Carl Sagan and James Randi.
When the Comprehensive Text was written, China was still dealing with the effects of the Cultural Revolution and the Great Leap Forward, which included starvation and malnutrition. Even in the early 80′s we didn’t take a lot of these indications seriously. All the same, before the gate theory all acupuncture claims were considered “implausible”. What the Comprehensive Text (and most acupuncture texts in general) needs is much more specificity. For example, acupuncture could conceivably speed the relief of neurological symptoms in the course of recovery from Beriberi. Long-term effects like Korsikof Psychosis have no effective treatment and acupuncture might be worth a try. There are basic science studies supporting a role for acupuncture in diabetes management. Acupuncture is worth trying for the joint pain and swelling of Lupus. And so on… In other words, acupuncture could play a role, and perhaps a critical one, in symptom management in many of these diseases without actually serving as a primary treatment for the disease itself. We also have to take into account the fact that for almost all these conditions, Oriental Medicine would consider herbs and Western drugs to be the main treatment, with acupuncture serving only a supportive role. To the authors of the Comprehensive Text, that might have been so obvious that it wasn’t worth mentioning.
John,
Thanks for the perspective. Yes, more specificity about indications and claims will help. I’ve met several acupuncturists (mostly the ones who never studied herbs) who strongly feel that acupuncture can treat/cure any condition (if only the select the right points and twirl the needle the right direction). So our hope that not taking the indications in A Comprehensive Text too literally is “obvious” hasn’t been reflected in my interactions with other acupuncturists. When I was a student, the meme was “it’s no longer wondered IF acupuncture works, but WHY it works.” With the better systematic reviews, it seems the IF question is back on the table for even much pain treatment (i.e. comparing different acupuncture systems to credible placebos). I still feel that many of my patients get *value* from my acupuncture, but that cost/benefit calculation is tricky at times, especially with more serious conditions. Part of writing about this is to elucidate the difference between what acupuncture *could* do, what it *can’t* do, and what it *can* do. The fact that the Comprehensive Text has remained unrevised since 1981 and is still regarded as a top text in the field isn’t very encouraging. Again, the most hopeful spin to put on this is that there is a great need for good research to find the best uses and limits of acupuncture. There should be a good market for updated texts following such research (if the profession values scientific progress).
Kevin
This is from my forthcoming book, which will deal with acupuncture and other sorts of Chinese medicine. I asked our US National Institutes of Health for all the “scientific papers” published on acupuncture. They gave me 308, most ineligible because of basic flaws. We were surprised at the very bad quality of scientific input… The excerpt:
………………………………………………………………….
Here are the ten rules that Lawrence Herman and I drew up in advance of Mr. Herman’s examination of the data, for rejection of any of the 308 papers. At the end of each entry is a number indicating the number of papers that were rejected by that rule. Many papers earned inclusion in more than one category:
1. If no abstract of the paper was available, it would have taken weeks of research and examination of data to evaluate the methodology and conclusions. Very seldom is a proper paper presented without a proper summary of methodology, and those that were so prepared, were also vague and inconclusive. [1]
2. If no experimental conclusion was reached. That is essential to the integrity of any paper. [0]
3. If an insufficiently large database was used. Some had only a dozen examples. [66]
4. If no English version was available. In many cases, these were in Chinese, and not available in another language. [22]
5. If electro-acupuncture, laser-acupuncture, acustimulation*, auripuncture**, or acupressure, were tested, since these are esoteric variants of the classic needles-at-the-magical-point process, and not the claim being examined. [65]
*this is merely manually rubbing the traditional acupuncture points
**acupuncture done only on the ear, that organ representing the entire body
6. If the tests were not done double-blind. This is an absolutely rigid requirement for a proper protocol, especially in examining a procedure that is highly sensitive to prejudices and preferences. [14]
7. If it is stated in the paper itself that the tests were not sufficiently controlled. This was mentioned a few times in the summary of tests, and the results are thus damned at that point. Without proper controls, we have no test. [4]
8. If the test(s) involved acupuncture plus drugs. Not evaluable due to possible compromising effects of legitimate pharmacological factors. [4]
9. If the material offered was simply irrelevant to the basic claim that needles inserted at the designated magical points on human beings would bring about relief or cures. [133]
10. And, of course, if the results were negative. [35]
NOTE: Items 3, 5, 6, 7, 8, and 10 immediately invalidated a paper and removed it from consideration. Papers with insufficient databases, no double-blind provisions, no controls, and/or negative results, cannot serve to support a claim. Any favorable result inferred from such a procedure is frivolous. Many of these papers were simply irrelevant to actual acupuncture – category 9 – and therefore rejected; the very basic acupuncture claim is that it brings about pain relief or other benefits, and that should be the basis for judging the tests. The vague and varied nature of most of these papers would confuse any reader, even those experienced in the jargon and provisional nature of the terminology used. Many papers showed that the experimenters changed goals and directions at a whim, made unacceptable and unexplained exceptions, and generally produced material that offended any accepted scientific standards.
It appears to me that the NIH may have simply done an Internet search for the word “acupuncture” in any purportedly “scientific paper” they could find, and listed them to send to me, depending on quantity trumping quality and thus providing me with a mass of essentially useless data. Mr. Herman did the work that the spectacularly-funded NIH should have done, and I believe that my point is proven: there exists no properly-derived scientific database that supports the validity of acupuncture, and what real data does exist, denies it. The 308 papers cited to me by the NIH, and the comprehensive analysis of them provided by Lawrence Herman, are far too lengthy to be included in these pages, but any interested reader can contact the JREF – at http://www.randi.org – to obtain a complete summary and run-down of the material for a cost-only fee.
The NIH avoided answering my challenge by choosing to ignore and maneuver around it, and the acupuncturists – the quacks – continue, unchecked, to claim – and receive – endorsement and support from major health-care centers and agencies. And we pay the costs.
……………………………………………..
I think you can under stand my dismay at seeing the bad quality of Chinese “scientific” research, and all the problems with determining the real facts behind acupuncture claims…
James Randi.
Dear Amazing Randi,
Thank you for taking the time to leave a comment on my blog. I look forward to reading your new book, as I have learned immensely from the many books you’ve authored which I have and appreciate. I started studying nutrition, then herbal medicine, then Chinese medicine in the pre-internet era, and have always hoped that the theories and claims of TCM had been or would be supported by good research. I was dismayed to find, upon delving into it for years, that most of the dramatic claims for acupuncture point functions were supported by only implausible theories and vague anecdotes. With the internet’s vastly increased access to both good and bad information, even the amateur researcher or patient needs to quickly learn how to tell good evidence from bad evidence. Your 10 points are very clear and reasonably stated.
Given the long, well-documented history of charlatanism in the “energy healing” world, I didn’t expect to find quality research supporting any of the claims of external Qigong healing when I researched the available abstracts. Yet I was shocked to discover how bad the research was; for example, studies where the abstract only mentions “External Energy Healing” but the full paper notes that massage and cupping was also allowed.
While the Cochrane Collaboration’s systematic review system, which weights studies for quality, shows a few areas of pain and nausea where acupuncture seems somewhat better than placebo, the positive results are not impressive and certainly don’t verify any of the special point functions as taught in Traditional Chinese Medicine (such as the five elements energetics). This has led me towards George Ulett’s (another skeptical magician and author) approach of using electroacupuncure and a dermatome approach to mostly treat musculoskeletal pain.
Part of my writing approach is to review most of what I’ve been taught to create what I can of a “scientific evidence influenced” practice. This is heavy on basic exercise, pragmatic dietary advice, conservative acupuncture/massage/electrotherapy with honest informed consent, and increasingly helping people to question unproductive superstitious beliefs about health which cause wasted time and money (and in many cases fear, anxiety, and guilt).
As I uncover and write about more and more outright deceptive fraud in the TCM and CAM world, I am increasingly drawn to educate and warn people about it, which consistently goes back to increasing literacy about research design and good science. Of course, studying magic tricks and related history is (as you well know) one of the most enlightening and useful things for a researcher into esoteric and paranormal claims to do.
I would very much like to purchase your complete summary and run-down of the acupuncture research in addition to your upcoming book when it is available. I am also looking for good historical material about Chinese magic tricks used in religious and medical practice (I’ve discussed some historical trails of magic in this book review and in this post). If you have time to share any resources or titles from your extensive library, I’d be very grateful to carry on my studies in this niche.
Sadly, my attempts to recruit energy healers and muscle testers to help me make scientific history and split your million dollar prize have not brought any serious candidates. Your challenge is one of the best motivations for anyone with real psychic abilities to prove to the world that such abilities are real. I recently reviewed _Randi’s Prize: What sceptics say about the paranormal, why they are wrong and why it matters_, and hoped it would make a more convincing argument than it did. If you haven’t read that book, a quick scan of my review will tell you all you want to know and more.
Thank you for your years of hard work in researching and documenting these realms of human delusion, and thank you for making my day by posting a comment on my blog.
Regards,
Kevin O’Neil
i am disappointed at the reference of “MAGICAL POINTS” =in referring to acupuncture points, by Mr Randy…
this comment clearly shows bias.
skeptics are open mined and cynics have an agenda.
=you can NOT disprove a negative.
=if you do not have any DATA, then you simply do not know=that is all there is.
…but that is good because not knowing allows knowing to be.
until there is data,,, there is NONE, thus it is a unknown and NOT SCIENTIFIC to refer to them as “MAGICAL POINTS” even in jest…
=so when Mr Randy uses words like magical points he is clearly showing insincerity and is a clearly bias =thus why would even Mr Randy bother to study TCM if he was not going to be open minded about the possibility and thus it leads one to a POSSIBLE conclusion that Mr Randy just wanted to prove HIS AGENDA and not really scientifically investigate it…
Tim,
Thanks for your comment. There is a strong component of magical (as in religious magic) thinking in TCM theory. For example, the Five Phases (Wu Xing, five elements) is largely a magical system of correspondence. There are a group of points which theoretically function based on the interactions of these elements. There is no supportive scientific framework or research-based evidence for these points (between the knees and toes and elbows and fingertips) having any specific effect on the organs they are theoretically linked with. Research has been done on many of these acupuncture systems, though most of the research has been of poor quality. The best research shows little difference between systems of acupuncture, and unfortunately, also little difference between credible placebo acupuncture and “real” acupuncture.
I don’t see where Mr. Randi is trying to disprove a negative. The acupuncture texts clearly make positive claims about the functions of these points based on magical theories. If a patient or practitioner is deciding how much time and money to put into treating a serious condition (such as paralysis, diabetes, kidney failure, etc.) with acupuncture, it is fair to look for supportive research. So much of the acupuncture research (especially from China) has been of poor quality, it can’t be regarded as “proof” of these point functions. However, this poor research record becomes its own form of “data” regarding the existing evidence for acupuncture’s scientific support.
It is always appropriate to be suspicious of a researcher’s agenda. Researching ‘psi’ phenomenon has been an area where political, religious, and financial incentives have led to blatant fraud again and again. Only with careful observation and strict methodology can this type of fraud be detected. Since psychic healers and shamans have successfully used sleight-of-hand tricks for centuries, it has often required conscientious stage magicians like Harry Houdini and James Randi to expose them.
I suggest you read my other posts about Qigong research, here are some links:
http://ancientway.com/blog/?p=962
http://ancientway.com/blog/?p=1989
http://ancientway.com/blog/?p=2025
TCM and Taoism have been a large part of my life for 20 years. I’m specifically interested now in which claims and theories are supported by good research or suitable for doing good research. A large part of this process includes learning about what good research is and evaluating research which has been done. Rather than dismissing Mr. Randi for a suspected agenda, finding and sharing good research supporting TCM and acupuncture theories would be a more convincing path.
Regards,
Kevin
Here is an extract from my forthcoming book “A Magician in the Laboratory” in which I deal with the acupuncture matter. This is only a fraction what I wrote:
In response to the third, meticulously-addressed and very specifically-directed request I’d made to the NIH [National Institutes of Health] about their tacit acceptance of acupuncture, I first received a list of 308* “scientific” papers on tests done of acupuncture, originating from all over the world. These were of highly variable nature, most of them very vague and incomplete, and suffering from obvious flaws. Lawrence A. Herman has been a medical editor and/or writer since 1988, working for, among others, the medical publishing company Elsevier Science Inc., the Medical Education Broadcast Network, MediMediaUSA, and the international pharmaceutical company Sanofi S.A. He is currently Editorial Supervisor for the medical education company Scientiae, LLC in New York. With access to an excellent medical library/database, he volunteered to examine these documents for me. A summary of his findings is shown up ahead, and a detailed version of his report can be seen as APPENDIX # 4 of this book.
*later augmented to 310
Finally, after months of requesting an official and responsive answer from the NIH, I received a statement headed:
DEPARTMENT OF HEALTH & HUMAN SERVICES
Public Health Service
National Institutes of Health
National Center for Complementary and Alternative Medicine
Buried in that letter, following extensive descriptions of the valuable work being carried on by the NCCAM, I found this in answer to my question:
As a research agency, NCCAM funds research on complementary and alternative medicine (CAM). We do not endorse CAM treatments.
It went on to give references to many other sources of information about my question, all of them ambiguous and painfully “politically correct.” It was signed by Chris Thomsen, Director, Office of Communications and Public Liaison.
Here are the ten rules that Lawrence Herman and I drew up in advance of Mr. Herman’s examination of the data, for rejection of any of the 308 papers. At the end of each entry is a number indicating the number of papers that were rejected by that rule. Many papers earned inclusion in more than one category:
1. If no abstract of the paper was available, it would have taken weeks of research and examination of data to evaluate the methodology and conclusions. Very seldom is a proper paper presented without a proper summary of methodology, and those that were so prepared, were also vague and inconclusive. [1]
2. If no experimental conclusion was reached. That is essential to the integrity of any paper. [0]
3. If an insufficiently large database was used. Some had only a dozen examples. [66]
4. If no English version was available. In many cases, these were in Chinese, and not available in another language. [22]
5. If electro-acupuncture, laser-acupuncture, acustimulation*, auripuncture**, or acupressure, were tested, since these are esoteric variants of the classic needles-at-the-magical-point process, and not the claim being examined. [65]
*this is merely manually rubbing the traditional acupuncture points
**acupuncture done only on the ear, that organ representing the entire body
6. If the tests were not done double-blind. This is an absolutely rigid requirement for a proper protocol, especially in examining a procedure that is highly sensitive to prejudices and preferences. [14]
7. If it is stated in the paper itself that the tests were not sufficiently controlled. This was mentioned a few times in the summary of tests, and the results are thus damned at that point. Without proper controls, we have no test. [4]
8. If the test(s) involved acupuncture plus drugs. Not evaluable due to possible compromising effects of legitimate pharmacological factors. [4]
9. If the material offered was simply irrelevant to the basic claim that needles inserted at the designated magical points on human beings would bring about relief or cures. [133]
10. And, of course, if the results were negative. [35]
NOTE: Items 3, 5, 6, 7, 8, and 10 immediately invalidated a paper and removed it from consideration. Papers with insufficient databases, no double-blind provisions, no controls, and/or negative results, cannot serve to support a claim. Any favorable result inferred from such a procedure is frivolous. Many of these papers were simply irrelevant to actual acupuncture – category 9 – and therefore rejected; the very basic acupuncture claim is that it brings about pain relief or other benefits, and that should be the basis for judging the tests. The vague and varied nature of most of these papers would confuse any reader, even those experienced in the jargon and provisional nature of the terminology used. Many papers showed that the experimenters changed goals and directions at a whim, made unacceptable and unexplained exceptions, and generally produced material that offended any accepted scientific standards.
It appears to me that the NIH may have simply done an Internet search for the word “acupuncture” in any purportedly “scientific paper” they could find, and listed them to send to me, depending on quantity trumping quality and thus providing me with a mass of essentially useless data. Mr. Herman did the work that the spectacularly-funded NIH should have done, and I believe that my point is proven: there exists no properly-derived scientific database that supports the validity of acupuncture, and what real data does exist, denies it. The 308 papers cited to me by the NIH, and the comprehensive analysis of them provided by Lawrence Herman, are far too lengthy to be included in these pages, but any interested reader can contact the JREF – at http://www.randi.org – to obtain a complete summary and run-down of the material for a cost-only fee.
The NIH avoided answering my challenge by choosing to ignore and maneuver around it, and the acupuncturists – the quacks – continue, unchecked, to claim – and receive – endorsement and support from major health-care centers and agencies. And we pay the costs.
That ends my selected text. James Randi.
I think this is great deal in the world of chinese medicine and acupuncture. How to do good study. There are two big problems 1. EBM medicine 2. Chinese medicine itself
)))) EBM is great idea, but it not support things like individual prescription. In this concept when we want to proove something, we need really standardize everything. In reality this is not possible in acupuncture research for 100 % percent . Therfore many papers use electroacupuncture, because scientific journals require it.Another topic is placebo acupuncture. If we look about the classics, they say that acupuncture channel is not only some kind of canal under the skin, but it contains also flesh a and supported skin areas. And this is also one thing because it shows that needling acupoints or points next to acupoint are very simillar effects.Also when we look inside the huangdineijing, they also support much more needling of the channels then needling of the acupoints. So I think still the idea of pacebo acupuncture is wrong and it was created only to fit the EBM ( my personal opinion). Also EBM has many weaknes..as a medical doctor of intensive care I know that many western mediicne what we used in our daily practice, are not supported from EBM or some studies are against and some studies are pro effect of this drugs and therfore using of the drugs depend also about meaning of most of doctors in department or guidelines done by governement. So we have to careful in judge what is EBM or not. Another question is quality of practioners doing acupuncture and they skils in needling of the acupoints. As we know now is much more better then before and also what we know, that the quality of therapy depend also of the skilful acupuncturist. For example in many countries medical doctor take only 200 hours educations in acupuncture and this is not enough.We really need to find system how to reserch acupuncture and I think many good people in this field they really work hard to implement this idea to EBM medicine, but also we need that EBM recognize the need of individual treatment, on what is acupuncture an chinese medicine based.
Boris,
Thanks for your thoughts. I agree that better research design is appropriate, but I think that individualized treatments are not a huge barrier to gathering good data. In fact, some studies have looked at individualized herbal prescriptions versus a standardized prescription–the results haven’t yet confirmed that individualized prescriptions are significantly more effective (to put it gently). Some acupuncture studies have also looked at different styles vs. placebo and found that various styles have similar results, not much better than placebo (http://ancientway.com/blog/?p=538).
Studies which evaluate a whole system of individualized practices are often called “black box” studies, where you track the before-treatment status and the follow-up status without controlling the individualized protocols. One problem with most studies is the small number of participants. The internet is likely to change that and allow distributed data collection. For example, if one could get 10,000 people with headaches to try 10 different types of therapy (i.e. TCM acupuncture, five element acupuncture, massage, individualized TCM herbs, standardized herbs, prescription meds, over-the-counter meds, meditation, homeopathy, etc.), the data should be useful with 1000 people per group.
The recent development of the credible placebo acupuncture needle which retracts but still convinces the acupuncturist and the patient that they may have actually penetrated with a real needle gets over some of your objections about placebo acupuncture. This is certainly a dilemma I face as a practicing acupuncturist who respects scientific research: if well-done research consistently shows that non-piercing needles are as good as piercing needles, it would be better to use the method with the least amount of risk. For now, I’m going with some evidence that electroacupuncture with piercing needles (PENS-Percutaneous Electrical Nerve Stimulation) is better than TENS (Transcutaneous ENS) electro-pads on the surface of the skin.
Unfortunately, what I’ve seen too regularly in the TCM world is Anti-Science, where the findings of good research are rejected on the notion that research itself is flawed. While poorly designed individual studies can be flawed, the approach of the Cochrane Collaboration (http://www.cochrane.org) has accounted for methodological quality in their careful systematic reviews. The trend in acupuncture research is clearly that the better the study (in terms of careful blinding, credible placebo, good study size, etc.), the less benefit is seen.
One of the last hopes that the TCM world has is to quickly produce good quality, reproducible research showing a reliable positive effect which confirms TCM theories. It is partly to this end that I am writing about research quality and problems in the field to hopefully encourage better awareness of good and bad in research.
I presume that the poster is referring to me when he specifies “James Randy”? My name is James Randi…
In any case, I have personally visited the Beijing Acupuncture Center, and those in charge there are mainly concerned with two facts. First, they know that Chinese persons who believe that acupuncture works, obtain great peace of mind from receiving medical attention that they know to be Asian in origin, and second, they see acupuncture as a means of cutting down on the great number of citizens who are prone to go for medical care even when they don’t need it.
James Randi.
I don’t generally correct spelling in comments people leave here, as some readers correlate accuracy in typing to other observational skills.
While the good quality research evidence is slim, it seems that some acupuncture techniques, particularly modern ones such as electroacupuncture and dermatome-based trigger point therapy, may be better than placebo for treating some types of pain. It is amazing how effective a convincing placebo is for reducing pain. That is the complicating factor in getting an accurate picture of acupuncture’s strengths.
It is absolutely fair game to look at the magical theories of traditional acupuncture point selection rules (i.e. five phases, twirling a needle one way to tonify and opposite to disperse) and ask for supportive evidence. In looking for such evidence, it is fair to exclude studies which use more modern techniques, such as electroacupuncture (PENS–Percutaneous Electronic Nerve Stimulation).
Many prominent practitioners of acupuncture have dispensed with most of the metaphysical theories, though to different degrees. Miriam Lee, Felix Mann, and George Ulett come to mind. In fact, my first Chinese acupuncture teacher slyly mentioned that one could run an acupuncture practice with only 10 points, but we’d need to learn hundreds in school. The parties most adamant about special point functions and selection are the schools and seminars which make money teaching complicated systems. If they would offer research comparing their systems to basic or placebo systems, they would be in a better place to promote their style. I suspect that since the good research fails to support the superiority of any traditional acupuncture point selection over standard points or, in many cases, placebo acupuncture, many schools and acupuncturists willfully disregard it and slip into anti-science mode.
Yes, PENS certainly works, but has no relationship to acupuncture…
James Randi.
PENS (Percutaneous Electronic Nerve Stiumulation) has a relationship to acupuncture in that it is generally within the scope of practice of a licensed acupuncturist. Electroacupuncture, as it’s often referred to, is taught in many acupuncture schools and may be combined with either traditional (magical) or modern (dermatome and trigger point) point selection.
As an acupuncturist who is increasingly skeptical of what I was taught but wants to provide the most plausible, effective treatment I can for my patients, I am glad that PENS is available within my scope. While critics may prefer that nobody seeks or practices anything called “acupuncture,” with the current situation it is a more realistic hope that acupuncturists acknowledge the research situation and make the most responsible and ethical decisions possible with their patients’ best interests in mind. For me, that means not suggesting that acupuncture can cure internal diseases (a sentiment echoed by some of the main ancient acupuncture texts) and using PENS and other pragmatic approaches to treat painful conditions which have been evaluated by mainstream physicians.
Kevin
It’s worth taking a critical look at your new friend, Kevin:
http://www.dailygrail.com/blogs/Steve-Volk/2012/3/TOP-10-DEVELOPMENTS-FRINGE-OLOGY-5
John, I was aware of some of that ongoing case, but wasn’t aware that the stolen identity was from someone who is still alive. I have some compassion for them due to the unequal rights for same-sex partners (i.e. a man can “buy” a foreign bride and legally get her citizenship even without being able to communicate much), though I understand there were probably legal ways around the immigration problems.
I will not dismiss Randi’s lifetime of work for the personal misdeeds of his partner, though I can understand that it raises troubling questions for many.
The most appropriate way to prove Randi wrong would be to independently publish scientifically acceptable evidence of some paranormal ability (such as External Qi Healing) and then go after the Million Dollar Prize. If such research had already been published in Nature, JAMA, the Lancet, etc., or won a Nobel Prize, it would look bad for JREF if they denied the applicant or award.
This is why I attempted to recruit someone to convince me of their External Qi or Applied Kinesiology skills with the offer to pay them a little at first, help publish research in a reputable journal, and then go after the Million Dollar Prize (which we’d have split), documenting the whole process along the way. Granted my blog is still small, but this offer was well-placed on Google and had tens of thousands of views. I had no serious applicants for the year the offer was open (I could still be talked into it). Yet I regularly get promotional mailings for Applied Kinesiology seminars, where acupuncturists are ‘taught’ to diagnose and prescribe with paranormal techniques that have failed all double-blind tests. This is where my focus is–on simple scientific investigation of CAM claims. While I personally respect Randi’s contributions, I would delight in splitting the Million Dollar Prize and making scientific/medical history even more.