Proof against Homeopathy Supports Homeopathy

Proof against Homeopathy does in fact support Homeopathy

By Lex Rutten, Erik Stolper, David Spence, David Reilly, Ton Nicolai.

Suppose a salesman visits you with a basket full of more than a hundred apples. You ask: “are they all fine?” Behind your back, the salesman takes eight apples out of the basket. He shows you the apples and they look fine to you. “This is proof that all the apples are fine”, the salesman says. You ask: “can I  take some out too?” and you take eight rotten apples out of the basket, just like that. “That means nothing, because I have just proved that all the apples are fine”, the salesman answers.

It seems highly unlikely, but the editors of the first-class medical Journal ‘the Lancet’ were cheated likewise and then went on to propose that such a trick will mean the end of homeopathy. On August 27th 2005, the Lancet published a study, which, they said, proves that homeopathy is nothing more than a placebo effect (1). With a selection of 8 out of 110 studies on the effectiveness of homeopathy, medical science is of the opinion that it can prove millions of satisfied users (just for the Netherlands) wrong. This basket of apples was, therefore, warmly welcomed: for opponents of homeopathy, fine apples are studies that show no effect for homeopathy and rotten apples are studies with positive results for homeopathy.

Background

We have little knowledge of the underlying mechanisms of homeopathy. The theory that toxic materials can also cure is commonly accepted, for example in vaccination. But the concept that homeopathic medicinal products are still effective after a gradual process of high dilution and intensive succussion is beyond the bounds of current scientific understanding. Still this method has been used worldwide for centuries, especially by patients with chronic complaints. These patients have had wide experience with conventional medicine and have observed that conventional medicinal products may reduce their symptoms to some extent, but their complaints often recur. Such patients have found that, by using homeopathic treatment, their complaints may disappear completely, while the treatment hardly ever causes any serious side effects.

In 1999 a number of complementary forms of medicine, of which homeopathy was one, were experimentally included in the Swiss health insurance. Meanwhile a large research program with numerous research institutes was started to study the effectiveness and cost-effectiveness of these complementary treatments. The program was called Programm Evaluation Komplementärmedizin (PEK). One of the institutions was the Institut für Sozial und Preventiv Medizin (ISPM). This institution carried out the meta-analysis, recently published in the Lancet, and which is further discussed in this paper. However, the outcome of the meta-analysis was previously announced in the public press back in August 2003. The methods and the conclusion of this meta-analysis in particular, received a lot of criticism from the PEK management. The PEK management observed that different treatments for different diseases cannot be approached as if you are looking at one treatment for one disease. This criticism is not even mentioned in the Lancet publication, let alone discussed.

The provocation

Shang’s analysis was induced by positive results of former analysis of homeopathic research. Up to 1990, sceptics thought that the problem of homeopathy could be solved by double-blind study, this should show that homeopathy does not work better than placebo. But in 1991 the first reports showed that quite a lot of research on homeopathy had been done and that “…a conventional method with comparable results would be acknowledged”.(2) At a jubilee lecture of the Lancet in 1998, epidemiologist Vandenbroucke presented a portrayal of the available double-blind studies on homeopathy at that time. The above top right-hand dots represent the studies with better study quality and all the dots on the right represent the studies with a strong positive result. The portrayal is adapted from an analysis of 89 homeopathic studies by Linde (3). The top right-hand dots represent the most convincing studies.

Vandenbroucke challenged his audience to come up with a conventional treatment with better results.(4) From the analysis performed by Linde3 it was stated that although homeopathy was more effective than placebo, there was not enough evidence to conclude that homeopathy is clearly efficacious for any single clinical indication.

Bias

Proof for homeopathy is hard to accept for sceptics. However, it is also now well known that doubleblind studies are not as “beyond question” as was once assumed. Studies performed by researchers linked to pharmaceutical companies have more positive results than studies performed by others. Editors of medical journals discovered that they are being deceived: scientists were paid to attach their names to studies, withhold negative results and give a better picture of study results than is really the case. For instance many prescribed antidepressants turned out to be ineffective and also increased the risk of suicide. (5), (6)

Based on the positive outcomes of studies, as performed by or on behalf of the pharmaceutical industry, one may assume that positive outcomes of homeopathic studies are biased as well. However, one forgets that homeopathy has no framework for research. Most research is performed in cooperation with conventional research institutes and to obtain sufficient patients one needs the cooperation of conventional doctors. It is, therefore, much less likely that negative outcomes are withheld.

Objectivity

The objectivity of the evaluation of research is questionable as well. In 2001 Ezzo et al. analysed 160 evaluations of conventional studies performed by the renowned Cochrane institute. They concluded that only a small number of conventional treatments are based on solid scientific research (RCTs) and that the evaluation of the research is highly subjective.(7)

Evaluation of homeopathic studies might therefore also be highly subjective, especially if performed by opponents. The head of the ISPM (the institute that performed the meta-analysis), Egger, is known to have been an opponent of homeopathy for many years. Meanwhile, when you analyse all the available literature, it is almost impossible to claim that homeopathy is nothing more than a placebo effect. If that was the case, then homeopathy, in comparison to placebo, should have similar outcomes as studies with placebo, sometimes better, sometimes worse. However, in two-thirds of all studies analysed, homeopathy is more effective than placebo, in less than ten percent worse than placebo and comparable in the remainder of the studies. The only remaining arguments against proof for homeopathy are the possibility of deceit or errors made in research. A difficult question because errors are made throughout biomedical research, whether it’s homeopathic or conventional. But the question is: do errors occur more in homeopathic studies than in conventional studies?

Trends

If we are not certain about something, we look for trends. For example, cloudy weather means a higher chance of rain. A comparable trend in biomedicine is that the effect of treatment decreases as the quality of the research improves. This is observed in both conventional and homeopathic research.(8) The question is: is this trend is stronger in homeopathy than it is in conventional medicine.

The Swiss researchers assumed that the trend towards a negative outcome for homeopathy would be stronger with increasing quality of studies. They ignored the fact that homeopathic infrastructure for research is non-existent compared to conventional infrastructure. Conventional studies were more recent than homeopathic studies. At first the researchers evaluated all studies and found that they obtained the opposite result. Then they made a subdivision by diagnosis and, even then, homeopathy did not turn out to be worse. For acute upper respiratory infections homeopathy proved to work whichever way you look at it. Finally they came up with a combination of 8 studies which did show the desired trend and nobody is allowed to know which studies they are.

Lack of clarity

If you read the publication in the Lancet several times you get more and more doubts about it. It appears really strange that they did not state which eight studies they had chosen to demonstrate the desired trend. That is scientifically less than honest and this is further compounded when the previously expressed criticism of the PEK commission is not even mentioned. However, a picture is then displayed that should show what they really have done:

This first graph used a similar method to analyse the quality of the study as compared to Linde (vertical axis), but in contrast to the Linde graph, the intensity of the effect is reflected to the left of the dotted line. A striking contrast with the graph from Linde is the presence of just a few dots in the lower part of the axis. This picture does not correspond to the general expectation and observation of comparable pictures that studies with low quality (which should be plotted in the lower part of the graph) are always more numerous than studies with better quality. Perhaps the researchers have omitted studies. This is strange because they studied whether the omission of studies was causing bias or not. The publication shows that they excluded about 50 of the 155 studies, 43 on the basis of not being able to retrieve the publication, not providing sufficient information and inability to identify a matching trial of conventional medicine. This is where subjectivity may play a role. If we look at the list of analysed studies it is striking to note that three studies with positive results have been left out, which were ranked highly in previous analyses.(9), (10), (11),

What effect does the removal of dots in the lower part of the graph have in this picture? Three hypothetical lines are shown. The central line indicates that studies with low quality give better results (situated below left) and studies with high quality give less good results (situated more upper right). The upward slope to the right indicates the trend on which the conclusion of the authors is based. Add some dots to the lower right and upper left and the line will be more vertical and the trend disappears. By adding the three dots of the above mentioned good quality studies to the above left of the picture the trend becomes more positive.

In the second graph above we see conventional studies depicted and the slope of the line is more vertical, i.e. the effect decreases with increasing study size. However, in the text on page 729 the authors state the opposite, i.e. that for homeopathic studies the trend that the effect decreases with improved quality is (not significantly) less than for conventional studies. Thus, in conclusion, the pictures suggest the desired conclusion, but the data in the text demonstrates the opposite.

All sorts of comparisons were made in the Lancet paper between homeopathic studies and comparable conventional studies. The homeopathic studies proved to be of better quality in several respects, for example;

· 19% of the homeopathic studies were of good quality, whereas only 8% of conventional studies were.
· Heterogeneity was significantly lower in homeopathy than conventional research (p=0,011), therefore the error this study explicitly searched for occurs more often in conventional studies.

They have tried to demonstrate that studies of homeopathy are unreliable because of bad quality and larger heterogeneity than conventional studies, but the opposite proved to be the case. The decrease of the effect with increasing study quality was 24% lower than in conventional studies. However, from these data it cannot be concluded that homeopathy is better since this number did not reach statistical significance. On the other hand, the researchers, echoed by the editors of the Lancet, presumed to have proved the opposite based on a selection of 8 out of 110 studies, which cannot be checked.

Phantoms

Nobody believes that you can make an amputated leg grow with a medicine. Let us call research that has no positive outcome a phantom. Phantom research does not mean that medicines in general do not work, but that the outcome indication is a phantom.

Studies into such phantoms are nevertheless regularly made with less clear indications. In regular medicine, the necessary preliminary analyses are made in which similar phantoms are committed. In homeopathy, such preliminary analysis is hardly possible.

Thus it can happen that research is done in homeopathy on prevention of influenza. In day-to-day practice, homeopathy is not often used for this indication and it is quite possible that this is a phantom and that homeopathy does not work in the prevention of influenza.

That a piece research produces no positive results could still have a lot of other causes. There is a study into the use of homeopathy in asthma where an inappropriate measuring tool was used (12). According to this measuring tool, the condition of the patients before treatment was so good that no more improvement was possible as a result of the treatment.

Phantoms and studies with methodological shortcomings give false negative outcomes. The authors of the paper have excluded 30% of the available studies, but not on the rationale of false negative studies.

Fictitious conclusion

It is, therefore, possible to find qualitatively good studies with negative results. But this does not mean that the method does not work. Neither does the finding of a trend in some studies mean that this trend is present is all studies. If that were the case, one could make the assertion that the lack of effect with antidepressants implies that conventional biomedicine in general is not effective.

The authors of the Lancet paper have found two groups of studies, which they state belong to the same diagnosis and therapy and make a comparison possible between homeopathy and regular medicine. Of one group we know that it contains 8 studies of homeopathy on acute upper respiratory infections. Analysis of this group shows that the effect does not decline with increasing quality of the studies.

Scientifically speaking, this is a spectacular conclusion. The previous analyses show that evidence for homeopathy can measure up to evidence for conventional medicine. One previously accepted the facts that there was no proof for any single indication and that with increasing quality of study design the evidence would become weaker. These last two objections have now been removed.

From an opponent of homeopathy, such as Egger, one can expect that he wants to avoid this conclusion. A second group of 8 studies has been found which, according to the ISPM, can be boxed together. There is a lack of clarity concerning the criteria used to select this group of studies and the 8 studies are not identified. Nor is the reader allowed to see what has been done with these studies. Furthermore, the conclusion that has then been drawn from these studies has then been extrapolated to include all homeopathic studies. By circular reasoning, it is further concluded that the evidence for homeopathy in upper respiratory infections is no longer valid either.

It is scarcely believable that the Editors of a first class medical journal, such as the Lancet, can accept that the most vital data of the study are kept a secret. The authors have obviously not been honest and concealed the fact that the management of the PEK programme had provided strong criticism. This criticism is stated in the final report, which is dated April 24, 2005.
(www.bag.admin.ch/kv/forschung/f/2005/Schlussbericht_PEK.pdf).

According to the PEK management there is a heterogeneous group of disorders and treatment. This fact in itself leads to the results. The PEK management opposed overrating patient numbers. In the final report of PEK (page 83) the following table is shown:

This table states that when a strong effect is expected less patients are necessary for the conclusion. The odds ratio is then so low (the impact so much stronger) that the Standard Error (strongly dependent on the research numbers) is larger and the result is nevertheless significant. In example III in the table above, for example, in the verum group only 20% do not respond to the treatment against 50% in the placebo group. 95% reliability intervals of the odds ratio run from 0 to 0.76 (= 0.25 + 0,51). It is then very improbable that the odds ratio could be one (which means no specific effect). The proposition of the ISPM that larger patient numbers stand for better research is incorrect.

The PEK management states very clearly that this secret group of 8 studies should not be boxed together. Taking this into consideration, the whole basis of the Egger paper’s reasoning and certainly its final conclusion is incorrect.

Because of the secrecy, we do not know if there may be phantoms among the eight chosen studies. When looking especially at patient numbers it is obvious that the prophylactic action of homeopathy on influenza has been examined. In 3 such studies, numbers of 1595, 1270 and 501 patients were mentioned (13)(14)(15). These are phantom studies.

Other factors?

From the arguments outlined above, the conclusion must be drawn that the Egger paper does not meet the minimum, conventional criteria for publication in a top quality biomedical journal, such as the Lancet. The question remains as to why the editors of the Lancet approved the paper for publication, despite these obvious flaws. We can only speculate as to the answer to this question.

One possible reason that the Editors of the Lancet might have accepted the paper for publication was to counteract the upcoming publication of a report on Homeopathy by the World Health Organisation. The preliminary draft of this WHO report, which is currently being extensively revised, has been leaked into circulation and comes to the conclusion that homeopathic treatment is effective and that further research is warranted. It is notable that the editors of the Lancet have included a comment on this leaked draft WHO report in the same issue as the Egger paper.

It must also be pointed out that the Egger meta-analysis, as published in the Lancet, was only a small part of the PEK programme. The other studies that were performed as part of the PEK programme showed that homeopathic treatment is cheaper than conventional treatment, and that many patients treated with homeopathy show greater improvement than with conventional treatment. Furthermore, homeopathically treated patients experience less side effects and hospitalisation in homoeopathically treated patients is less.

The Swiss authorities did not welcome the studies of the PEK programme with positive outcomes for homeopathy. A preliminary report (dated March 14, 2005) of the PEK commission with the recommendation that homeopathy should remain part of the Swiss health insurance was omitted after political pressure. The Swiss homeopathic practitioners who wanted to publish the positive data of the PEK programme were forbidden to do so. The economist who was working on the project was fired and his data were withdrawn. The Swiss homeopathic practitioners are still in the possession of this economic evaluation that the PEK committee was not willing to publish (see http://www.dzvhae.com/portal/pics/abschnitte/030605064159_antrag_svha.pdf). These figures show that homeopathic treatment can reduce the general costs of medical treatment considerably.

Conclusion

Earlier analysis investigating the specific effects of homeopathy has shown that existing evidence for the efficacy of homeopathy can compete with evidence for conventional medicine. From the Lancet paper it can be concluded that the placebo-controlled clinical trials on homeopathy are of higher quality and are less heterogeneous than conventional studies. In addition, the decrease in effect size with increasing quality of the studies might be less for homeopathy than for conventional medicine. A third conclusion that can be drawn from the paper is that homeopathy is specifically effective for upper respiratory tract infections. According to the methods of the science philosopher Popper, which are the basis of the current double-blind placebo-controlled trials, evidence is now provided that homeopathy is more than a placebo effect: the hypothesis that swans are all white is falsified, as soon as the first black swan is found.

The paper, as published in the Lancet, by a scientist who is biased and not objective, has tried to come to a different conclusion. In doing so he has shown graphs that are not coherent with the conclusion drawn from these graphs, he has not identified which 8 studies were part of the sub-analysis and he has not taken into account, nor mentioned, the criticisms of the PEK commission itself. According to the commission, the heterogeneity as found in the sub-analysis of 8 studies is caused by the heterogeneity of the selection of the studies itself.

The editors of the Lancet have made a critical mistake in accepting the paper for publication since the readers are not able to follow the arguments that lead to the conclusions. The statement of the editors in the same edition of the Lancet that this publication should mean the end of homeopathy is biased; in fact, the paper proves the opposite.

For more than 15 years now, discussion and research effort have been directed towards proving that homeopathy is nothing more than a placebo effect. However, the arguments and analysis that have emerged, purporting to prove that this is the case, have failed to do so and have, at the same time, called into question the efficacy of conventional medicine just as much.

Contributors:

References

1 Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JAC, Egger M. Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homeopathy and allopathy. Lancet 2005;366:726-32
2 Kleijnen J, Knipschild P, Riet G ter. Clinical trials of homeopathy. BMJ 1991; 302:316-23
3 Linde K e.a. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trial. The Lancet 1997;350:834-43
4 Vandenbroucke JP. Medical journals and the shaping of medical knowledge. Lancet 1998;352:2001-6
5 Treffers PhDA, Rinne-Albers MAW. Geen plaats voor selectieve serotonineheropnameremmers (SSRI’s) bij depressieve kinderen en adolescenten. Ned Tijdschr Geneesk 2005;149:1314-7
6 Moncrieff J, Kirsch I. Efficacy of antidepressants in adults. BMJ 2005;331:155-7
7 Ezzo J, Bausell B, Moerman DE, Berman B, Hadhazy V. Reviewing the reviews. How strong is the evidence? How clear are the conclusions? Int J Technol Assess Health Care. 2001 Fall;17(4):457-66
8 Linde K, Jonas WB, Melchart D, Willich S. The methodological quality of randomized controlled trials of homeopathy, herbal medicines and acupuncture. Int J Epidemiol 2001; 30:526–31
9 Arnal-Laserre MN. Préparation à l’ accouchement par homéopathie: Expérimentation en double insu versus placebo. Académie de Paris. Université René Descartes. 1986. Score 90, result positive
10 Maiwald L, Weinfurtner T, Mau J, Connert WD. Therapie des grippalen Infekts mit einem homöopathischen Kombinationspräparat im Vergleich zu Acetylsalicylsäure. Arzneimittelforschung 1988;38:578-82. Score 80, result positive
11 Wiesenauer M, Gaus W. Wirksamkeitsnachweis eines Homöopathikums bei chronischer Polyarthritis. Aktuel Rheumatol 1993;18:159-162. N=176. Jadad/IV = 80/79. Result positive
12 White A, Slade P, Hunt C, Hart A, Ernst E. Individualised homeopathy as an adjunct in the treatment of childhood asthma: a randomised placebo controlled trial. Thorax 2003; 58:317-321
13 Attena F, Toscano G, Agozzino E, del Giudice N. A randomized trial in the prevention of influenza-like syndromes by homeopathic management. Rev Epidémiol Santé Publ 1995; 43: 380–82.
14 Ferley JP, Poutignat N, Azzopardi Y, Charrel M, Zmirou D. Evaluation en médecine ambulatoire de l'activité d'un complexe homéopathique dans la prévention de la grippe et des syndromes grippaux. Immunol Med 1987; 20: 22–28
15 Rottey EED, Verleye GB, Liagre RLP. Het effect van een homeopathische bereiding van micro-organismen bij de preventie van griepsymptomen: een gerandomiseerd dubbel-blind onderzoek in de huisartspraktijk. Tijdschr Int Geneeskunde 1995; 11: 54–58

Back