You are here: Home Research Clinical research Experimental studies Comprehensive systematic reviews

Comprehensive systematic reviews

Comprehensive systematic reviews and meta-analyses of the whole homeopathic literature

The first comprehensive systematic review of homeopathy research was published in the British Medical Journal in 1991. Of the 105 trials with interpretable results, 81 indicated positive results, which included RCTs that received high quality ratings for randomization, blinding, sample size, and other methodological criteria. The authors, three orthodox epidemiologists, came to the following conclusion: "The amount of positive evidence even among the best studies came as a surprise to us. Based on this evidence we would readily accept that homeopathy can be efficacious, if only the mechanism of action were more plausible". And "The evidence presented in his review would probably be sufficient for establishing homeopathy as a regular treatment for certain indications".

- Kleijnen J, Knipschild P, ter Riet G (1991). Clinical trials of homeopathy, British Medical Journal, 302: 316-323[PubMed]

In 1996, a comprehensive review by the European Commission Homeopathic Medicine Research Group, later published in a scientific journal (Cucherat), identified 184 controlled clinical trials. They selected the highest quality randomized placebo controlled trials, which included a total of 2,001 patients. Boissel used a rather unusual technique of combining p-values of the selected trials to answer the question: 'Does homeopathy have an effect?' and concluded that homeopathy was more effective than placebo though "the strength of this evidence is low because of the low methodological quality of the trials".
 A full report and update of this review, with at least two new trials added to the meta-analysis and a total of 2,579 patients, found “some evidence that homeopathic treatments are more effective than placebo” (P < 0.001); it concluded that “the strength of this evidence is low because of the low methodological quality of the trials”.
- Boissel JP, Cucherat M, Haugh M, Gauthier E (1996). Critical literature review on the effectiveness of homeopathy: overview of data from homeopathic medicine trials. In: Homeopathic Medicine Research Group, Report of the, Commission of the European Communities, Directorate-General XII–Science, Research and Development, Directorate E–RTD Actions: Life Sciences and Technologies–Medical Research, Brussels 1996; Chap.11: 195-210
- Cucherat M, Haugh MC, Gooch M, Boissel JP (2000). Evidence of clinical efficacy of homeopathy–A meta-analysis of clinical trials. European Journal of Clinical Pharmacology, 56:27–33 [PubMed]

A comprehensive systematic review and meta-analysis published in The Lancet in 1997 included 186 placebo controlled studies of homeopathy, from which data for analysis could be extracted from 89. The overall mean odds ratio for these 89 clinical trials was 2.45 (95% confidence interval 2.05–2.93) in favour of homeopathy, which means that the chances of homeopathy giving benefit were 2 to 3 times greater than those of placebo. Even when considering a "worst-case" scenario in which only high-quality studies, of high or medium dilutions, published in MEDLINE-listed journals, and with predefined measures of primary outcome were analysed, the pooled odds ratio was 1.97 and remained statistically significant. The main conclusion was that the results "were not compatible with the hypothesis that the effects of homoeopathy are completely due to placebo". In further analysis, the authors clarified that higher quality trials were less likely to be positive than those of lower quality – which is comparable to the conclusion from similar analyses in conventional medicine –, though the difference from placebo remained statistically significant until analysis was restricted to just five trials. There was insufficient volume of evidence to enable conclusions to be drawn about what homeopathic treatment is effective in which diagnosis.
- Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, Jonas WB (1997) Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet, 350: 834-843 [PubMed]

Linde and Melchart published another review in 1998, based on the same set of studies as the Linde et al. study of 1997. Using more stringent inclusion criteria, they concentrated on 32 trials of homeopathy and concluded in favour of an effect of homeopathy, but stressed the weakness of the evidence and the need for more high-quality trials of homeopathy.
- Linde K, Melchart D (1998) Randomized controlled trials of individualized homeopathy: a state-of-the-art review. Journal of Alternative and Complementary Medicine, 4:371-388 [PubMed]

In 2005 an other comprehensive review was published in The Lancet comparing 110 placebo controlled trials of homeopathy and 110 matched trials of conventional medicine. Homeopathy and conventional medicine showed a similar positive treatment effect overall. Twenty-one homeopathy trials and 9 in conventional medicine were judged of higher methodological quality. From these, the results of 14 unspecified "larger trials of higher quality" (8 homeopathy, 6 conventional medicine) were analysed. The mean odds ratio was 0.88 (95% CI, 0.65-1.19) for the 8 homeopathy trials, and 0.58 (95% CI, 0.39-0.85) for the 6 conventional medicine trials. In this study, in contrast with the 1997 review, an odds ratio less than 1.0 indicated an effect greater than placebo. The authors concluded there was "weak evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions. This finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects".
The authors' analysis has been criticised to be prone to selection bias, especially when the set of 21 high-quality trials was reduced to 8 trials with large patient numbers. Obviously, the results depend on how the threshold for "large" studies was defined from these 21 studies. The overall results – and the conclusions drawn from them – change depending on which subset of homeopathic trials is analysed. The choice of other meaningful subsets could lead to the opposite conclusion. For example, four of the 21 trials dealt with preventing or treating muscle soreness and an earlier systematic review already showed that homeopathic medicines are probably not helpful in this condition. When restricting the analysis to the remaining 17 trials an overall statistically significant effect can be found. In addition, there is no external criterion why a "large" trial should have N=98 or more patients as defined by the authors (apart from being able to include a statistically non-significant trial with a sample size of exactly 98). For example, if the data set was split at a threshold of N=66, the median sample size of all 110 homeopathic trials, there had been a significant effect in favour of homeopathy. Surprisingly, the authors themselves highlight – but dismiss – the fact that 8 trials of homeopathy in upper respiratory tract infections have strongly positive findings overall. Clearly, the conclusions by this group of scientists under the direction of Professor Matthias Egger, well known for his anti-homeopathy standpoint, are not so definite as they have been reported and discussed.
- Shang A, Huwiler-Muntener K, Nartey L, Juni P, Dorig S, Sterne JA, Pewsner D, Egger M (2005). Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet, 366:726–32 [PubMed]
- Lüdtke R, Rutten AL (2008) .The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials. Journal of Clinical Epidemiology, 61:1197-1204 [PubMed]
- Rutten AL, Stolper CF (2008) The 2005 meta-analysis of homeopathy: the importance of post-publication data. Homeopathy, 97:169-177 [PubMed]

Share |