|Evidence of clinical efficacy of homeopathy: a meta-analysis of clinical trials
|Cucherat M, Haugh M C, Gooch M, Boissel J P
To establish, using a systematic review and meta-analysis, whether there is any evidence from randomised controlled clinical trials (RCTs) of the efficacy of homeopathic treatment in patients with any disease.
MEDLINE was searched from 1968 to June 1998, EMBASE from 1985 to June 1998, BIOSIS Previews from 1970 to June 1998, PsycINFO from 1967 to 1995, CINAHL from 1983 to 1995, and AMED from 1986 to 1995; the British Library Stock Alert Service and SIGLE were also searched (search dates unclear). Additional material was obtained by examining the reference lists of selected papers, handsearching of homeopathic journals and conference abstracts, and by contacting colleagues and pharmaceutical companies. There were no language restrictions.
Study designs of evaluations included in the review
RCTs. Only RCTs with a clearly-defined primary outcome measure were included. All were double-blind.
Specific interventions included in the review
Homeopathic treatments compared to placebo or a treatment without active constituent. A treatment was considered to be homeopathic if the dilution was greater than 3C (one molecule of the original principle in 1E6 molecules of solvent) or if it was presented as homeopathic by the manufacturer. Treatments in included studies were: Hepar sulfuris calcareum D4, Caulophyllum (5 degrees C), fixed mixed grass pollens (30 degrees C), opium (15 degrees C), Raphanus (15 degrees C) plus opium (15 degrees C), traumel ointment, fixed Oscillococcinum (2 trials), aconit (4 degrees C), intraarticular traumel R, Calendula, rheumaselect, individualised treatment (3 trials), euphorbium compositum S nasal spray, and bronchiselect.
Participants included in the review
Patients with any disease, including boils and pyodermia, dystocia, active hay fever, post-surgery ileus (2 trials), acute ankle sprains, influenza-like syndrome (2 trials), post-operative pain agitation, knee joint haematoma, second- and third-degree burns, rheumatoid arthritis, headache, acute childhood diarrhoea, allergic asthma, chronic sinusitis and bronchitis.
Outcomes assessed in the review
The outcome measures were different for each included study and included the following: healing time, success within 2 hours of treatment, visual analogue scale of overall symptom intensity (2 trials), delay to the first stool (2 trials), composite criteria of treatment success, recovery rate within 48 hours of treatment, sedation within 15 minutes, joint mobility, composite criteria of treatment success, change in mean attack frequency over the course of the trial, duration of diarrhoea, cumulative score, length of productive cough and a multiple end point (rate of patients affected and duration of the disease).
How were decisions on the relevance of primary studies made?
The authors do not state how the papers were selected for the review, or how many of the reviewers performed the selection.
Assessment of study quality
The only criterion for quality used for selection was adequate concealment of treatment allocation (by a suitable randomisation method). The authors do not state how the papers were assessed for quality, or how many of the reviewers performed the quality assessment.
The data were extracted using a data extraction form that was designed and tested by 4 reviewers with homeopathic and non-homeopathic backgrounds.
For each trial, data were extracted by 2 independent reviewers, and any disagreements were resolved by discussion with a third. All 11 authors took part in this process.
Data were extracted for the categories of study identification and year of publication, disease setting, homeopathic treatment, primary outcome, number of patients evaluated and randomised, results (p value), blinding, lost to follow-up (%), and type of placebo used.
Methods of synthesis
How were the studies combined?
The combination of the significance levels (p values) was used to pool the studies. The authors used 8 methods for calculating the p value: the sum of logs, the sum of Z, the weighted sum of Z, the sum of t, the mean Z, the mean p, the count test and the logit method. The results from the most conservative results are presented. A two-sided approach was adopted because of the format of the tested hypothesis. When possible, the p value was recalculated using the most powerful test suitable for the nature of the outcomes tested in the individual trials, i.e. Fisher's exact test for the binary outcomes and Student's t-test for the continuous outcomes.
Publication bias was assessed by adding fictive non significant trials to the real trials. Trials were added until the combined p value became greater than 0.05.
How were differences between studies investigated?
The authors do not report any tests for homogeneity, but several sensitivity analyses were performed subgrouping by quality and also assessing the effects of blinding on the results. The sensitivity analyses, to assess the effects of quality criteria, divided the trials into 4 groups on the basis of their quality from low to high: single-blind or unblinded randomised trials; randomised double-blind trials; randomised double-blind trials, with less than 10% of patients lost to follow-up; and randomised double-blind trials, with less than 5% of patients lost to follow-up.
The authors also analysed separately those trials evaluating fixed and individualised prescriptions, since these are two important homeopathic approaches.
Results of the review
Sixteen studies were included in the review with 5,180 participants. One of the trials had 3 treatment groups so there was a total of 17 comparisons.
Eleven of the 17 comparisons (65%) gave statistically-significant results in favour of the homeopathic treatment. A statistically non significant trend in favour of the placebo was observed in 3 comparisons. The combined p value for the 17 comparisons was highly significant (p=0.000036). However, sensitivity analysis showed that the p value tended towards a non significant value (p=0.08) as trials were excluded in a stepwise manner, based on their level of quality.
The addition of 63 fictive, non significant comparisons was needed to obtain a p value greater than 0.01, and 155 comparisons were needed to obtain a p value greater than 0.05; this suggested that publication bias is unlikely.
The authors state that there is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies.
The authors stated the research question and inclusion and exclusion criteria. The research question was extremely broad. It is possible it was too broad to be addressed in a meta-analysis. The literature search was reasonably thorough and the authors tested for possible publication bias.
The method used to assess the quality of the included studies was not stated, but issues of study and methodological quality were addressed and further analysed in sensitivity analyses. Trials were excluded if the quality was unacceptable. The authors detail the data extraction process, but there is no report on how the articles were selected, or who performed the selection and quality assessment.
The data extraction is reported in tables and discussed in the text of the review. Due to the disparate nature of the included studies, the studies were combined using p values which only state whether the findings of an individual study are statistically or not statistically significant. Clinical evidence of treatment effectiveness cannot be determined from the results of these studies. Further sensitivity analyses were performed to assess the effects of differences in study quality and methodology.
The authors' conclusions appear to follow from the results, but should be viewed with great caution because of limitations in the quality of the review process.
Implications of the review for practice and research
Practice: The authors did not state any implications for practice.
Research: The authors state that it could be worthwhile to perform well-designed and suitably sized RCTs. A clinically-relevant primary outcome should be clearly defined and the intention-to-treat principle should be respected.
Cucherat M, Haugh M C, Gooch M, Boissel J P. Evidence of clinical efficacy of homeopathy: a meta-analysis of clinical trials. European Journal of Clinical Pharmacology 2000; 56(1): 27-33
Other publications of related interest
This additional published commentary may also be of interest. Crawford CC, Jonas WB. Homeopathy needs more study. FACT 2001;6:126-7.
Subject indexing assigned by NLM
Homeopathy /methods /statistics & Humans; Randomized Controlled Trials as Topic /methods /statistics & numerical data; numerical data
Date bibliographic record published
Date abstract record published
This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.