To assess whether the clinical effect reported in randomised controlled trials (RCTs) of homeopathic remedies is equivalent to that reported for placebo.

A previous review was utilised (see Other Publications of Related Interest no.1) in which MEDLINE and EMBASE had been searched up to 1990. Additional searches of MEDLINE were carried out by an information specialist for the years 1966 to August 1995, using the search terms homeop*, homoeop*, homeopathy, homoeopathy, and alternative medicine. The authors also searched MEDLINE using additional search terms (not listed), and searched EMBASE from 1989 to October 1995. Further material was sought by contact with homeopathic researchers and manufacturers. Several other databases were accessed, such as those of the Woodward Foundation (USA), CISCOM, AMED, Hom-Inform, IDAG (the Netherlands), the Central Council of Research in Homeopathy (India), as well as several individual collections. Conference proceedings, abstract booklets, homeopathic books and the bibliographies of the retrieved articles were handsearched. Studies reported in any language were considered.

Study designs of evaluations included in the review

Placebo-controlled trials of parallel design, with explicit statement that random allocation was used or that a double-blind design was used, were eligible for inclusion. These had to have been published as a journal article, abstract, thesis, conference proceeding, unpublished report, book section or monograph, with sufficient data to allow the outcome rates to be estimated per trial arm. Single case studies were excluded.

Specific interventions included in the review

Homeopathic treatment, classified into four categories (classical, clinical, complex and isopathic) and into three levels of dilution (low, medium and high). The specific agents used varied across the studies and were listed in the paper. The control participants received placebo.

Participants included in the review

People with various conditions who were being treated with homeopathy were included, as were those who were considered to be at risk of certain conditions and were receiving homeopathy as a preventive measure The conditions included the following: allergic asthma, pollinosis, warts, minor burns, pyodermia, skin lesions, dermatoses, anal fissure, diarrhoea, gastritis, cholecystopathia, irritable bowel, sprains, haemarthrosis, cramps, dental neuralgia, migraine, seasickness, aphasia, stroke, menopause, vaginal discharge, premenstrual syndrome, childbirth, mastodynia, cystitis, cough, upper respiratory tract infection, pharyngitis, running nose, otitis media, sinusitis, rheumatoid arthritis, osteoarthritis, fibrositis, myalgia, agitation (connected with surgery or anaesthesia), post-operative ileus, tooth extraction, prevention of complications (connected with surgery or anaesthesia), haematomae, varicosis, conjunctivitis, and overweight. Studies recruiting healthy volunteers for homeopathic 'provings' or assessment of physiology were excluded.

Outcomes assessed in the review

The outcomes varied across the trials and were selected for inclusion into the meta-analysis according to a hierarchy for preferred outcomes, in terms of their relevance. The first preference was any predefined main outcome measure, defined as the outcome on which the sample size was calculated. The second preference was the patient's global assessment of improvement, and the third preference was the physician's global assessment of improvement. The fourth preference comprised those measures considered to be important by the reviewers.

The authors also attempted to investigate the reproducibility of the effects of homeopathic treatment. This required at least three independent replications on the same clinical condition, with the same model of homeopathy, remedy, outcome measurement, and a similar population.

How were decisions on the relevance of primary studies made?

The studies were assessed independently by two reviewers, one of whom made the final decisions in cases of disagreements. However, it was also implied that a third reviewer would have been consulted, if necessary, to resolve disagreements about selection. Prediscussion reliability of the selection process was assessed using the k-statistic on a random selection of half of the trials.

Two quality scores were used. The first assessed random allocation, double-blinding, and the reporting of withdrawals (see Other Publications of Related Interest no.2). The second (developed by the authors) covered the adequacy of concealment, the handling of withdrawals, baseline group comparability, and the adequacy of inferential statistics. Two quality scores were applied to each included trial by two independent reviewers, and any disagreements were resolved by discussion. Inter-observer reliability before discussion was checked with the intraclass correlation coefficient for both validity scores.

The data were extracted by two independent reviewers onto pretested forms and entered into a spreadsheet. The categories of data extracted were listed in the paper.

How were the studies combined?

Odds ratios (ORs) with associated 95% confidence intervals (CIs) were calculated for each trial on an intention to treat basis, then pooled using both fixed-effect and random-effects models. In trials where continuous data were reported, the differences between the means divided by the pooled standard deviation were converted into ORs.

How were differences between studies investigated?

A chi-squared test for heterogeneity was used to assess the variance in effect sizes among the trials. The effects of publication bias (estimated by plotting the OR against the inverse of the variance for each individual trial), study quality, publication source, outcome preference, and a 'worst-case' situation (comprising the above plus only medium- and high-potency studies), were assessed using sensitivity analyses.

Eighty-nine trials with 9,283 participants were included.

The overall OR (89 trials) was 2.45 (95% CI: 2.05, 2.93) in favour of homeopathy (random-effects model). Results from the various sensitivity analyses indicated that this finding was robust. The summary OR for trials categorised as high quality (26 trials) was 1.66 (95% CI: 1.33, 2.08). When the results were adjusted for publication bias (89 trials), the summary OR was 1.78 (95% CI: 1.03, 3.10). However, there was statistically-significant publication bias (p=0.033).

Tests for reproducibility.

Four trials on the effects of a single remedy on seasonal allergies had a pooled OR for ocular symptoms of 2.03 (95% CI: 1.51, 2.74) at 4 weeks. Five trials on post-operative ileus had a pooled mean effect size difference of -0.22 standard deviations (95% CI: -0.36, -0.09) for flatus, and -0.18 standard deviations (95% CI: -0.33, -0.03) for stool (both p<0.05).

The results of this meta-analysis were not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo. However, we found insufficient evidence from these studies to suggest that homeopathy is clearly efficacious for any single clinical condition.

Overall this was a rigorously conducted and well-presented systematic review. The research questions, selection criteria for primary studies, quality assessment, and methods of pooling data were explained clearly. The search strategy was thorough, and an estimation of the impact of publication bias was included. Details of the primary studies were tabulated. However, the results should be treated with some caution due to the fact that the trials included in the meta-analysis differed markedly in terms of the participants, interventions and outcomes. In addition, the results of the statistical test for heterogeneity were not reported, although the authors state that this was carried out. The authors' conclusions are appropriate given the evidence arising from this review.

The authors state that further research on homeopathy is warranted provided it is rigorous and systematic.

1. Kleijnen J, Knipschild P, ter Riet G. Clinical trials of homoeopathy. BMJ 1991;302:316-23. 2. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17:1-12. 3. Meta-analysis of homeopathy trials [correspondence]. Lancet 1998;351:365-8. 4. Wittwer H. Meta-analysis on homeopathy trials. Homeopath Links 1997;10:181-2.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.