To investigate whether the protective efficacy of bacille Calmette-Guerin (BCG) against tuberculosis (TB) decreases with time since vaccination.

The authors used all RCTs from previous reviews of the topic. Additional unspecified searches did not reveal further trials. All publications from each trial were retrieved. [A:The authors have provided additional information regarding their literature search. The authors believed the previous reviews were extensive and well documented and it was not necessary to duplicate the searches.]

Study designs of evaluations included in the review

Randomised controlled trials (RCTs). These had to have reported the BCG efficacy against TB disease in uninfected (purified protein derivative negative) humans; have allocated the vaccine and placebo groups randomly (or systematically for early trials); and have published sufficient information to allow the efficacy to be estimated for different periods of time after vaccination. The follow-up for the trials ranged from 11 to 23 years.

Specific interventions included in the review

Vaccination with BCG vaccine or placebo.

Participants included in the review

The participants were uninfected (purified protein derivative negative) humans ranging in age from newborns to adults.

Outcomes assessed in the review

The efficacy of BCG was based on the number of cases of TB per person-year of observation. Thhis was assessed during the first 2 years and after 2 years, during the first 10 years and after the first 10 years, and overall. The annual change in the rates of TB in the controls were also derived. Mortality was not an outcome.

How were decisions on the relevance of primary studies made?

The authors do not state how the papers were selected for the review, or how many of the authors performed the selection.

The authors do not report a method for assessing validity. [A:The authors clarify that the studies were not assessed for quality because the included studies were among the earliest controlled trials and the earliest ones (which started in the 1930s) certainly did not use random allocation in the sense that it is done now.]

Two readers extracted the data from each trial independently, and any disagreements were resolved by discussion.[A:The authors state that they did not use all randomised controlled trials from previous reviews of the topic. They only included studies for which enough information was published to allow the estimation of efficacy for different periods of time after vaccination. Similarly, only a subset of the studies provided sufficient information to allow estimated effects before 2 years and after 10 years.]

How were the studies combined?

The rate ratio (RR) for unvaccinated, compared with vaccinated, individuals was calculated for each trial, along with 95% confidence intervals (CIs). Fixed-effect and random-effects models were used to calculate the average effect of BCG after 10 years. RRs were also derived for the annual change in rates of TB in the controls. To assess changes in the effect of BCG over time, the average annual change in the effect of BCG was calculated; the effect after 2 years was compared with the effect in the rest of the follow-up, and the effect after 10 years was compared with that in the first 10 years of follow-up.

The authors also calculated correlations between the rate of change of efficacy and overall efficacy, weighting each study both equally and according to the inverse of the variance of the log RR for the effect of the vaccine.

How were differences between studies investigated?

The authors used the methods of DerSimonian and Laird (se Other Publications of Related Interest) to test the heterogeneity of the log RRs for the effect of BCG, and for time trends in the effect of BCG between the trials. The authors stated that the number of studies was too small to enable a meaningful exploration of the reasons for between-study variability.

Ten RCTs with 252,119 participants were included.

The number of cases per person-years in the placebo group was 955 out of 1,610,256. The number of cases per person-years in the vaccine group was 752 out of 2,499,500.

In 3 of the 10 trials, the RR increased by 1 to 18% per year, corresponding to increased efficacy; in the remaining 7 trials, the RR decreased from 5 to 14% per year, corresponding to decreased efficacy.

There was highly significant heterogeneity in the effect of BCG between trials, both before and after 2 years: the chi-squared values were 25.55 (d.f.=6, p=0.0003) and 120.8 (d.f.=6, p<0.0001), respectively.

There was no significant heterogeneity between trials in the effect of BCG after 10 years: chi-squared was 7.77 (d.f.=6, p=0.26)).

There was statistically-significant heterogeneity in the change in the effect of BCG between time periods, both for the 2-year and 10-year comparisons: the chi-squared values were 18.24 (d.f.=6, p=0.006) and 13.04 (d.f.=6, p=0.042), respectively.

The rates of change in efficacy were not related to overall efficacy. The efficacy also varied between trials in the first 2 years after vaccination, at more than 2 years after vaccination, and in the first 10 years after vaccination.

In trials of efficacy 10 years after vaccination, the average efficacy over all the trials was 14% (95% CI: -9, +32) when using a random-effects model, and 12% (95% CI: -5, +27) when using a fixed-effect model. The fixed-effect model gave an RR of 1.14 (95% CI: 0.95, 1.37, p=0.15) for unvaccinated, compared with vaccinated, individuals.

The authors state that even where BCG gives good initial protection, this may wane with time at a rate corresponding to a 5 to 14% annual decrease in the RR of TB in unvaccinated, compared with vaccinated, individuals. Published studies do not provide evidence of a protective effect of BCG against TB more than 10 years after vaccination.

The authors conducted a reasonable review and summarised their results in several tables and graphs. It was not possible to determine whether the review may have missed relevant studies since the authors did not use the literature searches from previous reviews of this topic; they only made reference to this previous work.

The inclusion criteria for the individual trials were clearly stated. However, there was no quality review of the included studies. Also, there were no details of how the included studies were chosen, e.g. how many reviewers were involved, whether there was independent assessment, and whether the reviewers were blinded.

The authors discovered very significant heterogeneity between the trials and were unable to combine the results for 3 of the 4 stated outcome measures. This, along with the lack of an audit trail for the literature review and the lack of a quality review of the included trials, suggest that the conclusions of this review should be viewed with caution.

Practice: The authors do not state any further implications for practice

Research: The authors state that studies are required to examine the effect of revaccination with BCG, and that these should be conducted in more than one country.

DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177-88.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.