The authors concluded that systemic perioperative chemotherapy provided benefit in the treatment of patients with resected stage IV colorectal cancer, but that the results should be interpreted cautiously; further research is needed. Given the uncertainties surrounding the pooling of the trials and statistical methods used, the authors' advice to interpret their findings with caution should be heeded.

To assess the effectiveness of adding perioperative chemotherapy to surgery for the treatment of resected stage IV colorectal cancer.

PubMed, the Cochrane Library, and LILACS were searched from 1980 to January 2009 with no language restrictions. Search terms were reported. Abstracts from the American Society of Clinical Oncology, reference lists of original articles and relevant reviews were scanned.

Randomised controlled trials (RCTs) that compared surgery plus perioperative chemotherapy versus surgery alone in patients with pathologically-proven colorectal cancer who were to undergo curative resection were eligible for inclusion.

The outcomes of interest were overall survival and recurrence-free survival, disease-free survival, or progression-free survival. Some trials reported toxicity, intra-hepatic recurrence, and time to failure.

The median/mean age of included patients was under 65 years; most were male. The number of metastases varied between one and six. Chemotherapy regimens included 5-fluorouracil/folinic acid alone, or 5-fluorouracil/folinic acid plus additional drugs: FOLFOX 4 (comprising folinic acid, fluorouracil and oxaliplatin), mitomycin C or floxuridine (FUDR). Drugs were administered systemically or by continuous hepatic artery infusion (or both). Surgical procedures included anatomic and non-anatomic resections.

The authors did not state how many reviewers screened studies for inclusion.

Two reviewers assessed trial quality according to Cochrane Collaboration criteria of randomisation, allocation concealment, blinding, intention-to-treat (ITT) analyses, and pre-specification of primary outcome. Criteria were scored as adequate, inadequate or unclear.

Two reviewers independently extracted or calculated hazard ratios (HRs) and 95% confidence intervals (CIs). Where only survival probabilities at each time-point were presented in Kaplan-Meier curves, crude log hazard ratios and variances were calculated using the individual patient data (IPD) reconstruction technique. Where survival curves were not available, relative risks (RRs) and 95% confidence intervals were calculated from incidence data. Additional data were sought from primary authors where necessary.

A fixed-effect model (or random-effects model where there was evidence of statistical heterogeneity) was used to combine hazard ratios and 95% confidence intervals. Statistical heterogeneity was assessed using the Q statistic and I².

Subgroup analyses were undertaken for type of therapy (intra-arterial versus systemic). Sensitivity analyses were undertaken to include only high quality trials; the influence of each trial was also assessed by omitting one study at a time.

Publication bias was assessed using Begg's funnel plot.

Eight RCTs (1,058 evaluable patients; 533 undergoing surgery, 525 undergoing surgery with perioperative chemotherapy) were included in the review. Sample sizes ranged from 11 to 364 patients. Where reported, 59.3 to 97.5% of patients received their assigned treatment, 494 underwent surgery, and 391 completed chemotherapy. Where reported, 848 patients underwent resection. Most trials reported methods of randomisation and withdrawal, but none were reported as double-blind. Three trials were considered high quality. Follow-up ranged from 18 to 144 months (where reported).

Overall survival: The addition of perioperative chemotherapy to surgery did not significantly reduce overall mortality (I²=26%). Subgroup analyses did not significantly alter the results, although heterogeneity was no longer evident for trials using systemic chemotherapy (two RCTs).

Recurrence-free survival: Patients receiving perioperative chemotherapy in addition to surgery showed statistically significant improvements compared with surgery alone (HR 0.77, 95% CI 0.67 to 0.88; I²=25%). Subgroup analyses by type of therapy and analyses including only high quality trials did not significantly alter the findings. However, there was evidence of substantial heterogeneity in the subgroup analysis using intra-arterial chemotherapy (I²=54%); when a random-effects model was used for this data set, the results were no longer statistically significant (HR 0.72, 95% CI 0.51 to 1.02, five RCTs).

Toxicity: The most frequent grade 3 and 4 toxicities were generally mild and acceptable (as listed in the review). There were 10 treatment-related deaths (12%) in two trials.

There was evidence of publication bias, but this was no longer evident when one small trial was excluded.

Systemic perioperative chemotherapy provided benefit in the treatment of patients with resected stage IV colorectal cancer. However, the results should be interpreted with caution due to limitations of the evidence; further research is needed.

The inclusion criteria of the review were appropriate; it was not clear whether the selection stage of the process was undertaken in duplicate, so reviewer error and bias could not be ruled out. The literature search included appropriate sources to identify published and unpublished data, and was not restricted by language, which reduced the potential for missed data. There was some evidence of publication bias, but it was unclear how robust these findings were as only a small number of trials were included in the review.

Appropriate methods were used to assess trial quality; only three trials were reported to be of high quality. However, the authors did investigate quality in the analyses and it did not seem to significantly affect the findings. It was difficult to interpret the review at times as definitions were not provided. There was some evidence of clinical heterogeneity in the therapy regimens. The authors acknowledged that the trials used different definitions for survival outcomes. Log hazard ratios and their variances would have been the most appropriate summary statistics for survival data, but these data were not available for all trials. It was unclear how appropriate it was to combine hazard ratios and relative risks, which did not take into account censoring and time to an event, or the effect this may have had on the results. It was also unclear how the authors used the IPD reconstruction technique when they stated that only published data were used in the review. One systemic therapy trial was included in analyses for overall survival and recurrence-free survival, but no weighting was given to the trial; the authors did not provide any explanation for this.

Given the uncertainties surrounding the pooling of the trials and statistical methods used, the authors' conclusions to interpret their findings with caution should be heeded, and their recommendation for further research seems appropriate.

Practice: The authors did not state any implications for practice.

Research: The authors stated that their findings should be confirmed by an IPD-based meta-analysis. They also stated that further clarification is needed to determine which patient group would benefit by perioperative chemotherapy, whether applied preoperatively or postoperatively, and which drug or combination of drugs would be most effective. Greater efforts are also needed to improve chemotherapeutic regimens to minimise toxicities and improve treatment compliance.

P.U.R.E. Research Programme, State Ministry for Innovation, Science, Research and Technology of NorthRhine-Westphalia, Germany; the FoRUM Foundation of the Ruhr-University Bochum, Germany.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.