This review concluded that pre-operative chemotherapy plus surgery results in a significant survival benefit over surgery alone in patients with non-small-cell lung cancer. The results of the review reflect the evidence reviewed, but the limitations of the included studies, and the exclusion of 5 eligible trials because of inadequate reporting of the outcome data, reduce the reliability of the conclusions.

To assess the effectiveness of pre-operative chemotherapy in the treatment of non-small-cell lung cancer (NSCLC).

MEDLINE (1966 to 2005) and the Cochrane Library were searched using a published search methodology for finding randomised controlled trials (RCTs). This search was supplemented by a search of four trial registers (cancer.gov, the meta Register of Controlled Trials, ClinicalTrials.gov and the UK Coordinating Committee on Cancer Research National Register of Cancer Trials) for ongoing and unpublished trials. The authors stated that conference proceedings of large international oncology meetings were searched, but did not specify which conferences. The bibliographies of publications and book chapters were handsearched. No language restrictions were applied to the search.

RCTs with concealed allocation of treatment and comparable treatment arms (to avoid confounding) were eligible for inclusion.

Trials that compared pre-operative chemotherapy plus surgery with surgery alone were eligible for inclusion. Trials in which chemotherapy was given as a second-line treatment were excluded, as were trials of patients who had had a previous cancer. Details of the chemotherapy regimen (all platinum-based, either cisplatin or carboplatin) and of other treatments offered to patients (e.g. radiotherapy) in the included studies were given in the review.

Studies of participants with NSCLC were eligible for inclusion. The participants in the included studies were aged from 32 to 83 years and almost 80% were men with a good performance status. Half of the participants had squamous cell carcinoma, and a quarter had adenocarcinoma of the lung.

The authors did not state any inclusion criteria relating to the outcomes, but did state that the primary outcome was overall survival. A priori secondary outcomes were local recurrence-free survival, distant recurrence-free survival and overall recurrence-free survival.

The authors did not state how the papers were selected for the review, or how many reviewers performed the selection.

Validity was assessed by comparing the methodology of each trial to the Consolidated Standards of Reporting Trials (CONSORT) statement. The authors did not state how the validity assessment was performed.

Where possible, the authors included analyses based on all randomised patients. Where available, the hazard ratio (HR) and variance were extracted from the trial; alternatively, they were extracted from other summary statistics or from Kaplan-Meier curves. Where possible, several methods were used to estimate the HR, to test the reliability of these methods. The authors calculated absolute differences in survival and the percentage improvement in survival.

Two reviewers independently extracted the data and any discrepancies were resolved by consensus.

The trials were combined in a meta-analysis using a fixed-effect model for the primary analysis and random-effects models for the sensitivity analyses.

The χ² test was used to test for statistical heterogeneity between the studies. Consistency between trials was measured using the I² statistic. Publication bias was assessed using Begg's and Egger's tests.

Twelve RCTs (1,310 participants) were included in the review. Seven RCTs (988 participants) were included in the meta-analysis. The other 5 studies did not have sufficient survival data available for abstraction and so were excluded.

Quality indicators (randomisation procedure, allocation concealment, patient exclusions, calculation of end points) were insufficiently described in most of the studies. Only one study reported using intention-to-treat analysis.

There was no evidence of publication bias, as assessed by Begg's test (p=0.29) or Egger's test (p=0.94), and Begg's funnel plot was fairly symmetrical.

There was good evidence that chemotherapy plus surgery had a more beneficial effect on survival than surgery alone (HR 0.82, 95% confidence interval, CI: 0.69, 0.97, p=0.02). There was no evidence of statistical heterogeneity (p=0.98).

The absolute improvement in 5-year survival associated with chemotherapy plus surgery versus surgery alone was 6% (increasing from 14 to 20%). The absolute benefit differed by disease stage: 4% for stage Ia, 6% for stage Ib, 7% for stage IIa and IIb, 6 to 7% for stage IIIa and 3 to 5% for stage IIIb.

When excluding the 2 largest studies, chemotherapy plus surgery was still found to have a more beneficial effect on survival than surgery alone, but the effect did not reach statistical significance (HR 0.79, 95% CI: 0.59, 1.08, p=0.14). There was no evidence of statistical heterogeneity (p=0.90).

Sensitivity analyses found no evidence of a difference in effectiveness when trials were grouped according to type of chemotherapy used (platinum plus vinca alkaloid/etoposide; platinum plus taxane; other platinum regimen; p=0.99) or additional therapy (post-operative radiotherapy or chemotherapy; p=0.58).

Disease-free survival was analysed in 3 studies (457 participants). Chemotherapy plus surgery had a more beneficial effect on survival than surgery alone (HR 0.78, 95% CI: 0.52, 0.99, p=0.04); there was some evidence of statistical heterogeneity (p=0.07).

There is a significant survival benefit for patients with NSCLC who receive pre-operative chemotherapy plus surgery over those who receive surgery alone.

The objective of the review was clear, as were the inclusion criteria for the participants and interventions. Study design inclusion criteria were not sufficiently specific, as it is not clear how the authors assessed whether differences between treatment arms could have confounded the results of a trial. Furthermore, the authors stated that RCTs with concealed allocation were eligible for inclusion, but it appears that none of the included trials reported allocation concealment. Given this, and as no outcome inclusion criteria were stated, subjective decisions could have been made at the point of choosing trials to include. The search appears thorough, and the authors made an effort to identify trials which were ongoing, had not been published, or had only been published in abstract form, thus minimising the likelihood of publication bias. The authors noted that the quality of the included studies was generally poor, and that this could have impacted upon the reliability of the synthesised results. The authors also noted that the exclusion of 5 eligible trials because of limitations in the reporting of outcome data was a limitation of the review.

The methods used for the synthesis seem broadly appropriate, although the authors' decision to use a fixed-effect model for the primary analysis prior to investigating heterogeneity is unusual. The method used to calculate CIs for the absolute differences in survival does not seem appropriate.

While the results of the review reflect the evidence reviewed, the limitations of the included studies, and the exclusion of 5 eligible trials because of inadequate reporting of the outcome data, reduce the reliability of the conclusions of the review.

Practice: The authors did not state any implications for practice.

Research: The authors stated that an ongoing individual patient data meta-analysis is needed to address whether pre-operative chemotherapy offers benefits in certain patient groups.

Not stated.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.