To quantify the performance of colposcopy for the diagnosis of squamous intraepithelial lesions (SIL) of the cervix, in order to provide a standard against which other new technologies can be easily compared.

Cumulated Index Medicus was searched from 1960 to 1965, and MEDLINE from 1966 to 1996, for publications in the English language. The keywords used were 'colposcopy', 'diagnosis', 'positive predictive value', 'negative predictive value', 'likelihood ratio' and 'receiver operating characteristic curve'.

Study designs of evaluations included in the review

The authors did not specify what study designs were included.

Specific interventions included in the review

Studies using colonoscopy for diagnosis were eligible for inclusion in the review.

Reference standard test against which the new test was compared

The included studies had to compare the diagnostic performance of colonoscopic impression with the results of the reference standard test, colonoscopically directed biopsy.

Participants included in the review

The included studies had to be of women referred to colonoscopy clinics with abnormal Papanicolaou (Pap) smears. Studies were excluded if referrals were based on abnormal bleeding. No further details of included participants were reported.

Outcomes assessed in the review

Studies were included if they reported raw data, in the form of contingency tables, sufficient to calculate overall summary statistics.

The outcomes calculated assessed in the review were the sensitivity (weighted by sample size), specificity (weighted by sample size), positive and negative predictive values (PPV), and the likelihood ratios positive and negative. Two different diagnostic thresholds were used: one distinguishing normal tissue from all abnormalities and one distinguishing normal tissue, atypia, and low-grade SIL from high grade SIL and cancer.

How were decisions on the relevance of primary studies made?

The authors do not state how the papers were selected for the review, or how many of the reviewers performed the selection.

The authors do not state that they assessed validity.

The authors do not state how the data were extracted for the review, or how many of the reviewers performed the data extraction.

Data were extracted on: study authors; number of participants; the proportions of biopsy positive and biopsy negative patients with each diagnosis derived from colonoscopic impression.

How were the studies combined?

Weighted (by sample size) mean sensitivities and specificities were calculated for both diagnostic thresholds. Mean positive and negative predictive values were calculated using Bayes' theorum.

Summary receiver operating characteristic (sROC) curves were created, using the logistic transform method of Littenberg and Moses (see Other Publications of Related Interest), to show the overall performance of the test across all settings presented in the literature. The summary statistic, area under the ROC curve, was also calculated for comparison purposes.

How were differences between studies investigated?

The authors do not state how differences between the studies were investigated.

Eighty-six studies were identified by the search, of which 9 (6,281 patients) met the inclusion criteria and were included in the review.

The weighted mean sensitivity and specificity of colposcopy in distinguishing normal from abnormal tissue (atypia, low-grade SIL, high grade SIL, and cancer) were 0.96 and 0.48, respectively (based on data from all nine studies). The mean PPV was 0.82 and the mean NPV was 0.79. For distinguishing normal tissue, atypia, and low-grade SILs from high-grade SILs and cancer, the weighted sensitivity and specificity were 0.85 and 0.69, respectively (based on data from eight studies). The mean PPV was 0.57 and the mean NPV was 0.85. Likelihood ratios for 'negative' test findings or findings of atypia ranged from 0.002 to 0.38, for low-grade SIL from 0.38 to 1.19, and for high-grade SIL or cancer from 2.49 to 18.7. The sROC curves showed reasonable performance, with an area under the curve of 0.80 for distinguishing normal from abnormal tissue, and 0.82 for distinguishing normal, atypia and low-grade SIL from high-grade SIL or cancer.

Colposcopy compared favourably with other medical diagnostic tests in terms of sensitivity, specificity and area under the ROC curve. New diagnostic methods for the cervix can be compared with colposcopy using these quantified values.

This review was based on a clear study objective and well defined inclusion criteria. The primary data were tabulated, and the data synthesis was appropriate for the data presented. However, more detail of participant characteristics would have improved the generalisability of the review.

The limitation of the literature search to MEDLINE and English language articles is likely to have resulted in incomplete retrieval of the available published data. In addition, no attempt to identify unpublished data was reported and the potential impact of publication bias was not assessed, although some issues relating to publication bias were discussed..

The validity of the included studies was not assessed and no details of the review methodology were reported. It is therefore possible that methodological flaws in the primary studies and/or weaknesses in the conduct of the review may have introduced biases with the potential to prejudice the results of the review.

The authors' conclusions are reasonable, based upon the data presented.

The authors make no specific recommendations for future research or current practice.

The Whitaker Foundation.

Littenberg B, Moses LE. Estimating diagnosis accuracy from multiple conflicting reports: a new meta-analytic method. Med Decis Making 1993;13:313-21.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.