This review found that the Hybrid Capture II assay has greater accuracy than repeat Pap smear for the diagnosis of cervical intraepithelial neoplasia of grade II or worse at the ASCUS threshold for equivocal Pap smears. Differences between tests just reached statistical significance. Poor reporting of review methods and a limited assessment of study quality mean it is difficult to assess the reliability of these conclusions.

To determine the accuracy of human papillomavirus (HPV) DNA testing to detect histological confirmed cervical intraepithelial neoplasia of grade II or worse (CIN2+) in women with equivocal results on a previous Papanicolaou (Pap) smear. The review also assessed the accuracy of repeat cytology at thresholds of atypical cells of undetermined significance (ASCUS), low-grade squamous intraepithelial lesion (LSIL) and high-grade squamous intraepithelial lesion (HSIL) in studies in which the Pap smear was repeated, and looked at differences in accuracy between these two triage tests.

MEDLINE, EMBASE and the Cochrane Library were searched for studies published between 1992 and 2002. The search terms, which were reported, did not include a diagnostic filter. Reference lists of relevant articles and tables of content of relevant journals were screened for additional studies.

Study designs of evaluations included in the review

Inclusion criteria were not defined in terms of the study design.

Specific interventions included in the review

Studies that assessed an HPV DNA detection test were eligible for inclusion. The threshold for positivity was that specified by the manufacturer. If studies reported data on cytologic triage tests then these data were also included. Three thresholds were considered for an abnormal cytologic test: ASCUS or worse, LSIL or worse, and HSIL or worse. The following HPV DNA tests were used in the included studies: ViraPap, ViraType, Hybrid Capture I and II, and Hybrid Capture Tube. Most studies used conventional Pap smear as the cytologic test, but some studies assessed the ThinPREP liquid-based technique.

Reference standard test against which the new test was compared

Studies in which the reference standard was colposcopy and colposcopy-directed biopsies, with or without endocervical curettage for histological confirmation, were eligible for inclusion. Histologic examination of material obtained by colposcopy-directed biopsy, loop excision or endocervical curettage was considered to provide compete ascertainment of the disease status. The threshold of CIN2+ was used.

Participants included in the review

Studies in which women had an initial Pap smear of the uterine cervix with atypical squamous/glandular cells of unspecified significance (ASCUS/AGUS) were eligible for inclusion. The patients were recruited from colposcopy clinics, or from gynaecologic services to which women had been referred because of abnormal smear test results. In some studies women had repeated atypical cytology. Some studies excluded women with a history of cervical intraepithelial neoplasms, cervical surgery or biopsy,

Outcomes assessed in the review

Inclusion criteria were not defined in terms of the outcomes. The outcomes reported in the review were the sensitivity and specificity.

How were decisions on the relevance of primary studies made?

The authors did not state how the papers were selected for the review, or how many authors performed the selection.

The authors did not state that they assessed validity. However, data were extracted on methodological variables: procedures for reference standard verification and blinding of interpreters to other test results.

For some studies only selected data were included. Data were only extracted on the presence of oncogenic and high-risk HPV types. Data were extracted as 2x2 tables of test performance at each of the tests and thresholds considered. The sensitivity, specificity, positive predictive value, negative predictive value, test positive rate, prevalence of disease (presence of CIN2+), and positive and negative likelihood ratios were calculated for each set of 2x2 data. To allow an assessment of the difference in accuracy of the HPV DNA test and the repeat Pap smear tests, the ratios of the sensitivity and specificity of these two tests were calculated. The authors did not state how many reviewers performed the data extraction.

How were the studies combined?

Measures of accuracy and individual ratios of sensitivity/ and specificity for the two tests were pooled. If heterogeneity was present (p<0.10), random-effects models were used to pool the data. In the absence of heterogeneity, fixed-effect models were used to pool data weighted according to the inverse variance. Publication bias was assessed using the Egger test.

How were differences between studies investigated?

Differences between the studies were assessed visually through forest plots. Heterogeneity was assessed statistically using Cochran's Q test. The effects of study characteristics on estimates of accuracy was assessed by subgroup analysis.

Fifteen studies reported in 17 articles were included (5,454 women).

Six studies stated that histologic interpretation was blinded to the triage test results; in 3 studies expert histologists reviewed the histopathologic diagnosis.

Triage by HPV testing (15 studies).

The sensitivity ranged from 26.7 to 100% and specificity from 48 to 97%. The pooled sensitivity and pooled specificity were 84% (95% confidence interval, CI: 78, 91) and 73% (95% CI: 63, 83), respectively. There was considerable heterogeneity between the studies for both sensitivity and specificity for all HPV tests. The test that showed the least amount of heterogeneity was the Hybrid Capture II assay (8 studies). The pooled sensitivity was 95% (95% CI: 93, 97) and the pooled specificity 67% (95% CI: 58, 76) for this assay.

Triage by repeat cytology at threshold ASCUS or worse (9 studies).

The sensitivity ranged from 60 to 85% and the specificity from 45 to 72%. The pooled sensitivity and pooled specificity were 82% (95% CI: 74, 84) and 58% (95% CI: 50, 66), respectively. There was considerable heterogeneity between the studies for both sensitivity and specificity.

Triage by repeat cytology at threshold LSIL or worse (7 studies).

The sensitivity ranged from 20 to 59% and the specificity from 78 to 96%. The pooled sensitivity and pooled specificity were 46% (95% CI: 34, 57) and 89% (95% CI: 82, 96), respectively. There was considerable heterogeneity between the studies for sensitivity; heterogeneity was reduced for specificity.

Triage by repeat cytology at threshold HSIL or worse (2 studies).

The sensitivities were 35% and 25%. The specificities were 97% and 99%.

Comparison between tests (4 studies).

The pooled ratio of the sensitivity of the Hybrid Capture II assay to the sensitivity of repeat cytology at a threshold of ASCUS or worse was 1.16 (95% CI: 1.04, 1.29), suggesting significantly greater sensitivity of this assay. The ratio of specificities was 1.05 (95% CI: 0.96, 1.15), suggesting similar specificity. When using the threshold of LSIL or worse, the sensitivity ratio was 1.69 (95% CI: 1.54, 1.85) and the specificity ratio was 0.71 (95% CI: 0.64, 0.80). When using the threshold of HSIL or worse, the sensitivity ratio was 2.80 (95% CI: 2.43, 3.31) and the specificity ratio was 0.57 (95% CI: 0.44, 0.74).

There was no overall evidence of publication bias.

The Hybrid Capture II assay had greater sensitivity and similar specificity compared with repeat Pap smear testing, based on a threshold of ASCUS for the detection of CIN2+ in women with equivocal cytologic results.

The review addressed a clear question that was supported by defined inclusion criteria. The literature search was adequate but no attempts were made to identify unpublished studies. Although the authors assessed publication bias, the validity of these tests in the area of diagnostic accuracy studies is questionable and so the possibility of publication bias remains. No details of the review process were reported and it was unclear whether appropriate steps were taken to minimise bias. A formal quality assessment was not undertaken and only two aspects of methodological quality were considered in the results and discussion; the validity of the individual studies on which the authors' conclusions were based is therefore unclear.

The methods used to synthesise the results were acceptable and heterogeneity was investigated. However, a more robust analysis based on more complex methods for the meta-analysis of test accuracy studies might have been more reliable. Subgroup analyses were carried out but these should be interpreted with some degree of caution given the large number of variables investigated and the relatively small number of included studies. Overall, the conclusions are supported by the data presented, although it should be noted that the difference in sensitivity between the two tests only just reached statistical significance. The conclusions should be interpreted with some degree of caution, owing to the possibility of publication bias, the possibility of bias in the review process, and the failure to fully assess study quality.

Practice: The authors stated that the Hybrid capture assay is a better triage method than repeat cytology for women with ASCUS.

Research: The authors did not state any implications for research.

European Commission (Directorate of SANCO, Luxembourg), Europe against Cancer program; Federal Services for Scientific, Cultural and Technical Affairs of the Federal Government, Belgium.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.