This review assessed the diagnostic accuracy of tests for primary bladder cancer. The authors concluded that none of the tests under investigation reached a level of sensitivity that would be acceptable in clinical practice. The review was well carried out, but the strength of the results was hindered by the study design and the lack of detail on patient characteristics.

The objective was to assess the diagnostic performance of cytology and several urine-based tests for primary bladder cancer.

MEDLINE and EMBASE were searched from January 1990 to November 2001 for German and English articles; the search terms were reported. Reviews were also checked for further references.

Study designs of evaluations included in the review

Diagnostic cohort and case-control study designs were included in the review. No inclusion criteria were specified.

Specific interventions included in the review

Studies were included if they assessed at least one of the following index tests: cytology, bladder tumour antigen (BTA), BTA stat, BTA TRAK, NMP22, telomerase and fibrin degradation product (FDP). Cut-off (threshold) values were presented for BTA TRAK and NMP22.

Reference standard test against which the new test was compared

Studies were only included if cystoscopy and/or histopathology were used as the reference standard.

Participants included in the review

Participant characteristics were not specified as part of the inclusion criteria. However, it appears that a mixture of urological and other patients were included in the retrieved studies. No further details were given.

Outcomes assessed in the review

Studies that presented a 2x2 table providing information to calculate sensitivity or specificity measures (on a per-patient analysis), and had reported a diagnosis of primary bladder cancer, were included in the review.

How were decisions on the relevance of primary studies made?

One reviewer selected potentially eligible studies from the abstracts. Two people independently reviewed the retrieved articles to determine final inclusion. Both stages used predefined criteria. Any disagreements were resolved by consensus.

The studies were assessed according to the following criteria: study design (cohort or case-control); type of control group; clear description of the study population (e.g. age or gender distribution, signs and symptoms or description of bladder cancer stage); description of index and reference tests (e.g. cut-off point for positivity and techniques used); consecutive patient selection verification by the reference standards; and blinded assessment of the index and reference tests. The two reviewers who decided upon the relevance of the primary studies carried out the validity assessment.

Two reviewers extracted the data using a standardised score form. Data were collected on methodological quality, industry sponsorship, clinical characteristics pertaining to a specific tumour marker and method of sample collection, disease grade according to the WHO classification, and outcomes. The authors calculated sensitivity and specificity from the raw data as reported in the primary studies.

How were the studies combined?

The results of the studies were plotted in receiver operating characteristic (ROC) space to explore the variation between studies and whether variation may be due to threshold differences. An analysis was conducted (using a bivariate random-effects model) to pool results for the sensitivity and specificity of the various tests. There was no reported weighting of the individual studies.

How were differences between studies investigated?

A multivariate analysis was carried out to explore additional reasons (e.g. study characteristics and quality markers) for variations in sensitivity and specificity between the studies.

Forty-two studies, comprising 5,706 patients, were included in the review: 20 cohort studies, 21 case-control studies and 1 case-control/cohort study. The number of patients entered for each test was as follows: cytology, 3,444; BTA, 715; BTA stat, 1,300; BTA TRAK, 829; NMP22, 2,290; telomerase, 855; and FDP, 157.

Results of the ROC analysis showed a greater variation in sensitivity than specificity for the cytology, BTA and NMP22 tests; the opposite was noted for BTA TRAK and telomerase. A threshold effect was noted for telomerase and this was subsequently confirmed in a correlation analysis. Cytology had a pooled specificity of 94% (95% confidence interval, CI: 90, 96), which was statistically significantly higher than for other tests: BTA 79% (95% CI: 70, 86), BTA stat 75% (95% CI: 64, 84), BTA TRAK 65% (95% CI: 45, 81), telomerase 86% (95% CI: 71, 94), or NMP22 78% (95% CI: 72, 83). The results for telomerase showed the most favourable sensitivity data at 75% (95% CI: 71, 79) and was statistically different (P=0.04) from NMP22 (67%, 95% CI: 60, 73), BTA TRAK (66%, 95% CI: 62, 71), BTA (50%, 95% CI: 30, 65) and cytology (55%, 95% CI: 48, 62), but not statistically significant (P=0.13) from BTA stat (70%, 95% CI: 66, 74). A meta-analysis of studies employing the FDP test was not possible, but the sensitivities and specificities from the two available studies were, respectively, 78% and 76%, and 91% and result not given.

In the multivariate analysis, the only significant results (P<0.05) were that case-control designs showed a lower sensitivity (48%, 95% CI: 39, 57) than cohort designs (61%, 95% CI: 52, 69, P=0.04). None of the other variables showed a significant effect.

The results of the validity assessment showed that there were clear descriptions of demographic profile in 40% of the studies, the reference test in 7% of the studies, and the marker test in 86% of the studies. All but one of the cohort studies verified patients by the reference standard. Blind interpretation of studies was noted in 33% and use of a consecutive sample in 11% of studies.

Cytology had the best specificity and telomerase the best sensitivity. None of the tests reached a level of sensitivity that would be acceptable to clinical practice.

This review addressed a topical research question and appeared to be thoroughly conducted. However, the selection of studies by only one reviewer at the abstract stage, along with language restriction and the failure to assess the effect of reliance upon published studies, could have introduced bias into the results. Verification or incorporation bias cannot be ruled out (although not relevant to the case-control study design). The validity assessment procedure was explicit, but it was unclear whether this was carried out independently. The validity results regarding descriptions of demographic profile and reference test, along with the extent of consecutive sampling and blinded interpretation, were disappointing.

The pooling of sensitivity and specificity data by tests was probably appropriate, given that the authors made cautious assumptions and also investigated heterogeneity. The generalisabilty of the results is potentially limited by the broad inclusion criteria for the participants and by the reliance (in over half of the included studies) upon a case-control design, which also has the potential to contain the greatest sources of bias. On the basis of the data presented, the authors' conclusions on the unacceptability of the findings in practice are well founded.

Practice: The authors stated that none of the tests under investigation in this review reached a level of sensitivity that would be acceptable to clinical practice.

Research: The authors stated that possible areas of research include the testing of telomerase in cohort studies of patients with suspected primary bladder cancer, in order to represent clinical practice. In addition, they proposed that combining cytology with telomerase in patients with symptoms of bladder cancer is a potentially valuable area for exploration.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.