This well-conducted systematic review assessed the accuracy of phase-based tests for the diagnosis of tuberculosis. It concluded, in line with the data presented, that phage-based assays have high specificity but variable sensitivity.

To review studies on the accuracy of bacteriophage-based tests for the detection of Mycobacterium tuberculosis in clinical specimens. In particular, the review aimed to determine the overall accuracy of phage-based tests compared with the 'gold' standard of culture; the accuracy of phage-based tests compared with smear microscopy; and to investigate how the accuracy of phage-based tests varies by sputum smear status.

PubMed (1985 to November 2004), Web of Science (1985 to November 2004), EMBASE (1988 to November 2004), and BIOSIS Previews (1993 to November 2004) were searched; the search terms were reported. The reference lists of primary studies and review articles were screened and experts in the field were contacted for additional studies. Details of studies were also obtained from test manufacturers. The review was restricted to studies reported in the English language.

Study designs of evaluations included in the review

No inclusion criteria relating to the study design were specified. The included studies were diagnostic cohort studies and diagnostic case-control studies.

Specific interventions included in the review

Studies of phage tests were eligible for inclusion. The studies assessed both commercial and in-house assays. The commercial tests evaluated were FASTPlaque and PhageTek. All but one of the studies used a threshold of 20 or more plaques as a positive test; the other study classified any visible plaques as positive. All studies of commercial assays processed samples using a standard N-acetyl-L-cysteine-sodium hydroxide (NALC-NaOH) method.

Reference standard test against which the new test was compared

Studies that included culture (either liquid or solid media) as the reference standard were eligible for inclusion. The reference standards used in the included studies were BACTEC 460, Lowenstein Jensen (LJ) medium, BACTEC and LJ, and LJ or Amplified Mycobacterium tuberculosis Direct Test.

Participants included in the review

No inclusion criteria relating to the participants were specified. Most studies included patients with suspected pulmonary tuberculosis, although some studies included non-respiratory specimens in addition to sputa. Some studies included only untreated patients; others included both treated and untreated patients.

Outcomes assessed in the review

In order to be included in the review, the studies had to report sufficient data for the calculation of both the sensitivity and specificity.

How were decisions on the relevance of primary studies made?

Two reviewers independently screened titles and abstracts. Any disagreements were resolved through consensus.

The authors stated that the studies were assessed for methodological quality using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool (see Other Publications of Related Interest) but the assessed items reported were not those listed in the QUADAS checklist: study design (cross-sectional versus case-control), blinding (single-/double-blind versus unblinded interpretation of index test and reference standard results), and potential for verification bias (complete versus partial/differential verification of index test results by reference standards). The authors did not state how many reviewers performed the validity assessment.

One reviewer extracted the data using a piloted data extraction form. A second reviewer independently extracted the data from a subset of the included studies. Inter-rater agreement was reported to be 100%. Data on the sensitivity and specificity (with 95% confidence intervals) of phage-based assays using sputum culture on solid and/or liquid media as the reference standard were extracted. For an evaluation of the accuracy of phage-based assays in comparison with microscopy, studies that reported direct comparisons of sputum microcopy with phage-based assays (against a common reference standard) were evaluated. The majority of studies reported sensitivity and specificity in terms of the number of specimens rather than the number of individual patients.

How were the studies combined?

The pooled sensitivity and specificity were calculated using Meta-Disc (version 1.1) software. The joint distribution of sensitivity and specificity was summarised using a summary receiver operating characteristic (sROC) curve.

How were differences between studies investigated?

Heterogeneity was investigated using subgroup analyses and statistical methods (exact method not reported).

Thirteen studies (5,820 samples) reported in 12 articles are included. Eleven studies used a diagnostic cohort (or cross-sectional) design; two studies were diagnostic case-control studies.

Verification was complete in all studies. Only three studies reported a blind assessment of phage tests and reference standard results.

The sensitivity ranged from 21 to 94% (test for heterogeneity, p<0.001). Owing to the significant heterogeneity, sensitivity was not pooled. The specificity ranged from 83 to 100% and was greater than 90% in all but two studies. The in-house assays were as specific as the commercial tests.

Accuracy according to sputum smear status (5 studies).

Smear-positive samples tended to give higher estimates of sensitivity than smear negative samples: the sensitivity ranged from 29 to 87% in smear-positive samples and from 13 to 78% in smear-negative samples. The specificity ranged from 60 to 88% in smear-positive samples and from 89 to 99% in smear-negative samples.

Comparison between phage-based studies and sputum microscopy (n=10).

sROC plots suggested that phage assays have slightly higher accuracy than microscopy (the area under the sROC curve was 0.95 for phage assays and 0.86 for microscopy).

Phage-based assays have high specificity but variable sensitivity. The accuracy of phage-based assays is slightly higher than that of microscopy.

This was a well-conducted and clearly reported review. The review question was focused and supported by well-defined inclusion criteria. A detailed literature search, which included attempts to locate unpublished studies, was conducted. The review was restricted to English language studies, thus there is some potential for language bias: two studies were excluded because they were in languages other than English. Details of the review process were reported and some attempt was made to minimise errors and bias, although it would have been preferable for the data extraction to have been carried out entirely in duplicate or at least double checked. A formal quality assessment was undertaken and some of the results were reported. The methods used to combine the studies were appropriate and the analysis benefited from the inclusion of relevant forest plots and sROC curves. The authors conclusions are supported by the results presented.

Practice: The authors stated that 'because of the overall low sensitivity, the similarity of phage-based assays to sputum microscopy with respect to accuracy and the need for a fairly advanced laboratory infrastructure, phage-based assays cannot replace conventional diagnostic tests at this point'.

Research: The authors stated that further research is required to identify methods that can enhance the sensitivity of phage-based assays without compromising the high specificity.

Fogarty AIDS International Training Program, grant number I- D43-TW00003-16; National Institutes of Health, NIH/NIAID grant number R01 A1 34238.

Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Method 2003;3:25.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.