This review considered the accuracy of a commercial line probe assay for the detection of rifampicin-resistant tuberculosis. The authors concluded that the assay has high accuracy when used in culture isolates, but that there is a paucity of data on its accuracy when applied directly to clinical samples. These conclusions are likely to be reliable for the specimens studied.

To evaluate the accuracy of the INNO-LiPA Rif. TB (LiPA) commercial line probe assay for the detection of rifampicin (RIF)-resistant tuberculosis (TB) in culture isolates and clinical specimens.

MEDLINE, EMBASE, BIOSIS Previews and the Science Citation Index were searched from 1990 to 2004; the search terms were reported. An internet search (Google Scholar) was conducted in December 2004, and experts in the field and the kit manufacturer were contacted for additional articles and unpublished data. No language restrictions were applied to the search, but studies not available in English or Spanish were excluded from the data extraction process.

Study designs of evaluations included in the review

The included studies were required to evaluate a minimum of 10 RIF-sensitive and 10 RIF-resistant specimens.

Specific interventions included in the review

Studies evaluating the accuracy of the commercial LiPA test kit in culture isolates or clinical specimens were eligible for inclusion. The clinical specimens used in the included studies were sputum, bronchial aspirate, urine, tissue biopsy, cerebrospinal fluid, faeces, skin exudates and gastric juice aspirate.

Reference standard test against which the new test was compared

The included studies were required to compare LiPA with a reference standard, (proportion method, radiometric BACTEC 460 method, or minimum inhibitory concentration method). The majority of the studies used either the proportion method or the radiometric BACTEC 460 method as the reference standard.

Participants included in the review

No inclusion criteria for the study participants were set.

Outcomes assessed in the review

Studies that assessed accuracy (sensitivity and specificity) were eligible for inclusion.

How were decisions on the relevance of primary studies made?

Two reviewers independently screened titles and/or abstracts of all retrieved citations. Any disagreements were resolved by consensus. The full texts of all relevant articles were examined.

The methodological quality of the included studies was assessed using criteria derived from the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool (see Other Publications of Related Interest). The criteria used were: prospective enrolment of consecutive patients; comparison with an appropriate reference standard; blind and independent comparison with the reference standard; and partial or complete verification of test results using the reference standard. The quality assessment process was not specifically described, but it appears to have formed part of the data extraction process.

One reviewer extracted the data from all included studies using a piloted data extraction form. A subset of 5 of the 15 included studies were independently data extracted by a second reviewer, and any disagreements were resolved by consensus. The level of agreement between raters on sensitivity and specificity values was measured. For each study, the sensitivity and specificity of LiPA were calculated, with 95% confidence intervals (CIs), from 2x2 contingency tables. Other data extracted included the proportion of RIF-resistant samples that were determined to be multi-drug resistant TB.

How were the studies combined?

Pooled estimates of the sensitivity and specificity of LiPA, along with 95% CIs, were calculated. Summary receiver operating characteristic (ROC) curves, fitted using the regression method of Moses et al. (reference given) were presented, along with estimates of the Q* index and the area under the curve (AUC) and their standard errors (SEs).

How were differences between studies investigated?

Between-study heterogeneity for sensitivity and specificity was examined visually using forest plots. Subgroup analyses were presented for the accuracy of LiPA in culture isolates and in clinical specimens. The results of chi-squared tests were illustrated on the forest plots, but were not mentioned elsewhere in the paper.

Fifteen studies, evaluating 1,738 specimens, were included in thereview.

Methodological quality.

Of the 15 included studies, only two reported blinding of the researchers to the results of the reference standard and/or LiPA. No study prospectively enrolled consecutive patients. All studies had complete verification of LiPA results with a reference standard.

Inter-rater agreement for sensitivity and specificity was 80%.

Accuracy of LiPA in culture isolates (14 studies).

The sensitivity ranged from 82 to 100% and the specificity ranged from 92 to 100%. The AUC, derived from the summary ROC curve, was 0.99 (SE=0.0017) and the Q* index was 0.97 (SE=0.005).

Accuracy of LiPA in clinical specimens (4 studies).

The sensitivity ranged from 80 to 100%, while the specificity was 100% for all 4 studies. One study stated that 13 of the 60 samples tested were indeterminate because of failure at the polymerase chain reaction stage; these were excluded from the analysis.

Rifampicin resistance as a marker for multi-drug resistant TB (4 studies).

Four studies determined the number of RIF-resistant samples that were also isoniazid resistant (the criteria for multi-drug resistant TB). On average, 91% of RIF-resistant samples were also isoniazid resistant.

The authors stated that the cost per sample of a commercial LiPA kit is $45, rising to as much as $116 when import and transport costs are considered.

LiPA is a highly sensitive and specific test for the detection of RIF resistance in culture isolates. It appears to have lower sensitivity when used in clinical specimens, but there is a paucity of data in this area. More evidence is required before LiPA can be used to detect multi-drug resistant TB in at-risk populations.

The article reports an apparently well-conducted review with a clearly stated aim and relevant inclusion criteria. Extensive literature searches were reported, but the restriction of included studies to those available in English or Spanish might have resulted in the omission of some relevant data (it was stated that 6 articles were excluded on this criterion). Measures were taken to avoid the introduction of error and bias in the review process. The methodological quality of the included studies was assessed using some criteria relevant to diagnostic accuracy studies. However, although the QUADAS tool is cited, only some of the criteria contained in this tool were assessed; no reasoning for the selection of criteria was provided and the inclusion of some further criteria might have been helpful, particularly given the authors' noted concerns about the possible exclusion of indeterminate test results and the effects on reported accuracy estimates.

The methods used to summarise the data were broadly appropriate. However, no formal heterogeneity testing was described, although some chi-squared values were reported and overall pooled estimates of sensitivity and specificity were presented on forest plots. The authors commented that the proportion of Mycobacterium TB samples that were RIF-resistant was 67%, which is higher than prevalence rates seen in routine practice settings, and that specimens in the review are therefore not representative of specimens that a TB laboratory might actually receive. The authors' conclusions follow from the data presented and are likely to be reasonably robust for the samples studied.

Practice: The authors stated that LiPA may have a potential role in ruling in and ruling out RIF resistance. They further stated that the diagnostic accuracy of LiPA needs to be interpreted cautiously in areas of low prevalence.

Research: The authors stated that additional studies are needed to establish the accuracy of LiPA when used directly on clinical specimens (an application that could reduce the time to diagnosis from 2 to 6 weeks to 24 to 48 hours). Study designs should include sputum samples from patients suspected of having multi-drug resistant TB. Indeterminate test results, the proportion of RIF-resistant specimens meeting multi-drug resistant TB criteria, patients' sputum smear status, and turnaround time for diagnosis should also be reported. Studies are also needed to determine the clinical usefulness of rapid diagnosis of RIF-resistant TB in terms of its effect on clinical outcomes and transmission rates. Finally, studies are needed to assess the cost-effectiveness of LiPA in comparison with conventional drug susceptibility testing.

Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3:25.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.