|A meta-analysis of the diagnosis performance of the direct agglutination test and rK39 dipstick for visceral leishmaniasis
|Chappuis F, Rijal S, Soto A, Menten J, Boelaert M
The authors concluded that the direct agglutination test and the rK39 test have good to excellent diagnostic accuracy and seem comparable to one another. The limited search, poor reporting of review methods, and differences between the studies make it difficult to confirm the reliability of the authors' conclusion.
To compare the performance of the direct agglutination test and rK39 dipstick for the diagnosis of visceral leishmaniasis.
MEDLINE was searched for all relevant articles published from January 1986 to December 2004; the search terms were reported. Further studies were obtained by citation tracking of reviews and primary studies.
Study designs of evaluations included in the review
Diagnostic accuracy studies were eligible for inclusion in the review. Studies were not restricted by quality. The primary studies included phase I, II and III studies.
Specific interventions included in the review
Studies evaluating the direct agglutination test and the rK39 immunochromatographic assay used on serum or blood samples for visceral leishmaniasis were eligible for inclusion. Studies of the rapid version of the direct agglutination test (the fast agglutination screening test) were excluded. The included studies evaluated freeze dried and aqueous antigen types of agglutination tests, and Kalazar Detect and other types of rK39 dipsticks. The studies were conducted in South Asia, East Africa and elsewhere.
Reference standard test against which the new test was compared
The authors stated that they included studies if the reference standard was considered 'adequate' to correctly classify the target condition. The reference standards in the selected studies included: microscopical examination of lymph node, bone marrow, or spleen aspirate; detection of parasites in culture or by polymerase chain reaction; or clinical response to antileishmanial drugs in patients with clinically suspected disease who had positive serology results.
Participants included in the review
Studies were included if current clinical visceral leishmaniasis was the target condition (not leishmanial infection or past visceral leishmaniasis). Studies evaluating the tests in patients coinfected with the human immunodeficiency virus were excluded. The primary studies included various types of control participants: healthy non-endemic controls, healthy endemic controls, patients with cross-reacting diseases, and patients with the same clinical syndrome as confirmed cases but with leishmaniases excluded by tests with high negative predictive value.
Outcomes assessed in the review
Studies were included if the absolute numbers of true positive, false positive, true negative and false negative observations were reported or could be calculated from the presented data.
How were decisions on the relevance of primary studies made?
The authors did not state how the papers were selected for the review, or how many reviewers performed the selection.
Assessment of study quality
The validity of the individual studies was assessed using the 14-item QUADAS (Quality Assessment of Diagnostic Accuracy Studies) scale for diagnostic studies. Two reviewers independently assessed validity, with any disagreements resolved by a third reviewer where necessary.
The authors did not state how the data were extracted for the review, or how many reviewers performed the data extraction.
The number of true positives, false positives, true negatives and false negatives were either extracted or calculated for each study. Sensitivities, specificities and diagnostic odds ratios (DORs) were calculated, along with their 95% confidence intervals (CIs).
Methods of synthesis
How were the studies combined?
Meta-analyses for sensitivity and specificity were carried out using logistic regression models accounting for overdispersion. A meta-analysis of DORs was carried out using the random-effects model of DerSimonian and Laird, with studies weighted using the Mantel-Haenszel method. Publication bias was assessed by plotting the estimated sensitivity and specificity against the number of cases and controls in each study.
How were differences between studies investigated?
Heterogeneity was examined by conducting subgroup meta-analyses stratified by study phase, sample size, study quality (score 7 or less versus score >7), geographical region, species of Leishmania, type of direct agglutination test antigen, brand of dipstick and type of controls.
The authors investigated any potential threshold effect by plotting the log odds of a positive test result in diseased participants against the log odds of a negative test result in non-diseased participants.
Results of the review
Forty-three studies were included in the review: 13 studies (n=3,795) evaluated the rK39 dipstick and 30 studies (n=5,574) evaluated the direct agglutination test.
The pooled sensitivity for the rK39 dipstick was 93.9% (95% CI: 87.7, 97.1). One study seemed inconsistent with this estimate (67.2%, 95% CI: 54.1, 78.2). The pooled specificity was 95.3% (95% CI: 88.8, 98.1); the estimated specificity was heterogeneous across the included studies. The pooled sensitivity for the direct agglutination test was 94.8% (95% CI: 92.7, 96.4) and the pooled specificity was 97.1% (95% CI: 93.9, 98.7); the estimated specificity was heterogeneous across the included studies.
Sensitivity appeared to be higher amongst studies conducted in South Asia (97.1%, 95% CI: 91.7, 99.0) than amongst those conducted in East Africa (79.0%, 95% CI: 46.7, 94.2).
Specificity estimates were highly influenced by the type of controls, with studies using healthy controls showing high specificities. The pooled specificity in participants with clinically suspected disease in phase III studies was 90.6% (95% CI: 66.8, 97.9) for the rK39 dipstick and 85.9% (95% CI: 72.3, 93.4) for the direct agglutination test.
The DOR was 128 (95% CI: 26, 629) for the rK39 dipstick and 76 (95% CI: 13, 433) for the direct agglutination test.
The diagnostic performance of the direct agglutination test and the rK39 dipstick for visceral leishmaniasis is good to excellent and seem comparable.
This review reported the findings of what appeared to be an adequate systematic review, though some relevant details were not reported. The authors stated study inclusion criteria relating to the participants, interventions and outcomes, but did not appear to have limited selection by study design or reference standard. Only the MEDLINE database was searched, which might have resulted in some relevant studies being missed. No specific attempts to minimise either publication or language bias were reported. It was unclear how many reviewers were involved in the selection of studies or extraction of data, so there may have been the potential for error or bias to influence these processes. Two reviewers assessed the quality of the included studies using a validated checklist and the results were reported. However, the potential impact of study quality did not appear to have been incorporated into the synthesis of these studies. Similarly, the authors reported investigating publication bias, but did not present the results of this investigation.
In general, the analysis seemed appropriate, with the authors checking for any potential threshold effect amongst the included studies and investigating sources of heterogeneity. However, it was unclear how much of the heterogeneity could be explained by the variables examined. The conclusion about the relative effect of the two different tests was not based on direct comparison and thus cannot be definitive. The limited search, incomplete reporting of review methods, and differences between the studies make it difficult to confirm the reliability of the authors' conclusion.
Implications of the review for practice and research
Practice: The authors stated that the review results support the inclusion of these diagnostic tests in country policies. They stated that there is an urgent need for the tests to be more widely used. In addition, since the performances of the direct agglutination test and the rK39 dipstick were comparable, the choice should be made on the basis of other criteria such as cost, feasibility and sustainability. It should be noted that the review did not directly compare the diagnostic accuracy of these two tests, thus any comments about relative accuracy may not be reliable.
Research: The authors stated that the limited sensitivity of the rK39 dipstick in Sudan and the suboptimal specificity of these serological tests when used for patients with clinically suspected visceral leishmaniasis mean that there is a need for further research and development of better diagnostic tools. Adopting quality criteria for diagnostic accuracy studies would increase the reliability of the results and enable more accurate policy recommendations to be made more rapidly. Each new type of rK39 dipstick should be evaluated in each endemic region before it is included in diagnostic algorithms.
Chappuis F, Rijal S, Soto A, Menten J, Boelaert M. A meta-analysis of the diagnosis performance of the direct agglutination test and rK39 dipstick for visceral leishmaniasis. BMJ 2006; 333: 723
Subject indexing assigned by NLM
Agglutination Tests /methods /standards; Enzyme-Linked Immunosorbent Assay /methods /standards; False Negative Reactions; False Positive Reactions; Humans; Leishmaniasis, Visceral /diagnosis; Sensitivity and Specificity
Date bibliographic record published
Date abstract record published
This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.