The authors concluded that in-house assays had superior accuracy to commercial kits for the detection of Helicobacter pylori infection in children. Methodological flaws in the review, together with heterogeneity in many of the data sets, meant that the pooled estimates of accuracy reported are of limited value. The authors' conclusion fails to consider variation in performance between individual commercial kits.

To assess the accuracy of antibody-based tests for the detection of Helicobacter pylori (H. pylori) infection in children.

Pubmed, EMBASE and LILACS were searched from January 1997 to May 2007 for English- and Spanish-language studies. Search terms were reported and included methodological terms relating to test accuracy studies. The bibliographies of identified studies were screened for additional articles.

Studies that compared antibody-based tests with a reference standard (culture, histology and/or urea breath test) for the diagnosis of H. pylori infection in children (aged 19 yrs or under) were eligible for inclusion. Prospective or retrospective cross-sectional or case-control studies with a minimum of 30 participants were included. Included studies were required to report numbers of true positives (TP), false negatives (FN), false positives (FP) and true negatives (TN) or positive and negative predictive values. The main outcome measures reported in the review were sensitivity and specificity, and positive and negative likelihood ratios (LRs). The antibody-based tests assessed by included studies were serum enzyme-linked immunosorbent assay (ELISA)-IgA, serum Western-blot and serum, urine and saliva ELISA-IgG. These included both in-house assays and commercial kits. The majority of studies included histology in the reference standard.

Studies were assessed independently by two experts. Disagreements were resolved by consensus. Where necessary, corresponding authors were contacted for further details.

The methodological quality of included studies was assessed using criteria based on the Standards for Reporting of Diagnostic Accuracy (STARD) studies statement. These criteria assessed: use of an appropriate, independent reference standard to establish diagnosis; blinding of those conducting and interpreting the antibody-based test to the results of the reference standard; recruitment of a prospective, consecutive sample; and verification bias.

The authors did not specify how many reviewers performed the quality assessment.

Data were extracted into an Excel spread sheet that recorded numbers of true positives (TP), false negatives (FN), false positives (FP) and true negatives (TN), and cross-checked for input errors. These data were used to estimate positive and negative predictive values, positive and negative LRs and diagnostic odds ratio (DOR) for each study. The authors did not specify how many reviewers were involved in the data extraction and checking process.

Estimates of sensitivity and specificity, with 95% confidence intervals (CIs) were calculated for each study and presented on forest plots, along with pooled estimates of sensitivity and specificity for each index test. Between-study heterogeneity was assessed using the Χ² statistic and explored using subgroup analyses of technique, immunoglobulin class and source of test (in-house or commercial). Summary receiver operating characteristic (SROC) curves were fitted for each index test.

Thirty eight studies (31 case-control and seven cross-sectional) reporting 68 data sets were included in the review. The median sample size was 111 (interquartile range 76 to 184). A total of 9,455 children were included in the analyses, 3,441 of which were H. pylori positive. Thirty two of the 38 studies were prospective. Only two studies reported measures to avoid verification bias. Eleven studies reported at least single-blinded interpretation of the index test.

Accuracy of ELISA:

The pooled estimates for serum IgA (seven studies, 579 participants) were: sensitivity 42.6 (95% CI: 36.4 to 49.0); specificity 90.9 (95% CI: 87.2 to 93.8); LR+ 4.4 (95% CI: 2.7 to 7.1); and LR- 0.60 (95% CI: 0.45 to 0.79). No significant between-study heterogeneity was identified.

The pooled estimates for serum IgG (42 studies, 5,632 participants) were: sensitivity 79.2 (95% CI: 77.3 to 81.0); specificity 92.4 (95% CI: 91.6 to 93.3); LR+ 10.2 (95% CI: 8.1 to 13.0); and LR- 0.19 (95% CI: 0.15 to 0.25). There was significant between-study heterogeneity (p < 0.0001).

The pooled estimates for urine IgG (four studies, 738 participants) were: sensitivity 59.1 (95% CI: 53.3 to 64.7); specificity 92.9 (95% CI: 90.1 to 95.1); LR+ 9.6 (95% CI: 3.9 to 23.4); and LR- 0.23 (95% CI: 0.08 to 0.68). No significant between-study heterogeneity was identified.

The pooled estimates for saliva IgG (five studies, 1,387 participants) were: sensitivity 69.1 (95% CI: 63.8 to 74.1); specificity 94.7 (95% CI: 93.2 to 96.0); LR+ 14.4 (95% CI: 7.3 to 28.6); and LR- 0.33 (95% CI: 0.28 to 0.39). There was significant between-study heterogeneity (p = 0.012).

Accuracy of Western blot:

The pooled estimates for Western blot (10 studies, 1,119 participants) were: sensitivity 91.3 (95% CI: 88.9 to 93.3); specificity 89.0 (95% CI: 85.7 to 91.9); LR+ 8.2 (95% CI: 5.1 to 13.3); and LR- 0.06 (95% CI: 0.02 to 0.16). There was significant between-study heterogeneity (p = 0.001).

Subgroup analyses:

The serum ELISA-IgG data set was split into in-house assays and commercial kits. Heterogeneity persisted in the commercial kit data set only. In-house assays had a higher diagnostic odds ratio than commercial kits, with in-house at 224.8 (95% CI: 87.5 to 577.5) and commercial kits at 46.9 (95% CI: 32.4 to 67.9). Further restriction of the in-house ELISA to those that used whole-cell antigen only resulted in the highest diagnostic odds ratio at 292.8 (95% CI: 101.8 to 841.7).

Western blot test and in-house ELISA using whole-cell antigen were the most accurate tests for the diagnosis of H. pylori in children. The use of antigens from local strains may partially explain the high accuracy of in-house methods. In-house ELISA might be suitable for use in developing countries.

The review addressed a clearly stated research question that was defined by appropriate inclusion criteria. Limitation of the search to English- and Spanish-language studies left open the possibility of language bias. The use of search terms (sensitivity and specificity) for diagnostic accuracy studies is not generally recommended and further increases the risk that relevant data may have been omitted. Measures were taken to minimise error and bias in the study selection process, but it was not clear whether similar measures were applied to the whole data extraction process. The models used to generate pooled estimates of accuracy measures and to fit the SROC curves were not specified and no test for threshold effect (variation in diagnostic performance with threshold) was described. Given the considerable statistical heterogeneity observed in some data sets and the inclusion of large numbers of different commercial kits, the generation of pooled estimates of accuracy would appear to be of limited value. The summary ROC curves presented would have been more informative if they had data points marked to differentiate between the different assays. The authors' conclusion that in-house assays had superior accuracy to commercial kits is flawed in that it represents a crude comparison with a heterogeneous data set (it may be that some commercial kits perform better than others and/or better than in-house assays, but this cannot be assessed as individual data sets are not reported).

Practice: The authors stated that since the accuracy of commercial ELISA kits varied widely, clinicians may have to rely upon tests developed locally to provide clinically useful results. In particular, local antigen sources using H. pylori strains isolated from the community and validated for the population may be needed.

Research: The authors stated that more well-designed studies of commercial ELISA tests were required.

Not reported.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.