|
A systematic literature review of spiral and electron beam computed tomography: with particular reference to clinical applications in hepatic lesions, pulmonary embolus and coronary artery disease |
Berry E, Kelly S, Hutton J, Harris K M, Roderick P, Boyce J C, Cullingworth J, Gathercole L, O'Connor P J, Smith M A |
|
|
Authors' objectives To determine the effectiveness, cost-effectiveness and diagnostic performance of computed tomography (CT). Diagnostic performance was addressed with particular reference to hepatic lesions, pulmonary embolism (PE) and coronary artery disease (CAD).
Searching MEDLINE, BIDS-ISI, EMBASE, the Cochrane Library, Inside Information Plus, and OCLC (via FirstSearch) were searched; the search strategies were reported in full. Additional studies were sought from Internet websites, and by handsearching the reference lists of all retrieved articles and the abstracts from three International Society of Technology Assessment in Healthcare conferences. In addition, academic centres and manufacturers were contacted to identify ongoing studies and a database of grey literature (supplied by the British Library) was searched. Only primary studies published in full in the English language, before 1997, were eligible for inclusion.
Study selection Study designs of evaluations included in the reviewAll primary studies of greater than 10 patients were eligible for inclusion.
Specific interventions included in the reviewStudies of clinical applications of spiral computed tomography (SCT) or electron beam computed tomography (EBCT) were eligible for inclusion.
Reference standard test against which the new test was comparedThis inclusion criterion is only relevant for that section of the review that dealt with diagnostic performance in specific clinical areas. Diagnostic accuracy studies were included in this section of the review if they reported any reference standard. To be included in the quantitative meta-analysis, the studies were required to use the following reference standards: pathology or histology of the liver for studies of hepatic lesions, and angiography for studies of PE and CAD.
Participants included in the reviewSince this review focused largely on technology rather than disease, no inclusion criteria were specified with respect to the study participants, other than the studies had to be of human patients. The application of CT technology to all types of patient was of interest.
Outcomes assessed in the reviewStudies of the effectiveness of CT were required to report information linking the use of CT scanners with patient outcome, or therapeutic decisions or confidence in those decisions. In order to be included in the quantitative meta- analysis, the studies of diagnostic performance had to report sufficient data for the construction of 2x2 tables.
The reference databases from all sections of the review were also used to identify studies that provided comparative information on radiation dose.
How were decisions on the relevance of primary studies made?Study selection was a three-stage process using predefined inclusion criteria. The authors did not state how many reviewers performed the selection.
Assessment of study quality No formal quality assessment of studies of the effectiveness of CT was carried out because of the paucity of data in these sections of the review. The limitations of the studies identified were discussed in the text.
Studies of diagnostic impact and performance were assessed using criteria specific to design features that are likely to lead to bias in diagnostic studies. The areas addressed were: the appropriateness of the spectrum of included patients and selection criteria applied; the appropriate application of the reference standard (partial and differential verification bias and incorporation bias); independence of the interpretation (diagnostic review, test review, comparator review and clinical review); time elapsed between the test and reference standard (disease progression bias); the recording of withdrawals and observer variability. The checklist used was reported in full, but the authors did not state how the validity assessment was performed.
Data extraction Data extraction forms were used. One reviewer extracted data for SCT of liver lesions and EBCT, while two reviewers extracted data for SCT-PE. The data extracted were patient and study characteristics, details of the index test and reference standard, and data to construct 2x2 tables. Sensitivity, specificity, accuracy, positive and negative predictive values, and odds ratios were calculated.
Methods of synthesis How were the studies combined?Studies in the patient outcome, and therapeutic and diagnostic impact sections of the review were combined in a narrative. Studies reporting on radiation dose and diagnostic performance studies meeting the qualitative inclusion criteria were also summarised in text and tables. No SCT liver lesion study, 4 SCT-PE studies and 12 EBCT-CAD studies satisfied criteria for inclusion in the quantitative analysis. Seven EBCT-CAD studies were sufficiently similar to justify pooling. The results of studies meeting the inclusion criteria for quantitative analysis of diagnostic performance were grouped by clinical objective and plotted in receiver operating characteristic (ROC) space.
The summary measures, i.e. sensitivity, specificity, positive and negative predictive values, accuracy, and diagnostic odds ratio were calculated and reported for each study. A summary ROC (sROC) curve was constructed for 7 studies that compared EBCT (where a calcium score of greater than zero was considered positive) with angiography (where a stenosis equal to or greater than 50% was considered positive) for the diagnosis of clinically significant CAD. The sensitivity (true-positive rate) at the mean false-positive rate (1 minus the specificity) was reported as a summary measure for this sROC curve.
How were differences between studies investigated?Differences between studies in the narrative sections of the review were discussed in the text. For the 7 EBCT-CAD studies pooled using sROC analysis, a multivariate regression analysis was used to analyse the influence of differences between the studies on the summary results. The following covariates were included in this analysis: disease progression bias, blinding bias, number of slices, number of pixels, and number of patients.
Results of the review Patient outcome or therapeutic impact was adequately assessed by 9 studies, three for SCT and six for EBCT. No studies were identified that specifically addressed the diagnostic impact of either modality. However, 12 studies (9 SCT liver lesion studies, 1 SCT-PE study and 2 EBCT CAD-studies) comparing the role of CT with alternative technologies were included in this section of the review. Forty-nine studies (19 SCT liver lesion studies, 4 SCT-PE studies and 26 EBCT-CAD studies) satisfied the criteria for inclusion in the review of diagnostic performance.
The 3 EBCT studies that addressed patient outcome indicated that EBCT may improve patient outcome in a variety of clinical scenarios, but the results were inconclusive.
There was insufficient information for a qualitative synthesis on the therapeutic impact of using SCT and EBCT.
No studies that were specifically designed to address the diagnostic impact of either entity were identified. Studies included in this section of the review reported sparse and conflicting data, allowing no firm conclusions to be drawn.
Four studies that compared the diagnostic performance of SCT applied to liver lesions with a reference standard had a high level of between-study heterogeneity, allowing no firm conclusions to be drawn. Those that compared SCT with conventional CT showed an approximately 10% increase in the number of lesions detected.
SCT performed better for the detection of PE in central vessels alone than in central and peripheral vessels together.
There was insufficient information for a qualitative synthesis on the prediction of asymptomatic CAD using EBCT. Six studies on the diagnosis of symptomatic CAD using EBCT had a low specificity (high false-positive rate). The most likely role for EBCT is in excluding obstructive CAD in the older population. The sROC plot of EBCT for symptomatic CAD showed a sensitivity of 95% and a specificity of 49% at the mean false- positive rate (0.51). The multiple regression analysis found no variable that significantly affected summary measures of diagnostic performance. A univariate analysis indicated that increasing the number of slices improves diagnostic performance.
Inter- and intra-observer reproducibility were acceptable, but inter-examination reproducibility was not.
Comparisons of SCT with conventional CT for radiation dose suggested no significant difference. There were insufficient data available on radiation dose in EBCT.
Cost information The authors presented short summaries of the four economic studies included in the review, but no specific cost information was reported.
Authors' conclusions The authors concluded the following.
MEDLINE and BIDS-ISI are comprehensive sources of references in this area.
There is no strong evidence about any aspect of the use of latest generation CT (SCT and EBCT) at the health economics, patient outcome, or therapeutic impact levels.
SCT detects liver lesions that are not seen with conventional CT.
EBCT has low specificity when applied to the diagnosis of symptomatic CAD.
While evidence suggests using EBCT for population studies, it does not support its use to track CAD progression in individuals.
The introduction of SCT will not cause a significant increase in radiation dose compared with similar examinations performed with conventional CT.
CRD commentary This was a wide ranging review that attempted to assess the effectiveness of a new technology in general, rather than in relation to specific conditions. The inclusion criteria were, therefore, appropriately broad and a number of clear research questions were defined. The search strategy was comprehensive and included attempts to identify unpublished studies. The description of the review process was sparse, so it is difficult to make any judgement regarding its rigour. The quality of the included studies was assessed and discussed and, where possible and appropriate, the impact of aspects of methodological quality on diagnostic performance was investigated.
The use of a narrative synthesis in the majority of the review was appropriate. Quantitative synthesis, where conducted, used rigorous methods and was clearly reported. The authors' conclusions were suitably cautious given the paucity of evidence in this area. Clear and comprehensive guidance was provided with a view to directing future research.
Implications of the review for practice and research Practice: The authors did not state any implications for practice.
Research: The authors stated that there was insufficient evidence in almost every area studied. A number of issues of study and reporting quality were identified by the review. Further high-quality studies are required.
Studies of diagnostic performance and impact should be designed to:
use a single reference standard, ideally the recognised 'gold' standard;
avoid verification bias by applying the single reference standard to all participants;
ensure that observers are blinded to the results of other tests, particularly the reference standard;
publish sufficient data for the completion of 2x2 contingency tables;
use published recommendations on sample size calculation;
comment on operator dependence/learning curves; and
publish details of the study designs to allow a proper assessment of study quality.
Bibliographic details Berry E, Kelly S, Hutton J, Harris K M, Roderick P, Boyce J C, Cullingworth J, Gathercole L, O'Connor P J, Smith M A. A systematic literature review of spiral and electron beam computed tomography: with particular reference to clinical applications in hepatic lesions, pulmonary embolus and coronary artery disease. Health Technology Assessment 1999; 3(18): 1-118 Indexing Status Subject indexing assigned by NLM MeSH Biomedical Technology; Coronary Disease /radiography; Liver Diseases /radiography; Pulmonary Embolism /radiography; Technology Assessment, Biomedical; Tomography, X-Ray Computed AccessionNumber 12000008094 Date bibliographic record published 31/01/2005 Date abstract record published 31/01/2005 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|