To review and reassess the published diagnostic performance statistics for magnetic resonance imaging (MRI) of the menisci and cruciate ligaments, and to illustrate the potential sources and effects of bias in the evaluation of this diagnostic technique.

MEDLINE and EMBASE were searched from January 1983 to June 1994 for articles published in the English language (the keywords were given). In addition, the RSNA Index to Imaging Literature, and the bibliographies in standard texts and all identified articles, were handsearched.

Study designs of evaluations included in the review

The included studies had to have a total sample size of at least 35 patients. Prospective and retrospective studies (unblinded, radiologist blinded, surgeon blinded, and double-blind) were included in the review.

Specific interventions included in the review

No inclusion criteria relating to the MRI technique were specified.

Reference standard test against which the new test was compared

The included studies were required to use arthroscopy as the reference standard.

Participants included in the review

No inclusion criteria relating to the study participants were specified. The included primary studies were of patients undergoing MRI of the menisci and/or cruciate ligaments.

Outcomes assessed in the review

The included studies were required to report sufficient data to allow the recalculation of diagnostic performance statistics (sensitivity, specificity and accuracy) and associated confidence intervals (CIs).

How were decisions on the relevance of primary studies made?

The authors did not state how the papers were selected for the review, or how many reviewers performed the selection.

The authors did not state that they assessed validity.

The authors did not state how the data were extracted for the review, or how many reviewers performed the data extraction.

How were the studies combined?

Diagnostic performance statistics were recalculated for each study and estimates of overall diagnostic performance were derived from the summed totals of true and false (positive and negative) MRI results.

How were differences between studies investigated?

Differences between the studies were investigated through a detailed discussion of the diagnostic performance statistics, in particular, the sampling error.

Twenty-two studies were included: 13 prospective (n=4,094) and 9 retrospective (n=947).

The overall sensitivity for MRI of menisci and cruciates was 0.88 (95% CI: 0.87, 0.90) and the overall specificity was 0.94 (95% CI: 0.93, 0.94). The sampling error varied widely amongst studies.

Diagnostic performance statistics are widely used. It is still not well appreciated that these are subject to sampling error; such errors make meaningful comparisons between published studies more difficult. Nevertheless, the results for meniscal and cruciate lesions were consistently high and supported the use of MRI for these common problems. The diagnostic performance of other applications of MRI should be subjected to a similar critical review.

The review addressed a clear research question with reasonable definition of appropriate inclusion criteria. The restriction of the review to English language publications represents a significant limitation, which the authors acknowledged. No attempt to identify unpublished data was reported, and publication bias was not assessed. It seems likely that relevant data may have been overlooked. The authors made limited reference to methodological quality, in terms of blinding and retrospective or prospective study design. The reporting of the review methodology was poor and few details of the included primary studies were provided. This makes it difficult to assess the general applicability of the conclusions of the review.

The method of calculating the pooled sensitivity and specificity values, by summing the 2x2 data from the individual studies, treats all of the included studies as a single population. This approach is rarely valid, and cannot be justified without formal assessment of between-study heterogeneity and the effects of different cut-off thresholds; no such assessment was reported for this review.

The authors' conclusions should be treated with caution given the considerable methodological limitations of this review.

Practice: The authors stated that the results for meniscal andcruciate lesions supported the use of MRI for these common problems.

Research: The authors did not state any implications for further research.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.