|Management of treatment-resistant epilepsy. Volumes 1: evidence report and appendices. Volume 2: evidence tables. Diagnosis
|Chapell R, Reston J, Snyder D, Treadwell J, Tregear S, Turkelson C
This review planned to evaluate which methods of re-evaluating refractory epilepsy can lead to improved patient outcomes. The authors addressed a different question, and concluded it was unclear whether blood prolactin levels can differentiate between epileptic and non-eplieptic seizures. The conclusion seems appropriate given that it was based on only five poor-quality trials and no review methodology was reported.
To determine which methods of rediagnosing or re-evaluating treatment-resistant epilepsy lead to, or can be expected to lead to, improved patient outcomes. Drug treatments and non-drug treatments are addressed in separate abstracts (DARE abstract numbers 12005008218 and 12005008219, respectively).
The authors searched the following: MEDLINE (1975 to 2002), EMBASE (1975 to January 2002), PsycINFO (1975 to January 2002), CIRRIE (November 2001), CINAHL (1988 to January 2002), the Cochrane Database of Systematic Reviews (Issue 4, 2001), the Cochrane CENTRAL Register (Issue 4, 2001), the Cochrane Review Methodology Database (Issue 4, 2001), DARE (Issue 4, 2001), NHS EED (to January 2002), ECRI Health Devices Alerts (1977 to January 2002), ECRI Health Devices International Sourcebase (1977 to January 2002), ECRI Healthcare Standards (1975 to January 2002), ECRI International Health Technology Assessment (1990 to January 2002), ECRI Library Catalogue (to January 2002), ECRI TARGET (to January 2002), ERIC (January 2002), Health and Psychosocial Instruments (to April 2001), LocatorPlus (to January 2002), NDA Pipeline (November 2001), REHABDATA (April 2001), U.S. Centers for Medicare and Medicaid Services (to January 2002) and the National Guideline Clearinghouse (to January 2002); the search strategies were reported. In addition, the reference lists of included studies were checked. Current Contents (Clinical Medicine) was also searched on a weekly basis. Only full-length, English language articles, published in 1985 or later, were included. Meeting abstracts were excluded.
Study designs of evaluations included in the review
There were no specific criteria relating to the study designs eligible for inclusion. If there were fewer than five studies for a given diagnostic procedure, and none of these were a randomised controlled trial (RCT) with 50 or more patients in the treatment group, none of the studies were included for that diagnostic procedure. If one of the studies was an RCT with 50 or more patients in the treatment group, the RCT was included even if there were fewer than five studies. When there were five or more controlled studies for a given diagnostic procedure, the uncontrolled studies were excluded for that diagnostic procedure. When there were five or more prospective studies for a given diagnostic procedure, the retrospective studies were excluded for that diagnostic procedure.
Specific interventions included in the review
There were no inclusion criteria specifically related to the interventions. Studies that met the inclusion criteria were identified for thirteen technologies. However, the only diagnostic procedure to be evaluated in more than five studies that were deemed to be of acceptable quality was blood prolactin, therefore this was the only diagnostic procedure to be evaluated in the review.
Reference standard test against which the new test was compared
No inclusion criteria were specified for the reference standard. The reference standard was either clinical opinion supported by electroencephalography, or was not reported (two studies did not report a reference standard for either group, and one reported a reference standard for the cases only).
Participants included in the review
Studies of people with epilepsy were eligible for inclusion. The studies had to have a minimum of 10 people to be eligible for inclusion.
Outcomes assessed in the review
Studies had to have reported diagnostic performance characteristics, or sufficient data for these to be calculated, or had sufficient duration of follow-up for conclusions to be drawn regarding patient outcomes. The authors stated that none of the studies reported whether the changes in management related to improvement in patient outcomes.
How were decisions on the relevance of primary studies made?
The authors did not state how the papers were selected for the review, or how many reviewers performed the selection.
Assessment of study quality
The authors examined studies for diagnosis yield bias, imperfect reference standard bias, differential reference standard bias, prevalence bias, spectrum bias, interpretation bias, patient bias, investigator bias and verification bias. Studies considered to have design flaws that would bias these results were excluded from the review. The authors did not state how many reviewers performed the validity assessment.
The authors did not state how the data were extracted for the review, or how many reviewers performed the data extraction.
The authors summarised the results of each study as the effect size, Hedges' d (method of calculation not referenced in the report, see Other Publications of Related Interest).
Methods of synthesis
How were the studies combined?
The studies were combined in a narrative. Evidence tables provided 2x2 tables, diagnostic performance characteristics, and the range of prolactin measurements in different groups.
How were differences between studies investigated?
The authors provided forest plots of hedges' d effect sizes. Receiver operating characteristic curves were also presented.
Results of the review
Thirty-eight studies were identified as evaluating diagnostic technologies. Of these, five diagnostic case-control studies (n=229) were included in the review.
All five studies were subject to reference standard bias, and four to prevalence bias and spectrum bias. None of the studies were subject to patient, diagnostic yield or verification bias.
Epileptic seizures versus syncopal seizures.
Two out of three studies reported blood prolactin levels to be significantly higher in the epileptic seizure group compared with the syncopal seizure group, and that this diagnostic test could discriminate between these types of seizures. The third study showed no significant difference in blood prolactin levels, therefore this diagnostic test could not discriminate between these types of seizures.
Epileptic seizures versus psychogenic seizures.
Two studies investigated the ability of diagnostic tests to distinguish between these types of seizure; both reported blood prolactin levels to be significantly higher in the epileptic seizure group compared with the syncopal seizure group, and that this diagnostic test could discriminate between these types of seizures.
The authors stated that definite conclusions cannot be drawn about whether blood prolactin levels have a useful role in differentiating between epileptic and non-eplieptic seizures.
The inclusion criteria were poorly defined. The authors undertook an extensive search and made attempts to locate published and unpublished studies. However, a restriction to English language studies was imposed, resulting in the potential for language bias and some studies being missed. As none of the studies reported whether patient outcomes were improved or not, the authors evaluated the ability of the diagnostic test to distinguish between types of seizure, rather than address the primary review question.
No details of the review methodology were provided, making it unclear if this was performed in duplicate to reduce error and bias. The exclusion of poor-quality trials is valid, however, to apply the same inclusion criteria for a review of diagnostic and effectiveness studies seems inappropriate. Considering the lack of diagnostic literature in this field, to exclude 33 of the 38 identified studies that passed quality criteria, on the basis that there were fewer than five studies of acceptable quality evaluating that technique, resulted in some important diagnostic techniques not being discussed. It might have been more appropriate to either discuss all these studies to give a broader overview of the diagnostic tools, or exclude them all on the basis of quality, highlighting the lack of good-quality data. The authors could have suggested how future researchers can design experiments to provide good-quality data to answer the entirely valid questions the authors had posed. The inclusion of just five, small, poor-quality diagnostic case-control studies, which are prone to bias, evaluating a single diagnostic tool seems inappropriate. The authors calculated an effect size for each study, without explaining or referencing the method used. The authors stated that they did not fit a summary receiver operating characteristic curve to the data of those studies attempting to distinguish between epileptic seizures and syncopal seizures. However, they plotted three thresholds derived from two of these studies on a graph, and fitted a curve.
Given the lack of methodological details, it is not possible to assess the reliability of the results,. However, when considering the poor quality of the few included studies, the authors' conservative conclusion seems appropriate.
Implications of the review for practice and research
Practice: The authors did not report any implications for practice.
Research: The authors suggested that future research should determine whether the use of the diagnostic test of interest ultimately leads to improved patient outcome. To do this, a prospective RCT with a reasonable follow-up period is needed.
Agency for Healthcare Research and Quality, contract number 290-97-0020.
Chapell R, Reston J, Snyder D, Treadwell J, Tregear S, Turkelson C. Management of treatment-resistant epilepsy. Volumes 1: evidence report and appendices. Volume 2: evidence tables. Diagnosis Rockville, MD, USA: Agency for Healthcare Research and Quality. Evidence Report/Technology Assessment; 77. 2003
Other publications of related interest
Hedges LV, Olkin I. Statistical methods for meta-analysis. Boston (MA): Academic Press, Inc.; 1985.
Subject indexing assigned by CRD
Epilepsy, Temporal Lobe /therapy
Database entry date
This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.