To address the question of comparative effectiveness of medication and CBT, taking advantage of the larger sample sizes afforded by such an approach.

MEDLINE (1966 to present) and PsycLIT (1974 to present) were searched with the keywords 'bulimia', 'bulimia nervosa', 'treatment outcome', 'clinical trial' and 'double blind', alone and in combination. Bibliographies of retrieved studies were reviewed for additional references. Journals that were specific to eating disorders were handsearched, going back approximately 8 years, for relevant studies.

Study designs of evaluations included in the review

Medication studies had to be randomised, double-blind, placebo controlled studies. Non-blinded medication studies were excluded. Combination studies that included a medication only condition were included if they were double-blinded. Psychosocial studies were randomised and ideally included a control group (no treatment or placebo). However, most studies compared two active treatments and so these were also included. Cross over studies were included.

Specific interventions included in the review

Psychosocial interventions including cognitive behavioural therapy (CBT), behavioural studies, and exposure and response prevention (ERP) studies. Common components amongst these studies included:

a. Meal planning/normalisation of food intake and patterning.

b. Exposure to new feared foods.

c. Cognitive restructuring.

d. Preventing purging following a binge.

e. Relapse prevention.

Medication included imipramine, desimpramine, fluoxetine, trazadone, phenelzine, brofaromine and isocarbid (dosages not reported).

Participants included in the review

Participants diagnosed with bulimia according to the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders.

Outcomes assessed in the review

Binge frequency, purge frequency, depression and eating attitudes (self reported restraint and weight/shape concern).

How were decisions on the relevance of primary studies made?

The authors do not state how the papers were selected for the review, or how many of the authors performed the selection.

Validity was not formally assessed. However, non-blinded medication studies were excluded in an effort to keep methodological rigour as strong as possible.

The authors do not state how the data were extracted for the review, or how many of the authors performed the data extraction. Effect sizes were calculated as the mean pre-treatment score minus the mean post treatment score divided by the pooled standard error of the pre and post treatment scores. If means and standard deviations were not reported in the studies, authors were contacted to provide the necessary information. If no response was received the effects size was estimated using various methods.

How were the studies combined?

Hedges and Olkin's procedure (see Other Publications of Related Interest no.1) for estimating effect sizes from a linear combination of estimates was used to determine an overall pooled effect size and 95% confidence interval weighted according to sample size for each outcome measure. Studies that were excluded because of significant heterogeneity remained so for all subsequent analyses. Positive effect sizes indicate positive improvement from pre- to post-treatment. Publication bias was investigated by calculating a fail-safe number. This calculates the number of unpublished studies with a zero effect size that would subsequently reduce the obtained mean ES.

How were differences between studies investigated?

Heterogeneity was investigated statistically using a chi-squared test. If significant heterogeneity was found to be present studies whose effect sizes were most discrepant from the average were excluded and the heterogeneity test was repeated until the chi-squared test was non-significant.

9 medication studies, 13 psychosocial studies and 4 combination studies. Many of the psychosocial studies included more than one active treatment which resulted in a maximum of 26 psychosocial effects sizes.

Effectiveness of medication (n=9).

Binge frequency (n=9): effect size = 0.66 (95% CI: 0.52-0.81), p-value for heterogeneity >0.05.

Purge frequency (n=8): 2 studies were omitted from the pooled estimate due to significant heterogeneity when these were included.

Effect size = 0.39 (95% CI: 0.24-0.54).

Depression (n=9): effect size = 0.73 (95% CI: 0.58-0.88), p-value for heterogeneity >0.25.

Eating attitudes (n=8): effect size = 0.71 (95% CI: 0.56-0.86), p-value for heterogeneity > 0.25.

Psychosocial treatment (n=26).

Binge frequency (n=17): effect size = 1.28 (95% CI: 1.09-1.47), p-value for heterogeneity >0.10.

Purge frequency (n=25): effect size = 1.22 (95 % CI: 1.06-1.39), p-value for heterogeneity >0.10.

Depression (n=19): effect size = 1.31 (95% CI: 1.10-1.51), p-value for heterogeneity >0.05.

Eating attitudes (n=15): 2 studies were omitted from the pooled estimate due to significant heterogeneity when these were included. Effect size = 1.35 (95% CI 1.12-1.58).

Drop-out and abstinent rates: Dropouts in medication trials were higher (25.4%) compared to psychosocial studies (18.6%) but the difference was not significant.

The fail-safe values ranged from 6 to 122, suggesting that it is unlikely that the results of the current meta-analysis were biased by unpublished null results.

Both medication and CBT were well tolerated and effective in the short term in reducing eating disorder symptomatology pre- to post- treatment, but psychosocial treatment was significantly more effective than medication in reducing self-reported binge and purge frequency, depression, and self-reported eating attitudes. There is also a suggestion that combined medication and CBT treatment is superior to medication alone and may or may not be better than CBT alone. The results of this meta-analysis must be taken with some caution as there is debate over the accuracy of the methodology.

A thorough literature search was conducted, although authors do not state whether any language restrictions were imposed or whether attempts were made to locate unpublished studies, thus the results may be subject to publication bias. The review lacks methodological details such as the number of authors involved in study selection and data extraction. No formal validity assessment was undertaken, however, the review was limited to randomised studies, and to double-blind placebo controlled studies for the medication section of the review. The methods used to pool the data were not appropriate. Pre- to post- treatment scores were compared thus the controlled element of the trials was lost: from the effect size calculated it is not possible to investigate whether these improvements would have occurred without treatment as would have been possible if the change in scores had been compared between treatment groups. Heterogeneity was investigated appropriately, however, the methods used to control for heterogeneity were inappropriate. Studies which were found to contribute to heterogeneity were dropped from further analysis, this is not an appropriate way to control for heterogeneity and instead reasons for the observed heterogeneity should have been investigated. This is especially a problem as the authors were pooling the results of trials of different medication types, some of which may be more effective than others. The authors' conclusions should therefore be interpreted with extreme caution.

Practice: The authors state that CBT is the current treatment of choice.

Research: The authors state that continued exploration into novel treatment options for this complex and often chronic disorder is critical.

1. Hedges LV, Olkin I. Statistical methods for meta analysis (1985). New York: Academic Press. These additional published commentaries may also be of interest. Weaver K. Review: medication and cognitive behaviour thaerapy control symptoms of bulimia nervosa. Evid Based Nurs 1999;2:120. Waller G. Review: medication and cognitive behaviour therapy control symptoms of bulimia nervosa. Evid Based Med 1999;4:145.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.