To investigate, in an explorative way, whether the methodological quality of eye movement desensitisation and reprocessing (EMDR)-therapy studies has effects on the therapy outcome, as measured by the effect sizes.

MEDLINE, PsycLIT and PILOTS were searched for all published studies, although the authors did not specify the years over which the searches were conducted or whether language restrictions were applied. It was not stated whether the reference lists of the retrieved studies were searched manually, or whether there was any attempt to identify further 'grey' literature.

Study designs of evaluations included in the review

Studies with at least 5 participants treated with EMDR, and using either a controlled design, follow-up, or both, were eligible for inclusion in the review, providing they had employed a standardised data-gathering instrument.

Specific interventions included in the review

The intervention was EMDR-therapy of post-traumatic stress disorder (PTSD).

Participants included in the review

The participants were patients suffering from various traumatisations, including PTSD (eg. Vietnam veterans and female rape victims). No other information was provided.

Outcomes assessed in the review

Standardised psychometric measures of target symptoms were assessed in the review. These included the Clinician Administered PTSD Scale (CAPS), the Impact of Event Scale (IES), the PTSD Symptom Scale (PSS), and Structured Interview for PTSD (SI-PTSD).

How were decisions on the relevance of primary studies made?

The authors do not state how the papers were selected for the review, or how many of the reviewers performed the selection.

The authors employed an index of study quality, which was adapted from the meta-analytic research of a European group working on the treatment of eating disorders (see Other Publications of Related Interest nos.1-2). The criteria were: study design, sample size, follow-up, diagnosis, therapist training and treatment fidelity check, statistical evaluation and documentation. A quality score (range: 0 to 16) was obtained by summarising the criteria. The quality scoring was carried out by one reviewer, then discussed in the research team. The authors state that the inter-rater variability was not assessed, due to the exploratory purpose of the review.

The authors do not state how the data were extracted for the review, or how many of the reviewers performed the data extraction.

Data were extracted on the study design, participants, target outcome measures, therapy adherence, therapy frequency, effect size (pre- versus post-intervention and follow-up), and study quality score. For all standardised psychometric measures of target (e.g. PTSD) symptoms, the effect sizes were extracted from the studies, or were computed whenever the scale means and standard deviations were available. The effect sizes were computed as weighted standard deviations of the pre- versus post-treatment mean values. Where only subscales were documented (sometimes IES, SI-PTSD), the mean effect sizes (arithmetic mean) were calculated for the whole scale. When several questionnaires had been employed to evaluate the results, the effect sizes were added and the mean was determined.

How were the studies combined?

The studies were pooled statistically in a meta-analysis. Standardisation was carried out by pooling the standard deviations of pre- and post-treatment means (see Other Publications of Related Interest no.3).

How were differences between studies investigated?

The authors do not report a formal test for heterogeneity.

Seventeen studies were eligible for inclusion in the review.

Five of the 17 studies received a low quality score (0 to 5 points). The effect sizes could, therefore, only be computed in two cases in this group. Seven studies received a medium quality score (5 to 9 points). It was possible to calculate the effect sizes for the pre- versus post- comparison from all of these.

The four studies with the highest score for methodological rating all had a higher rate of adherence to treatment. The mean effect sizes of the questionnaires on the PTSD target symptoms showed a clear relationship with study quality, with studies of high methodological quality showing better effect sizes than those where the treatment standards were not followed or controlled precisely. This applied both to the mean for all instruments and to the IES alone. The mean effect size for the 13 studies which allowed the calculation was 1.25 plus or minus 0.73. The information on the effect sizes was not representative, as the effect sizes could only be calculated for two of the five poor quality studies, and for the IES in only one case.

Further findings from the different studies were reported in the review.

Carefully planned studies, including treatment by well-trained therapists and with a sufficiently high number of therapy sessions, achieve very good effect sizes. In view of this, EMDR treatment can be considered to be an effective treatment for PTSD.

The review question and the study selection criteria were stated clearly. The literature search seemed reasonably comprehensive, although there was no mention of any efforts to find additional material through search of reference lists, conference indexes, and so on; relevant studies may therefore have been missed. The authors gave no information in respect of the dates for the literature search, or whether language restrictions were applied. Appropriate statistical tests seem to have been used in the meta-analysis, but the authors did not state whether any formal tests for heterogeneity were undertaken. In addition, it was not mentioned whether publication bias was assessed. It is unclear whether it was appropriate to pool all the studies, given the observed heterogeneity among the participants and outcome measures used.

The authors' conclusions seem appropriate in the light of the data they present and discuss.

Practice: The authors state that EMDR treatment can be considered to be an effective treatment for PTSD.

Research: The authors state that further research into the efficacy of EMDR should be conducted with studies of high methodological quality. Since there is a lack of further well-planned, well-performed and documented studies on differential indication in patients with various post-traumatic disturbance pictures and different clinical pictures of co-morbidity, these questions should be given priority over an analysis of the various components of EMDR treatment.

1. Cox GL, Merkel WT. A qualitative review for psychosocial treatments for bulimia. J Nerv Ment Dis 1989;177:77-84. 2. Hartmann A, Herzog T, Drinkmann A. Psychotherapy of bulimia nervosa: what is effective? A meta-analysis. J Psychosom Res 1992;36:159-67. 3. Hartmann A, Herzog T. Varianten de effektstarkenberechnung in meta-analysen: kommt es zu variablen ergebnissen? Z Klin Psychol 1995;24:337-43.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.