The review concluded that cognitive remediation therapy had a small-to-moderate beneficial effect on cognition and functioning in patients with schizophrenia. The use of adjunctive therapy and a more strategic cognitive remediation approach additionally improved functioning. The review was generally performed well but the limitations of the underlying evidence should be considered when interpreting the authors’ conclusions.

To evaluate the effectiveness of cognitive remediation therapy for schizophrenia.

MEDLINE, EMBASE, Current Contents Connect, Web of Science, PsycINFO and Cochrane Central Register of Controlled Trials (CENTRAL) were searched to June 2009 for publications with abstracts in English; search terms were reported. Members of the Cognitive Remediation Experts Working Group (representing English, French, German, Spanish and Italian speaking countries) were contacted for additional studies. The bibliographies of retrieved articles were handsearched for further studies.

Eligible studies included a comparison group and allocation procedure of cognitive remediation therapy for schizophrenia, with a cognitive or functional outcome distinct from the trained tasks. Interventions had to fulfil the standard Cognitive Remediation Experts Workshop definition and most participants (at least 70%) had to have a diagnosis of schizophrenia. The primary outcome was the global cognitive effect averaged across all reported cognitive outcomes.

Intervention for just over half of the studies was drill and practice and, for the remainder drill, plus strategy. Average length of treatment was 32.2 hours (range four to 130 hours), provided across an average of 16.7 weeks (range two to 104 weeks) with an average therapy intensity of 2.2 sessions per week (range 0.6 to five sessions). Computer-assisted programmes were used in 57.5% studies and were more common in drill and practice interventions. Drill and practice interventions tended to be shorter. Roughly a quarter of studies utilized adjunctive psychiatric rehabilitation. Most studies (75%) had active control groups. The mean age of participants was 35.8 years (standard deviation 7.11, range 15.3 to 48.3 years); the mean number of males was 67% (standard deviation 14.2%, range 30 to 100%); mean length of education was 11.7 years (range 10.2 to 13.4 years); with 47% inpatients and 45% outpatients and 7% studies including both. Where reported, symptom severity was mild to moderate but some studies included more severely affected patients. The average number of cognitive domains measured was 3.4 (range one to six).

The authors did not state how many reviewers performed the selection.

Methodological quality was assessed by two reviewers independently with a high degree of agreement (intraclass coefficient 0.9) using the Clinical Trials Assessment Measure method developed by Wykes et al. for psychological treatment. The 15-item measure had a maximum score of 100 and assessed the following criteria: sample characteristics; allocation to treatment (including allocation concealment, blinding, and randomisation); comparison treatments; outcome assessment (including standardised outcomes and blinding of participants); treatment description (including protocol and fidelity assessment); and appropriate analysis (such as intention-to-treat analysis). Agreed ratings were sent to study authors for approval and supplementary evidence requested where there was disagreement.

Effect sizes (Cohen’s d) with 95% confidence intervals (CI) were determined using the mean values for the intervention and control groups or, if they were not available, using the methods of Thalheimer and Cook. The global cognitive effect was then averaged across all reported cognitive outcomes. Follow-up effect sizes were calculated on the basis that there was a period with no treatment.

Two authors independently extracted data and uncertainties were resolved through contact with study authors.

Effect sizes (Cohen’s d) were weighted and pooled to give an overall effect size with 95% CI using a random-effects model. Between study heterogeneity was determined using Χ². Subgroup analyses were performed to find the effect of pre-treatment cognitive differences and outlier effects. Meta-regression analyses were performed to investigate mediator and moderator effects when there were at least 20 studies providing data such as methodological rigour, study vintage, and participant and treatment characteristics. All continuous moderator variables were examined for skewness. Publication bias was assessed using Begg’s method and visually with funnel plots.

Relevant data was available for forty studies (2,104 participants, range 10 to 145). The mean quality score was 57.4 (standard deviation 12.3, range 35 to 87). Internal validity problems included: small sample size (60% studies); lack of independent randomisation (70% studies); lack of treatment fidelity assessment (80% studies) and group allocation masking (73% studies). More recent studies tended to be larger and of higher quality. Mean attrition rate was 11.0% (range 0% to 47.5%). Drop-out was higher than 15% in 12 studies. There was no evidence for publication bias. If a meta-analysis showed significant heterogeneity, then p for heterogeneity was reported. Results were mostly reported post-treatment with few after follow-up.

Cognitive remediation therapy had a significant beneficial effect on global cognition (d 0.448, 95% CI 0.306 to 0.590; 38 studies; p= 0.00 for heterogeneity). The effect was still significant at follow-up (d 0.428, 95% CI 0.184 to 0.671; 11 studies).

Cognitive domains: There were significant benefits for: attention/vigilance (d 0.250, 95% CI 0.080 to 0.419; 16 studies); speed of processing (d 0.258, 95% CI 0.072 to 0.445; 24 studies; p=0.000 for heterogeneity); verbal working memory (d 0.346, 95% CI 0.186 to 0.506; 20 studies); verbal learning and memory (d 0.410, 95% CI 0.273 to 0.548; 23 studies); reasoning/problem solving (d 0.572, 95% CI 0.222 to 0.922; 25 studies; p=0.00 for heterogeneity); and social cognition (d 0.651, 95% CI 0.331 to 0.972; seven studies; p=0.03 for heterogeneity); but with no significant effect on visual learning and memory (10 studies; p=0.05 for heterogeneity).

Specific cognitive tests: There were significant benefits for Digit Span (d 0.422, 95% CI 0.226 to 0.617; 13 studies), the Trail Making Test (d 0.319, 95% CI 0.077 to 0.560; 13 studies; p=0.03 for heterogeneity) and the Wisconsin Card Sorting Test (d 0.335, 95% CI 0.188 to 0.481; 13 studies). There was no significant effect for the Continuous Performance Test (10 studies).

Other outcomes: There were significant benefits for: symptoms (d 0.177, 95% CI 0.034 to 0.321; 20 studies); and functioning (d 0.418, 95% CI 0.216 to 0.620; 19 studies; p=0.003 for heterogeneity). At follow-up the effect was still significant for function (12 studies) but not for symptoms (eight studies).

Mediator and moderator effects were reported. Excluding individual studies did not affect the overall effects. There was significant evidence for an increased effect on psychosocial functioning for interventions with adjunctive psychiatric remediation (d 0.59, 95% CI 0.30 to 0.88) and for those with a strategic approach.

There was a small-to-moderate durable effect of cognitive remediation on cognition and functioning in people with schizophrenia which was not affected by study methodology. The use of adjunctive therapy and a more strategic cognitive remediation approach additionally improved functioning.

The review addressed a well-defined question in terms of participants, interventions and relevant outcomes but relevant study designs were less clearly described. Relevant databases were searched and some efforts were made to identify unpublished studies and studies published in languages other than English. There was no evidence for publication bias. Study quality was assessed with suitable criteria. Data extraction and validity assessment were carried out with efforts to reduce error and bias but it was not clear whether this process applied to study selection. The initial numbers of participants in each included study were not reported.

Control groups of included studies appear to have received a range of "active" and "passive" interventions but insufficient detail was provided to guide the reader as to what these actually entailed. Statistical heterogeneity was assessed and there was evidence for heterogeneity for some outcomes. The statistical method used for the meta-analysis seemed appropriate. Suitable sensitivity and subgroup analyses were performed. [A: A sensitivity analysis of the studies of high methodological quality found no change in the effect sizes].

The review was generally performed well but the limitations in the underlying evidence should be considered when interpreting the authors' conclusions.

Practice: The authors stated that although the effect size was smaller for patients with more symptoms, cognitive remediation should be offered to all patients. Age did not appear to be a key moderator even though most studies focused on patients with a mean age between 30 to 40 years.

Research: The authors identified a need for further studies that focused on what variations in participants and interventions provided the most benefit. They recommended that future cognitive rehabilitation studies included both active and passive comparison groups. They did not advocate studies where patients were paid to take part in therapy.

National Institute for Health Research; King’s College London (UK).

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.