To investigate the effectiveness of case management, and to compare the outcomes for assertive community treatment and clinical case management.

Firstly, studies cited in previous reviews of case management in mental health services were examined. Secondly, MEDLINE and PsycLIT were searched using the terms 'case management', 'care management', 'care programming', or 'assertive community treatment' and 'mental health', 'psychiatric' or 'psychiatry and evaluation', 'outcome', 'comparison' or 'effect'. Only studies published between 1980 and 1998 were included.

Study designs of evaluations included in the review

There appeared to be no restrictions on study design, other than limiting inclusion to comparative studies. Only studies published in English-language journals appear to have been included. Studies for which an effect size or p-value could not be calculated, were excluded from the analysis.

Specific interventions included in the review

The authors included studies that compared either (1) the outcomes of a group receiving case management with those of a group receiving standard community care but not case management, or (2) the outcomes of a group receiving assertive community treatment with those of a group receiving another form of case management. Studies in which control groups received services for different amounts of time were excluded from the analysis.

Participants included in the review

Studies were included if they focused on the treatment of adults with serious mental illness such as psychosis, affective disorders, personality disorders, or anxiety disorders.

A total of 6,365 participants were included in the 35 studies comparing case management with usual treatment. Eighty-three percent (n=5,283) of the participants were single; these included those that had never married, were widowed, or were divorced. Fifty-six percent (n=3,564) of the participants were male. The mean age of the participants was 37 (standard deviation, SD=5) years. The mean number of previous psychiatric admissions was 6.6 (SD=2).

In the 19 studies that reported American Psychiatric Association, Diagnostic and Statistical Manual (DSM) axis I diagnoses for all patients, 61.6% (n=2,023) were diagnosed as having schizophrenia, 7.8% (n=257) had bipolar affective disorder, 9% (n=297) had another psychotic disorder, 11.4% (n=376) had depression, 2.1% (n=69) had neurosis, and 8.1% (n=267) had another diagnosis.

Outcomes assessed in the review

The outcome measures assessed were: symptoms, excluding the Global Assessment of Functioning and Global Assessment scales; the number of admissions; the number of days hospitalised; the proportion of the group who were hospitalised; contacts with mental health services; contacts with other services; drop-out rates from mental health services; social functioning; client satisfaction; family satisfaction; family burden of care; and the total cost of care.

How were decisions on the relevance of primary studies made?

The authors do not state how the papers were selected for the review, or how many of the reviewers performed the selection.

Each study was categorised on the basis of research quality, which was estimated using criteria similar to those used by Glass et al. (see Other Publications of Related Interest no.1). The study categories were: random assignment to conditions with attrition less than 20% (high rating); random assignment with attrition greater than 20% or differing between groups (medium-high rating); well-designed matching studies or analysis for covariance (medium-low rating); and weak or non-existent matching procedures (low rating). The authors do not state how the papers were assessed for quality, or how many of the reviewers performed the quality assessment.

The authors do not state how the data were extracted for the review, or how many of the reviewers performed the data extraction.

Each study was coded for client characteristics and several aspects of the study design, including sample size, study period, number of outcome measures used, attrition rates, and the method of assigning the participants to the treatment groups.

The Pearson product-moment correlation coefficient (r) and the one-tailed level of significance (p) were reported. One-tailed p-values were obtained by either halving the reported two-tailed probability if it favoured case management, or by subtracting the halved two-tailed value from one if it favoured the control group. If studies reported the result as not significant, p was coded as 0.5. When a study reported more than one result in domains that were combined, such as two different symptom measures, the mean p- or r-value was calculated. All r- and p-values were calculated by the first author, and recalculated after all the studies had been coded; any inconsistencies were resolved.

How were the studies combined?

The standardised measures for each outcome domain were calculated using either r- or p-values. The weighted mean r for each outcome domain was calculated by converting each r to a standard normal deviate (Fisher's z), and then weighting each z-value by the study's sample size and research quality. The 95% confidence intervals (CIs) were calculated for each weighted mean r. As a separate analysis, the combined p-value was calculated for each outcome measure by calculating the standard normal deviate for each reported p-value, then weighting by sample size and research quality.

Two techniques were used to compare the outcomes of assertive community treatment and generic or clinical case management. First, the combined p and weighted mean effect size were calculated for studies that directly compared these programmes. Second, all studies comparing case management with a control group were analysed by calculating the Q value (between groups) to compare differences between the outcomes of assertive community treatment and clinical case management.

The impact of publication bias was investigated by calculating Rosenthal's fail-safe N for each combined p (see Other Publications of Related Interest no.2), and by calculating the regression asymmetry test (see Other Publications of Related Interest nos.3-4).

Studies comparing assertive community treatment with another model of care management were analysed separately.

How were differences between studies investigated?

The effect sizes between groups were compared using homogeneity analyses, as described by Hedges and Olkin (see Other Publications of Related Interest no.5). Q values were then calculated for the heterogeneity within and between groups. The authors tested whether the quality of the research design was associated with any heterogeneity between studies. They also examined the impact of using measurement instruments that had not been described in peer-reviewed journals.

Forty-four relevant studies were identified, including 35 comparisons (n=6,365) of either assertive community treatment or another model of case management with usual treatment. Nine studies directly compared assertive community treatment with another model of case management. Nineteen studies compared assertive community treatment with usual treatment. Sixteen studies compared another model of case management with usual treatment; of these, there was one with a strengths model, one with a rehabilitation model, one with a generalist model, and 13 others that could not be classified further. These models were referred to as 'clinical case management'.

Case management versus usual treatment.

Compared with usual treatment, case management was associated with the following (combined one-tailed p-values given):

a greater improvement in symptoms (weighted mean r 0.16, 95% CI: 0.11, 0.21);

a reduction in the hospital stay (weighted mean r 0.24, 95% CI: 0.21, 0.28, p<0.001);

a smaller proportion of patients hospitalised (weighted mean r 0.10, 95% CI: 0.06, 0.14, p<0.001);

more contacts with mental health services (weighted mean r 0.24, 95% CI: 0.19, 0.28, p<0.001),

more contacts with other services (weighted mean r 0.33, 95% CI: 0.22, 0.43, p<0.001);

lower drop-out rates from mental health services (weighted mean r 0.33, 95% CI: 0.25, 0.41, p<0.001);

a greater improvement in social functioning (weighted mean r 0.15, 95% CI: 0.11, 0.19, p=0.007);

greater patient satisfaction with care (weighted mean r 0.23, 95% CI: 0.17, 0.29, p=0.28);

greater family satisfaction with care (weighted mean r 0.42, 95% CI: 0.29, 0.53, p<0.001); and

less family burden of care (weighted mean r 0.43, 95% CI: 0.23, 0.60, p=0.007).

However, the patients on care management programmes were admitted to hospital more frequently than those receiving usual treatment (weighted mean r -0.10, 95% CI: -0.16, -0.05, p=0.999).

Publication bias.

From the fail-safe N, 6 domains could be considered robust against publication bias on the basis of the reported p-values. These were: fewer days hospitalised, a smaller proportion of patients hospitalised, more contacts with mental health services, lower drop-out rates, a greater improvement in social functioning, and increased patient satisfaction. The regression asymmetry test was calculated for each outcome domain with the exception of contacts with other services, family burden of care and family satisfaction, which had sample sizes too small to plot. Three of the remaining 9 domains showed some evidence of publication bias: the proportion of patients hospitalised (p=0.015), contacts with mental health services (p=0.017), and patient satisfaction (p=0.05).

Study quality.

Nine domains had variance greater than would be expected by chance. Four of these 9 measures showed significant differences in outcomes according to the study quality, i.e. high versus low: the number of admissions, the days hospitalised, contacts with mental health services, and level of social functioning. However, the weighted mean r-values for the high-quality group were almost the same as those calculated for the sample as a whole.

Use of measures described in peer-reviewed journals.

The mean weighted effect sizes were significantly higher for the studies that used measures that had been reported in peer-reviewed journals, than for those which did not (i.e. they used non-reported measures). This was true for both domains where comparisons were possible: level of social functioning and client satisfaction.

Assertive community treatment versus clinical case management.

The weighted mean effect size for the days hospitalised was found to be significantly greater for assertive community treatment than for clinical case management (p=0.001). Assertive community treatment was significantly better than clinical case management in reducing the proportion of patients hospitalised (p<0.001), but the number of contacts with mental health services was significantly greater for patients in clinical case management programmes than for those in assertive community treatment programmes (p<0.001).

A standardised measure of total cost of care was included in the analysis of outcomes. This was found to be significantly lower for case management than for usual treatment (weighted mean r 0.13, 95% CI: 0.07, 0.19, p=0.43).

Both types of case management led to small to moderate improvements in the effectiveness of mental health services. Assertive community treatment had some demonstrable advantages over clinical case management in reducing hospitalisation. The two approaches had similar effects in improving clinical symptoms, patient and family satisfaction with services, and the patient's level of social functioning.

The review question was supported by appropriate inclusion criteria relating to the participants and the intervention.

The literature search was limited to MEDLINE and PsycLIT and so relevant unpublished papers might have been missed. Indeed, some evidence of publication bias was found in the analysis. Although a quality assessment was carried out, the approach used provided very limited information on study quality. In addition, it was unclear how many reviewers performed the quality assessment, selected the studies, or extracted the data. Some details of the included studies were available in tables, although further details would have been useful. The pooling of results from heterogeneous studies was inappropriate. However, the authors carried out a sensitivity analysis that excluded non-randomised studies, which was appropriate. The remaining randomised studies were more homogeneous and the results were similar to the overall pooled data.

The authors' conclusions appear to follow from the results presented.

Practice: The authors state that assertive community treatment should be targeted at patients who are at the greatest risk of hospitalisation. In addition, both assertive community treatment and high-quality clinical case management should be a feature of mental health programmes.

Research: The authors appear to state that future studies should measure how closely the evaluated programmes adhere to the case management models. Future studies should also take into account potential sources of confounding, as well as consider the differences in effectiveness of case management for different groups of patients.

Victorian Government Department of Human Services, Mental Health branch.

1. Glass G, McGaw B, Smith M. Meta-analysis in social research. Beverly Hills (CA): Sage Publications; 1981. 2. Rosenthal R. Meta-analytic procedures for social research. Newbury Park (CA): Sage Publications; 1984. 3. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple graphical test. BMJ 1997;315:629-34. 4. Steichen TJ. Tests for publication bias in meta-analysis. Stata Technical Bulletin 1988;41:9-15. 5. Hedges L, Olkin I. Statistical methods for meta-analysis. New York: Academic Press; 1985.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.