This review assessed whether Chronic Care Model (CCM) interventions improve outcomes for specific chronic illnesses. The authors concluded that clinical outcomes, processes of care, and to a lesser extent quality of life, were improved in patients receiving at least one element of the CCM. However, these findings may not be reliable given the limitations in the review methods.

To assess whether Chronic Care Model (CCM) interventions improve outcomes for specific chronic illnesses, and whether any individual components of CCM interventions are essential for effectiveness.

MEDLINE and the Cochrane Library were searched for systematic reviews of the chronic illnesses of interest, although the search dates and search terms were not reported. A total of 23 published systematic reviews were identified; these were used to identify papers relevant to this review. In addition, MEDLINE was searched from 1998 to June 2003 and several independently maintained bibliographies (e.g. the Chronic Care Bibliography) were checked for additional studies; topic experts were also consulted. Only studies published in the English language from 1993 onwards were eligible for inclusion.

Study designs of evaluations included in the review

Randomised controlled trials (RCTs) and non-randomised controlled trials were eligible for inclusion. Studies reporting insufficient statistics for data extraction were excluded from the review if the authors could not be contacted for further information.

Specific interventions included in the review

Studies that assessed interventions comprising of at least one or more elements of CCM were eligible for inclusion. Studies that did not compare interventions with usual care were excluded from the review. The interventions were classified according to the following six CCM elements: delivery system design; self-management support; decision support; clinical information systems; community resources; and health care organisations. The authors listed examples of interventions within each element. The most commonly reported interventions assessed in the included studies were self-management support and delivery system design, which were often combined with at least one other element; decision support interventions were also commonly reported. Eight studies included four CCM elements. Most of the interventions were conducted in an out-patient setting.

Participants included in the review

Studies that assessed patients with one of the following chronic illnesses were eligible for inclusion: asthma, congestive heart failure (CHF), non-insulin dependent diabetes mellitus and depression.

Outcomes assessed in the review

The authors did not specify which outcomes were eligible for inclusion in the review. A wide range of outcomes were reported by the included studies. However, in order to aid data collection and analysis, outcomes reported by the greatest number of included studies were chosen for inclusion. Details of the outcomes of interest (clinical, quality of life and process of care) selected for each of the four chronic diseases were reported. Multiple measures for the same outcome were reported for certain CCM domains (e.g. measures for depression and quality of life).

How were decisions on the relevance of primary studies made?

The studies were screened without the authors and their affiliations being masked. One reviewer was responsible for all aspects of the study selection process.

Study quality was assessed using the Jadad scale, with each study awarded a score between 0 (low quality) and 5 (high quality). The authors did not state how many reviewers performed the validity assessment.

The data were extracted using a standardised data extraction form. One reviewer was responsible for all aspects of the data extraction process.

Where data were reported in different forms (i.e. continuous, dichotomous or counts), all forms were generally extracted. The exception was asthma and CHF studies, where only dichotomous outcomes were reported for clinical outcomes (e.g. emergency department admissions and readmissions). Risk ratios with 95% confidence intervals (CIs) were calculated for dichotomous outcomes and Hedges g (adjusted for small-sample bias) for continuous outcomes. Where studies reported multiple follow-up times, only 12-month data (or the nearest equivalent time point) were extracted. Standard deviations were extracted where reported, or assumed to be equivalent to one quarter of the theoretical range for the measure concerned. The authors of the primary studies were contacted for missing data. In general, higher scores represented a worse outcome for clinical measures and a better outcome for measures of quality of life and processes of care; for quality of life, some measures were recoded so as to fit this pattern. The authors stated that further details of the included studies were available elsewhere.

How were the studies combined?

The studies were grouped according to type of outcome and pooled using a random-effects model across all chronic conditions; data were also stratified and pooled according to condition. Pooled effect sizes (relative risks and weighted mean differences, RRs and WMDs, respectively) were reported with 95% CIs. Data from pooled estimates of fewer than 5 studies were deemed unreliable (a priori decision). Adjusted random-effects regression models were used to assess the effects of each individual CCM element on the pooled effect sizes; where too few studies were available the data were considered 'not estimable'. Funnel plots and regression asymmetry tests were used to assess the risk of publication bias.

How were differences between studies investigated?

Statistical heterogeneity was assessed using the chi-squared test and the I-squared statistic. Potential sources of heterogeneity (chosen a priori) using study level variables (CCM element, type of chronic illness, duration of follow-up, in- or out-patient setting and Jadad quality score of at least 3) were investigated using random-effects meta-regression models. Post hoc regression analyses also investigated: number of CCM elements in the intervention; studies including or not including delivery system design; and studies including or not including self-management support.

One hundred and twelve studies (number of participants not stated) were included in the review: 93 RCTs and 19 non-randomised controlled trials.

Thirty-six studies (32%) scored 3 points on the Jadad scale for quality; none scored more than 3 points. Methodological limitations of the studies included a lack of double-blinding, although the authors acknowledged that this was often difficult in organisational interventions.

Overall results across all chronic conditions.

Fifty-two studies reporting continuous clinical outcomes for diabetes and depression found a significant improvement in the intervention group in comparison with the control group (effect size -0.23, 95% CI: -0.31, -0.15). Similarly, 46 studies reporting dichotomous clinical outcomes for all conditions found a significant effect in favour of the intervention group (RR 0.84, 95% CI: 0.78, 0.90). Quality of life (24 studies, all conditions) was found to be significantly improved in favour of the intervention group (effect size 0.11, 95% CI: 0.02, 0.21), as were process outcomes (32 studies; all conditions; RR 1.19, 95% CI: 1.10, 1.28).

Results according to chronic condition.

Continuous clinical outcomes in studies of depression (27 studies; effect size -0.25, 95% CI: -0.37, -0.13) and diabetes (25 studies; effect size -0.19, 95% CI: -0.29, -0.10) were found to be significantly in favour of the intervention. Dichotomous clinical outcomes in studies of asthma (9 studies; RR 0.82, 95% CI: 0.69, 0.98), CHF (19 studies; RR 0.81, 95% CI: 0.71, 0.92) and depression (14 studies; RR 0.83, 95% CI: 0.74, 0.93) were also found to be significantly in favour of the intervention. Only depression (3 studies; RR 0.18, 95% CI: 0.08, 0.28) and CHF (6 studies; RR 0.28, 95% CI: 0.06, 0.51) studies showed significantly improved quality of life outcomes in favour of the intervention; however, only 3 depression studies were included in the analysis, which may affect the reliability of the result. Process of care outcomes were significantly improved in favour of the intervention group in studies of depression (15 studies; RR 1.28, 95% CI: 1.11, 1.48) and diabetes (9 studies; RR 1.10, 95% CI: 1.01, 1.19).

Results according to CCM element.

Significant effects in favour of delivery system design were found for continuous clinical outcomes (33 studies; effect size -0.21, 95% CI: -0.40, -0.02), dichotomous clinical outcomes (30 studies; RR 0.77, 95% CI: 0.62, 0.96) and process of care (21 studies: RR 1.16; 95% CI: 1.01, 1.34). Significant effects in favour of self-management support interventions were found for continuous clinical outcomes (35 studies; effect size -0.22, 95% CI: -0.38, -0.05) and dichotomous clinical outcomes (36 studies; RR 0.81, 95% CI: 0.66, 0.99). Significant effects for decision support were only found for process of care (18 studies; RR 1.29, 95% CI: 1.08, 1.54). Clinical information systems, community resources and health care organisation were not shown to have significant effects when compared with control. The effects of quality of life, process of care and dichotomous clinical outcomes were not estimable for community resource interventions, owing to a lack of data.

Sensitivity analyses, meta-regression and publication bias.

None of the variables tested reached significance in the meta-regression analyses, and the pooled effects sizes from the sensitivity analyses (study quality) were similar to the overall effect sizes. Publication bias was noted for all CHF studies and asthma studies reporting dichotomous clinical outcomes.

Clinical outcomes, processes of care, and to a lesser extent quality of life, were improved in patients with chronic illnesses who were receiving at least one element of CCM.

This review answered a clear question that was described in terms of the intervention, population and study design. The outcomes were not defined a priori, but were selected according to those most commonly reported. Since this might have introduced bias, the outcomes analysed might not necessarily reflect the most appropriate and meaningful outcomes. The review relied on several previous reviews and independently maintained bibliographies for the identification of studies, which might have resulted in relevant studies being missed. The authors searched only a limited number of electronic databases, as an update, and did not report the search terms used. Similarly, studies might have been missed as only English language studies published after 1993 were included in the review. Experts were consulted to try and locate missing studies, but it was unclear whether appropriate attempts were made to locate unpublished studies; publication bias may therefore be a problem. The authors assessed publication bias using funnel plots and found evidence to suggest that it may exist for CHF and asthma studies. It is also possible that bias and error were introduced during the selection of studies and data extraction, as only one reviewer performed these tasks. The studies were assessed for validity using the Jadad scale, but it was unclear whether appropriate steps were taken to reduce errors and bias. The overall quality of the studies did not appear to be high as none of the studies scored more than 3 points (out of a possible 5).

The studies were analysed and pooled statistically. Data were pooled across all studies and according to chronic condition and CCM intervention element. The authors themselves acknowledged that pooling might not always have been appropriate. Attempts were made to investigate potential sources of heterogeneity using statistical tests and regression analyses, but these failed to explain all of the observed heterogeneity. Sensitivity analyses were also carried out to assess the effects of study quality on effect sizes. The authors also acknowledged a number of other limitations of their review. Overall, given the above limitations, the findings of the review may not be reliable and should be interpreted with caution.

The authors did not state any implications for practice or further research.

Robert Wood Johnson Foundation, grant numbers 034984 and 035678.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.