This review assessed the effects of explicit financial incentives for improved performance on health care quality measures. It concluded that incentives at the physician, provider group and payment-system levels have some positive effects, but further research is needed. The conclusions are supported by the evidence presented; however, the evidence was limited (especially regarding the payment-system level) and relevant studies might have been missed.

To assess the effect of explicit financial incentives for improved performance on health care quality measures.

The authors searched PubMed from January 1980 to November 2005, checked the bibliographies of retrieved articles, and contacted experts for missing or unpublished studies. The search terms were stated. The search was limited to studies written in English.

Study designs of evaluations included in the review

To be eligible, the studies were to have a concurrent comparison group and report baseline, pre-intervention data for the groups on the quality measure. Observational studies providing cross-sectional analyses were also eligible.

Specific interventions included in the review

Studies of explicit financial incentives for improving the quality of health care were eligible for inclusion. Studies that did not evaluate a specific intervention but examined the relationship between financial incentives and quality were also included. Details of each of the individual incentive programmes (e.g. improving access to health care for nursing home patients with debilitating and acute chronic conditions by providing incentives to admit severely dependent patients, for the attainment of health status goals and for discharging clinically appropriate patients) were described in the paper.

Participants included in the review

Studies of interventions directed at individual physicians, provider groups or payment systems were eligible for inclusion. The populations in the included studies were reported as skilled nursing facilities, Office of Substance Abuse and Medicaid clients, community mental health centres, primary care sites, solo or group practices, pharmacies, physician organisations, provider groups, community-based primary care practices, physicians and primary care physicians.

Outcomes assessed in the review

Studies that included a quantitative measure of the quality of health care were eligible for inclusion. Domains of quality were defined as access to care, structure of care, process of care, outcomes of care and patient experience of care.

How were decisions on the relevance of primary studies made?

At least two reviewers independently reviewed the abstracts of studies for inclusion. Any discrepancies were resolved through discussion and consensus amongst all five authors.

The studies were assessed according to a published methodological quality checklist and graded on a scale of 1 (poor) to 4 (excellent). The authors did not state how many reviewers performed the quality assessment.

Five authors extracted the data. The results of each study were categorised according to the effects of the financial incentive on quality as positive, partial effect, negative or no effect. Positive studies showed a statistically significant improvement on all quality measures; partial effect studies showed an improvement on some measures but not others; negative studies showed a statistically significant decrease for all measures of quality with the financial incentive. Any discrepancies were resolved through discussion and consensus amongst all five authors.

How were the studies combined?

The studies were combined in a narrative.

How were differences between studies investigated?

Differences between the studies were apparent in the text and tables.

Seventeen studies (number of participants unclear) were included in the review: 9 randomised controlled trials, 4 controlled trials with before-and-after data and 4 cross-sectional surveys.

Six studies were assigned a quality grade of 3, six were assigned a grade of 2, and five were assigned a grade of 1.

Of the 2 studies that evaluated financial incentives provided at the payment-system level, one found a positive effect on access to care while the other found a negative effect on access to care for the sickest patients.

Of the 9 studies that evaluated the use of financial incentives directed to provider groups, two reported improvements for all quality of care measures, five were classified as partial improvement studies, and two showed no effect of the intervention compared with the control group.

Of the 6 studies that evaluated the effects of financial incentives at the physician level, two reported a positive effect of the intervention and three reported some positive effects (partial studies).

Five studies found unintended effects of the incentive, including an unintended incentive to avoid the most severely ill patients (1 study), 'gaming behaviour', i.e. exaggerating the initial health status of the patients (1 study), and improvements in documentation rather than actual changes in the quality of health care (3 studies).

Two studies reported that individuals or groups with the lowest baseline performance improved the most; however, if threshold performance targets are used, they may gain the least performance pay (1 study).

One study investigated the cost-effectiveness of an incentive to improve access to health care for nursing home patients with debilitating acute and chronic conditions. Despite the administrative and incentive costs of the programme, the Markov model estimated an average cost-saving to the Medicaid programme of $3,000 per nursing home stay over time.

The review suggested that financial incentives at the physician level, the provider group level, and the health care payment system have some positive effects. Ongoing monitoring of incentive programmes is needed to identify any unintended effects on the quality of care. Rigorous research is needed to guide the implementation of financial incentives and to assess their cost-effectiveness.

The authors set out a clear objective and defined inclusion criteria. Only one database was searched and the search was limited to English language papers, which suggests that relevant studies might have been missed. Although an attempt was made to obtain unpublished data, publication bias was not assessed. Measures were taken to reduce the risk of bias in study selection. Five authors might have carried out the data extraction, but it was not entirely clear whether each data set was extracted in multiple duplicates. The quality assessment appears to have been carried out using appropriate criteria. Appropriate study details were presented and, given the differences between the studies, the narrative synthesis of studies seems appropriate. In general, the authors' conclusions appear to reflect the evidence they presented. However, the conclusion about the effects of incentive interventions at the payment-system level was supported by only one study (and conflicting evidence was identified). The validity of the authors' recommendation for further rigorous research is difficult to determine, given that the search was limited and relevant studies might have been missed.

Practice: The authors stated that ongoing monitoring of incentive programmes is critical to determine any potential unintended effects on quality of care.

Research: The authors stated that rigorous research, including randomised controlled trials and observational studies with concurrent control groups, are needed to guide the implementation of explicit financial incentives for health care quality and to assess their cost-effectiveness. Specific research questions which constitute the authors' proposed research agenda are provided in the paper.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.