This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn.

The use of two practice-based quality improvement (QI) programmes, QI-meds and QI-therapy, for patients suffering from depression. In QI-meds, nurse specialists were trained to support medical adherence to the programme through monthly telephone contacts or visits for 6 or 12 months. In QI-therapy, practice therapists were trained to provide individual and group cognitive-behavioural therapy, following a protocol developed at the San Francisco General Hospital Depression Clinic. In both QI programmes, the patients and clinicians retained the choice of treatment and resources. All local practice teams were trained in a 2-day workshop, and were provided with materials such as videotapes, lectures, manuals and pocket reminder cards.

Other: programmes to increase effectiveness of treatment.

Cost-utility analysis.

The study population comprised patients who intended to use the practice involved in the study over the next 12 months, and who were screened positive for depression on the basis of the World Health Organisation's 12-month Composite International Diagnostic Interview. According to this indicator, the patients were considered positive for depression if they reported at least 1 week of depression in the last 30 days, plus 2 or more weeks of depression (or loss of interest in pleasurable activities) in the last year or persistent depression over the year. Patients were excluded if they were younger than 18 years of age, or were not fluent in English or Spanish. They were also excluded if they were lacking in insurance coverage for the intervention therapists.

The setting of the study was primary care. The study was carried out in 6 non-academic managed care organisations in the USA. Forty-six of the 48 primary care practices were involved and 181 of the 183 clinicians participated.

The effectiveness evidence related to June 1996 to July 1999. The data on the resources used were gathered from July 1997 to July 1999. The price year was 1998.

The effectiveness evidence was derived from a single study.

The costing was undertaken prospectively on the same patient sample as that used in the effectiveness analysis.

Power calculations were not reported. The sample was selected on the basis of the Partners in Care study. Over a 5- to 7-month period between June 1996 and March 1997, 27,332 consecutive patients in the participating practices were screened. Of the 3,918 eligible patients, 2,417 had insurance coverage and 241 were found ineligible. The final sample of those who agreed to participate in the study comprised 1,356 patients (and 46 clinics). There were 443 patients in the usual care group, 424 patients in the QI-meds group, and 489 patients in the QI-therapy group.

The study was a randomised controlled trial, carried out in 46 clinical practices. The method of randomisation was not reported. The patients were followed for 24 months and were assessed (through mail surveys) at baseline, and at 6, 12, 18 and 24 months. The completion rates were 95 and 85% for the baseline and 24-month telephone surveys, respectively. For the mail surveys, the completion rate was 90% at baseline, 86% at 6 months, 84% at 12 months, 83% at 18 months, and 85% at 24 months.

The clinical study was analysed on an intention to treat basis. Only patients completing at least one follow-up were analysed. The primary health outcomes were the quality-adjusted life-years (QALYs) and a measure of the days worked.

The QALYs were calculated in two different ways. QALY-SF was obtained using the health utility index from the Short-Form, 12-Item Health Survey (SF-12), and utility weights were derived from a convenience sample of primary care patients through a standard gamble approach. QALY-DB was assessed using depression-burden days on the basis of positive scores of three measures: probable major depressive disorder, significant depressive symptoms, and poor mental health-related quality of life. These final data were partly integrated with some findings from the literature. The days worked were measured by taking the average of employment status at the start and the end of each period, and then multiplying by the number of workdays in 6 months, i.e. 116. The days off work due to illness were also measured, and were reported by the patients for the 4 weeks preceding each follow-up survey.

The study groups were comparable in terms of factors such as gender, marital status, rank in the distribution of household wealth, employment status, ethnicity, and medical co-morbidity. However, the groups differed statistically in terms of the following.

College education: 15% in the usual care group had a college education, compared with 22.9% in the QI-meds group and 21.5% in the QI-therapy group.

Current symptoms of lifetime disorders: 26.1% patients in the usual care group had these disorders, compared with 1.5% in the QI-meds group and 19.4% in the QI-therapy group.

Age: the QI-therapy patients were on average 3 years older than the patients in the other groups.

In terms of QALY-SF, the total value in the usual care group was 1.6624 (95% confidence interval, CI: 1.628 - 1.697). The incremental increase was 0.0115 (95% CI: -0.004 - +0.027, p=0.15) due to QI-meds, and 0.0226 (95% CI: 0.008 - 0.038, p=0.006) due to QI-therapy. The incremental effect of the pooled QI programmes was 0.0173 (95% CI: 0.004 - 0.030, p=0.007).

In terms of QALY-DB, it was assumed that depression reduced the value of a life-year by 0.2 to 0.4 QALYs. The total number of days of depression in the usual care group was 419.9 (95% CI: 398.9 - 441). The incremental change due to QI-meds was -25 days (95% CI: -63.1 - +13.2, p=0.19) or 0.0137 to 0.0274 QALYs-DB. The incremental change due to QI-therapy was -46.7 days (95% CI: -83.1 to -10.3, p=0.01) or 0.0258 to 0.0515 QALYs-BD. The incremental change of the pooled QI programmes was -36.5 (95% CI: -68.9 to -4.1, p=0.03).

In terms of days of employment, the total value in the usual care group was 279.2 (95% CI: 270.2 - 288.1). The incremental increase due to QI-meds was 17.9 (95% CI: -1.6 - +37.4, p=0.07), whilst that due to QI-therapy was 20.9 (95% CI: 2.4 - 39.3, p=0.03). The incremental effect of the pooled QI programmes was 19.5 (95% CI: 3.5 - 35.5, p=0.02).

The days missed from work due to illness were similar in the study groups.

QI programmes improved the patients' health status, reduced the days of depression burden, and positively affected the availability of the patients for work.

The benefit measures used in the economic analysis were QALY-SF and QALY-BD, as assessed and reported in the effectiveness analysis.

No discounting was conducted although the time horizon of the study was 2 years. The quantities of resources used were not reported, while the unit costs were reported for only a few items. The cost/resource boundary reflected the perspective adopted in the study. The direct cost items included in the study were for screening, intervention materials, initial nurse specialist assessment, 20 minutes' supervision by nurses and therapists per enrolled patient, emergency department visits, medical and mental visits, and psychotropic medications. The research-specific costs and inpatient costs were not included in the analysis. The costs were estimated on the basis of provider reimbursement rates, which were used as proxies for health care costs, and were derived from a national database. The quantities were estimated from the trial, although claims and encounter data from the practices were only consistently available for the first 6 months of the study. The quantities of resources were measured from July 1997 to July 1999. The price year was 1998.

Statistical analyses were conducted to assess the intervention effects on health care costs. The analyses used a 2-part model, due to the skewed distribution of the costs in the trial. The first model was the probability of positive costs, using logistic regression. The second model was the log of costs, using ordinary least squares.

Discounting was irrelevant although all the indirect costs were incurred over a time period of 2 years. The unit costs and the total quantities of resources used were not reported. Since the cost/resource boundary was that of society, the indirect costs were assessed and included in the analysis. These were derived using the hourly wages at baseline and gender-specific mean wages for those not working at baseline. The source of the indirect costs was not reported. The authors made assumptions relating to the costs of the patients' time for visits, medication, and travel. The indirect costs were measured from July 1997 to July 1999. The price year was 1998.

US dollars ($).

Sensitivity analyses were conducted on data relating to follow-up visits. The aim of the analysis was to avoid possible double counting. However, the cost-utility ratios did not change.

See the 'Effectiveness Results' section.

The total costs in the usual care group were $3,853 (95% CI: 3,282 - 4,389). The incremental cost due to QI-meds was $419 (95% CI: -467 - +1,306, p=0.35), whilst that due to QI-therapy was $485 (95% CI: -393 - +1,363, p=0.28). The incremental cost of the pooled QI programmes was $454 (95% CI: -305 - +1,214, p=0.24). The difference in the total costs among the study groups was not statistically significant.

The costs and the benefits were combined using an incremental cost-utility analysis. The incremental costs per QALY-SF of IQ-meds and IQ-therapy over usual care were $36,467 (IQ-meds) and $21,478 (IQ-therapy). The incremental cost range per QALY-BD was $15,331 to $30,663 for IQ-meds and $9,478 to $18,953 for IQ-therapy, over usual care.

From a societal point of view, practice-initiated quality improvement (QI) programmes proved to be cost-effective in comparisons with other accepted medical interventions implemented in the health systems. Both of the programmes improved the patients' welfare and facilitated the patients' availability for work, reducing the number of days off work. In particular, the QI-therapy programme showed a slightly better overall value than the IQ-meds intervention.

The rationale for the choice of the comparator was clear. Usual care was selected because it represented the routine intervention for patients with depression. You should assess whether QI programmes are actually implemented in your own setting.

The effectiveness analysis used a randomised, controlled trial and a very large sample size, which should ensure the robustness of the study's conclusions. In addition, appropriate statistical analyses were conducted on the outcome measures. However, the authors stated that patient reports represented the main source of the effectiveness evidence and the enrolment rate was lower than expected. Further, the study groups were not completely comparable at baseline, although appropriate multivariate techniques, conducted on all intention to treat analyses, were used to deal with the issue of confounding variables and the authors indicate that this, at least partially, dealt with the differences between patient groups

and the possibility of confounding variables cannot be ruled out. These issues tend to limit the internal validity of the analysis.

The benefit measures were derived from the effectiveness analysis. It is, perhaps, worth noting that the estimation of utility weights for QALYs was obtained from a different study population or from the literature.

The costs were analysed from a societal perspective. It appears that all the relevant categories of costs have been included in the analysis. Some cost items were not included, but these omissions were unlikely to have affected the study's conclusions. However, the authors stated that some cost estimations were not precise, despite the large sample size. Further, the costs were quite skewed in the sample, although statistical analyses were conducted to take into account variability of the data. Finally, reimbursement rates were used as proxies for costs. This has the limitation of not reflecting true opportunity costs, thus restricting the external validity of the results.

The authors did not compare their findings with those from other studies. The issue of generalisability to other settings was not addressed, and sensitivity analyses were only conducted for a single cost item. The authors acknowledged some limitations of their study, which have already been reported in the relevant fields of this abstract.

The study results suggest that practice-initiated interventions may actually improve the quality of care for depression, and increase the patients' and societal welfare. However, the implementation of such programmes leads to higher costs, therefore a greater consumer demand or public policy initiatives would be required to provide incentives for their implementation. In addition, the authors point out that "this result could suggest broader economic benefits of the intervention to families and purchasers - benefits that may not be fully captured in standard measures of QALYs".

Funded by the Agency for Healthcare Research and Quality, grant number Ro1-HS08349; the National Institute of Mental Health, grant numbers 5R01MH57992 and P50MH54623, and Research Scientist Award MH01170-05; the John D. and Catherine T. MacArthur Foundation, grant number 96-42901A-HE.