This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn.

The study compared five short-term outpatient interventions for adolescents with cannabis use disorders.

The motivational enhancement therapy plus cognitive-behavioural therapy 5 sessions (MET/CBT5) intervention comprised 2 individual MET sessions and 3 group CBT sessions, lasting a total of 6 to 7 weeks.

The motivational enhancement therapy plus cognitive-behavioural therapy 12 sessions (MET/CBT12) intervention comprised 2 individual MET sessions and 10 group CBT sessions, lasting a total of 12 to 14 weeks.

The Family Support Network (FSN) intervention used the MET/CBT12 programme plus 6 parent education group meetings, 4 therapeutic home visits, referral to self-help support groups, and case management.

The Adolescent Community Reinforcement Approach (ACRA) intervention comprised 10 individual sessions, 4 sessions with caregivers and family, and limited case management by the therapist over a period of 12 to 14 weeks.

The Multidimensional Family Therapy (MDFT) intervention comprised 12 to 15 sessions (usually 6 with the adolescent, 3 with the parents and 6 with the family) and case management over a period of 12 to 14 weeks.

Treatment.

Cost-effectiveness analysis.

The study population comprised adolescents aged from 12 to 18 years. To be included in the study, the adolescents needed to have self-reported one or more DSM-IV criteria for cannabis abuse or dependence (as described by the American Society of Addiction Medicine, 1996, see 'Other Publications of Related Interest' below for bibliographic details). They also had to have used cannabis in the past 90 days or 90 days prior to being sent to a controlled environment, and be appropriate for outpatient treatment. Adolescents could be included if they had concomitant alcohol or other drug abuse diagnoses, as well as psychiatric disorders, provided they were manageable as outpatients. The exclusion criteria included reported alcohol use on at least 45 of the 90 days before intake, reported use of other drugs on at least 13 of the 90 days before intake, and a reported acute medical or psychological problem that would prohibit full participation in treatment. Adolescents were also excluded if they had insufficient mental capacity to provide informed consent, lived outside of the programme's catchment area, a history of repeated violent behaviour or severe conduct disorder, or had insufficient English to participate.

The setting was secondary care. The study was set in outpatient clinics at specialist centres in four locations in the USA: Farmington (CT), St. Petersburg (FL), Madison County (IL) and Philadelphia (PA).

The effectiveness and resource use data were gathered between 1998 and 2000. The price year was 1999.

The evidence for the final outcomes was derived from a single study.

The costing was carried out prospectively on the same sample of adolescents as that used in the effectiveness analysis.

No power calculations were reported. Adolescents were recruited from the case flow of the participating sites, and through outreach to the juvenile justice system, schools, doctors and public service announcements. The study enrolled adolescents who met the inclusion and exclusion criteria , and who consented to participate, from sequential admissions to the four sites. The study was divided into 2 trials, each conducted at two of the four sites. MET/CBT5, MET/CBT12 and FSN therapies were delivered in Trial 1, while MET/CBT5, ACRA and MDFT therapies were delivered in Trial 2.

Of the 1,244 adolescents who were screened, 44% were excluded from participating, 20% being too severe for outpatient treatment and 24% not being severe enough. In all, 702 adolescents were eligible to join the study, of which 15% declined to participate. In the end, a total of 600 adolescents and their families were recruited. In Trial 1, the MET/CBT5 group comprised 102 adolescents, the MET/CBT12 group comprised 96 adolescents and the FSN group comprised 102 adolescents. In Trial 2, each therapeutic group contained 100 adolescents. The participants were primarily male (83%), white (61%) or African American (30%), enrolled in school (87%), and currently involved in the juvenile justice system (62%). Most of the adolescents began using alcohol or other drugs before the age of 15 (85%) and used cannabis weekly or daily (71%).

The study was a randomised trial without a no-treatment control group, which was undertaken at four sites. A randomly ordered list, generated by independent staff using Microsoft Excel, was used to allocate adolescents to a treatment group at each site. To prevent bias, adolescents were only assigned after they had completed intake assessments. Blinding of the clinical staff was not feasible. To prevent bias at follow-up, tacking and following logs were maintained separately from assignment logs. The follow-up was undertaken at 3, 6, 9 and 12 months. Participants were compensated between $25 and $50 to for attending each follow-up assessment. One or more follow-up interviews were conducted on 99% of the sample. Specifically, 98% of the adolescents were assessed at 3 months, 97% at 6 months, 96% at 9 months and 94% at 12 months.

The analysis of the clinical study was conducted on an intention to treat basis, including approximately 5% who did not receive any treatment. The primary health outcomes were the days of abstinence (from cannabis, alcohol and other drugs) over the follow-up period, and the percentage of adolescents in recovery at the end of the follow-up study. Recovery was defined as living in the community and reporting no substance use, abuse or dependence problems. The primary outcomes were measured using the Global Appraisal of Individual Needs (GAIN), which is a standardised semi-structured interview with 8 main sections (background, substance use, physical health, risk behaviours, mental health, environment, legal, and vocational). Data were also obtained from collateral informants, records and urine tests to validate the self-report data collected at intake, 3, and 6 months. The six groups were compared in terms of their demographic variables, patterns of substance abuse, other risky behaviours in the 90 days before intake, and lifetime history of behavioural interventions. The comparison revealed that adolescents in the Trial 2 sites were less likely to be employed, and more likely to be African American, female and sexually active, than those in the Trial 1 sites.

Within each trial, site differences were modelled with a dummy variable. Reflecting the randomised block design, conditions were modelled as nested within site, which produces a statistic for the significance of site effects, conditions across site effects, and conditions within site effects. Logistic regression was used to analyse differences in the percentage recovery at 12 months. Where there were significant differences by condition, Tukey multiple range tests were conducted to verify which condition or conditions were different in pair-wise comparisons. Statements about the size of an effect or trends were based on Cohen's (1988) effect size f (for multiple groups), with 0.10 being considered small, 0.20 moderate, and 0.40 or more large.

Over the 12-month follow-up, abstinence in Trial 1 was 1,269 days for MET/CBT5, 256 days for MET/CBT12 and 260 days for FSN. Abstinence in Trial 2 was 251 days for MET/CBT5, 265 days for ACRA and 257 days for MDFT.

The number of days of abstinence was not statistically significant by condition after controlling for site and baseline days of abstinence (Cohen's f was 0.06 in Trial 1 and Trial 2).

At the end of the 12-month follow-up, the proportion of adolescents in recovery in Trial 1 was 28% for MET/CBT5, 17% for MET/CBT12 and 22% for FSN. The proportion of adolescents in recovery in Trial 2 was 23% for MET/CBT5, 34% for ACRA and 19% for MDFT.

The percentage of adolescents in recovery was not statistically significant by condition after controlling for site and recovery status in the month before intake (Cohen's f was 0.12 in Trial 1 and 0.16 in Trial 2).

The authors concluded that all five interventions showed significant pre-post treatment effects that were stable over the 12-month follow-up. The clinical outcomes were similar between interventions. No significant differences were observed in terms of the number of days of abstinence, or in the pair-wise differences of the percentage of adolescents in recovery at 12 months.

The measures of benefit used were the days of abstinence over the 12-month follow-up period and the percentage of adolescents in recovery at the end of the study.

The authors did not describe the direct costs included in the analysis. Full details were given in the published cost analysis (French et al. 2002, see 'Other Publications of Related Interest' below for bibliographic details).

The cost estimates were based on data collected using the Drug Abuse Treatment Cost Analysis Program (DATCAP). This tool measured the accounting and opportunity costs by collecting data on programme resources. A total of 12 DATCAPs were completed according to the number of trial arms, treatment approaches and sites. A 6-month timeframe was selected as the study period, so discounting was appropriately not performed. The resource categories were personnel, supplies and materials, contracted services, buildings and facilities, equipment and miscellaneous items. In addition, programme revenues and client case flows were included. All costs attributed to the research component of the study were excluded. Since some interventions involved shared resources across two or more sites, cross-site costs were estimated according to how they were consumed. The average resource quantities used were reported, but only the average total cost per treatment episode for each intervention. The unit costs were not reported, and the source of such costs was unclear in both the parent and the present study. The mean treatment episode costs per intervention, calculated as the product of average weekly cost and length of stay in treatment, were reported. The price year was 1999.

A similar statistical analysis to that used in the effectiveness analysis was carried out for the costs.

The indirect costs do not appear to have been included, which is inconsistent with the stated perspective.

US dollars ($).

No sensitivity analysis of either the costs or benefits was performed.

See the 'Effectiveness Results' section.

In Trial 1, the cost per treatment episode was $1,113 for MET/CBT5, $1,185 for MET/CBT12 and $3,246 for FSN.

In Trial 2, the cost per treatment episode was $1,558 for MET/CBT5, $1,408 for ACRA and $2,002 for MDFT.

Overall, the average cost per treatment episode was $1,758.

The costs and benefits were combined by calculating the cost per day abstinent over the 12-month follow-up period, and the cost per person in recovery at the end of the study.

In Trial 1, the cost per day abstinent was $4.91 for MET/CBT5, $6.15 for MET/CBT12, and $15.13 for FSN. The average cost per day of abstinence over the 12 months post intake was $8.79, and it varied significantly between interventions (Cohen's f 0.48, p<0.05). Pair-wise comparisons using Tukey range tests showed that MET/CBT5 and MET/CBT12 cost significantly less than FSN.

In Trial 2, the cost per day abstinent was $9.00 for MET/CBT5, $6.62 for ACRA and $10.38 for MDFT. The average cost per day of abstinence over the 12 months post intake was $8.65, and it varied significantly between interventions (Cohen's f 0.22, p<0.05). Pair-wise comparisons using Tukey range tests were not significant, owing to variation between sites.

In Trial 1, the cost per person in recovery was $3,958 for MET/CBT5, $7,377 for MET/CBT12 and $15,116 for FSN. The average cost per person in recovery was $8,846, and it varied significantly between interventions (Cohen's f 0.72, p<0.05). Pair-wise comparisons using Tukey range tests showed that MET/CBT5 cost significantly less than the other two interventions, while MET/CBT12 cost significantly less than FSN.

In Trial 2, the cost per person in recovery was $6,611 for MET/CBT5, $4,460 for ACRA and $11,775 for MDFT. The average cost per person in recovery was $7,615, and it varied significantly between interventions (Cohen's f 0.78, p<0.06). Pair-wise comparisons using Tukey range tests showed that ACRA cost significantly less than MET/CBT5, and both cost less than MDFT.

In Trial 1, MET/CBT5 dominated the other conditions (i.e. it was both the most effective and least expensive). In Trial 2, ACRA dominated MET/CBT5 and MDFT, and MET/CBT5 dominated MDFT. Therefore, incremental cost-effectiveness ratios were neither necessary nor appropriate.

Follow-up beyond 12 months was not reported.

Overall, the clinical outcomes were very similar across sites and conditions. However, after controlling for initial severity, the most cost-effective interventions were 5- and 12-session regimens of motivational enhancement therapy and cognitive-behavioural therapy (MET/CBT5 and MET/CBT12) in Trial 1, and the Adolescent Community Reinforcement Approach (ACRA) and MET/CBT5 in Trial 2.

A justification was given for the interventions chosen to be the comparators. The justification was that results from prior studies had been favourable, the interventions had been demonstrated to be effective with related populations, and they had been recommended by expert panels as best practice for adolescent treatment. You should decide if these therapies are representative of interventions for adolescents with cannabis disorders in your own setting.

The analysis used a randomised trial, which was appropriate for the study question. The authors acknowledged the weakness in the study design due to the lack of a no-treatment control group. This means that the pre-post treatment effects that were observed could have been caused by participating in the study itself, a phenomena known as the Hawthorne Effect. The authors' justification for not including a control group was that they considered it to be unethical.

The study sample was randomly selected and was thus assumed to be representative of the study population. However, a high degree of selection was apparent, with 44% of the 1,244 adolescents screened being excluded from participation, and a further 15% of the 702 eligible adolescents declining to participate in the study. This could cause bias in the results. In addition, the two trial groups were not shown to be comparable at baseline.

The method of randomisation was reported, loss to follow-up was low, and the analysis of the clinical study was conducted on an intention to treat basis. However, no power calculations were reported, so it was not possible to ascertain whether the results obtained were due to the intervention or to chance.

The estimation of benefits was obtained directly from the effectiveness analysis.

The authors stated that the study was undertaken from a societal perspective. However, productivity losses do not appear to have been included in the analysis, as is required for this perspective. In effect, as 47% of the adolescents enrolled in the study had reported being employed in the previous 90 days, a societal perspective should have been adopted. In addition, the use of aggregated summary costs made it impossible to know what aspects of costs were included and which health care resources were used.

The costs and the quantities were not reported separately, which means that it would be difficult to rework the analysis for other settings. A statistical analysis of the prices was not presented. Discounting was not applied, which was appropriate given the time horizon of the cost analysis (6 months). The year to which the prices related was reported, and this increases the generalisability of the results.

The authors made appropriate comparisons of their findings with those from other studies. The authors stated that their intention was to generalise their findings to adolescents who present for publicly funded outpatient treatment in the USA. They made extensive comparisons (including socio-demographics, patterns of substance use, and history of behavioural interventions) between the study sample and national admission data for adolescents presenting for treatment of cannabis use problems, and highlighted differences between them. The authors acknowledged that both the clinical and economic outcomes may vary with other populations and settings. The authors do not appear to have presented their results selectively.

The authors acknowledged two further limitations to their study. First, the study relied on participant self report, whereas multiple types of measures would have strengthened the study. Second, the generalisability of the cost estimates to non-experimental settings was uncertain, particularly in view of the large variation in costs that was observed between the study sites.

The authors stated that replication studies are in progress to further evaluate each of the interventions studied. Comparisons of the study's findings with research or practice using other interventions for this patient group are needed. The cost estimates need to be replicated in view of the variation observed between the sites in this study.

Supported by SAM-HSA's Center for Substance Abuse Treatment.

American Society of Addiction Medicine. Patient placement criteria for the treatment of psychoactive substance disorders (2nd ed.). Chevy Chase (MD): ASAM; 1996.

French MT, Roebuck MC, Dennis M, et al. The economic cost of outpatient marijuana treatment for adolescents: findings from a multisite experiment. Addiction. 2002;97 Suppl 1:S84-97.

French MT. Drug Abuse Treatment Cost Analysis Program (DATCAP): Program version user's manual (7th ed.). Coral Gables (FL): University of Miami; 2001.