This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn.

The study analysed the effects of an educational health programme called "Mirame!" (MP) (Note: English translation = "Look at me!"). The MP was based on the Social Learning Theory, and its objective was to promote a healthy lifestyle for school age children and their families. It consisted of a set of 12 educational sessions that used videos, group discussions, brochures and activities, developed with the children, families and community, to inform participants and support behavioural changes.

Other: Health education and health promotion.

Cost-effectiveness analysis.

The study population included fifth- and sixth-grade school children from Chile.

The setting was the community. The economic study was carried out in the Metropolitan Region, Chile.

The effectiveness and resource use evidence related to 1993 to 1996. The price year was 2002.

The effectiveness data were derived from a single study.

The costing was undertaken on the same sample population as that used in the effectiveness study.

Sample size calculations showed that the sample size had a power of 80% to detect differences in risk factor prevalence at the 0.05 significance level. The study population included fifth- and sixth-grade school children from five selected municipalities of the Metropolitan Region. The sample selection was based on a three-stage process. First, counties were selected and assigned as intervention (IP) or reference population (RP). Second, schools within these counties were selected. Finally, fifth- and sixth-grade courses were selected. Details of the selection and allocation methods were not reported. The sample comprised 2,686 children from 12 different schools, of which 1,406 (48.0% female) were allocated to the IP group and 1,280 (48.8% female) to the RP group. The mean age of the participants was 10.71 years in the IP group and 10.78 years in the RP group. There was no evidence that the study sample was representative of the study population.

This was a non-randomised, within-group comparison study. The study comprised three stages:

a prevalence study at baseline for both groups;

a second stage where the MP was carried out; and

a third stage of evaluation, which was a new prevalence study similar in all respects to the initial one.

Finally, outcomes before and after the implementation of the MP were compared measuring the pre- (1993) and post- (1996) prevalence of the indicators selected.

No losses to follow-up were reported.

It was not stated whether an intention to treat analysis was performed. The effectiveness indicators reported were tobacco consumption and alcohol consumption by child gender, estimated as the percentages of boys and girls consuming them. Also reported were the odds ratios (ORs) of consuming tobacco given that alcohol was consumed, consuming marihuana given that tobacco was consumed, and consuming marihuana given that both alcohol and tobacco were consumed. The baseline prevalence was measured through an instrument used and tested by the University of Texas for the Hispanic population in the USA (Amezcua et al. 1990, see ,Other Publications of Related Interest- below for bibliographic details). Before it was applied, a pilot test was carried out to adapt and validate the instrument for the Chilean population.

Statistical analyses were performed, based on the net change criteria, to analyse the difference between the pre- and post-intervention variations observed between the IP and the RP groups. The significance level of each difference was also calculated with the proportion comparison method. Potential adverse effects were not measured. The groups were shown to be comparable in terms of their age and gender. However, comparisons of other relevant risk factors, such as socioeconomic status, were not presented (although socioeconomic data were reported to have been collected).

For boys, alcohol consumption in the IP group was 10.62% of initial prevalence and 21.93% of post-intervention prevalence, giving an absolute difference of 11.31%. The corresponding results in the RP group were 9.10% and 29.86%, with an absolute difference of 20.76%. The net difference (i.e. cases prevented with IP in comparison with RP) between groups was 9.46%, (p<0.001). For girls, alcohol consumption in the IP group was 5.99% of initial prevalence and 23.32% of post-intervention prevalence, giving an absolute difference of 17.33%. The corresponding results in the RP group were 4.80% and 29.40%, with an absolute difference of 25.20%. The net difference between groups was 8.0%, (p<0.0001).

For boys, tobacco consumption in the IP group was 4.45% of initial prevalence and 18.60% of post-intervention prevalence, giving an absolute difference of 14.15%. The corresponding results in the RP group were 5.36% and 26.38%, with an absolute difference of 21.02%. The net difference (i.e. cases prevented with IP in comparison with RP) between groups was 6.87%, (p<0.001). For girls, tobacco consumption in the IP group was 2.46% of initial prevalence and 27.27% of post-intervention prevalence, giving an absolute difference of 24.82%. The corresponding results in the RP group were 2.54% and 35.47%, with an absolute difference of 32.93%. The net difference between groups was 8.11%, (p<0.001).

In relation to the association between variables:

the OR consuming tobacco, given that alcohol was consumed, was 14.9 (95% confidence interval, CI: 10.97 to 20.22);

the OR of consuming marihuana, given that tobacco was consumed, was 24.22 (95% CI: 13.06 to 44.83); and

the OR of consuming marihuana, given that alcohol and tobacco were consumed, was 66.45 (95% CI: 30.56 to 142.87).

The implementation of the MP was related to a significantly greater control of tobacco and alcohol consumption. The intervention had a larger impact on alcohol consumption than tobacco consumption, although the net change was statistically significant in all the intervention groups.

The outcome measure used in the economic analysis was the percentages of cases of alcohol consumption and tobacco consumption prevented. These measures of health benefit were obtained directly from the effectiveness analysis.

The direct costs included in the economic analysis were the education costs incurred by the municipality. These included the direct contribution from the municipality and the subsidy from the state government. The incremental costs due to implementation of the MP were estimated. These comprised the following items: initial contact and diffusion; information talks; materials for 2 years for children, teachers, and schools; delivery expenses, training organisation, activities, and supervision; education activities with parents and tutors, general expenses (e.g. transportation, phone expenses) and overheads. An estimated overhead was included for the final calculation. The costs were adjusted to June 2002, although the method used was not reported. Discounting was carried out using the inflation rate observed when the programme was implemented (1993 - 1994) which, for the baseline analysis, appears to have been 7.5%. The quantities and the costs were not analysed separately, and both appear to have been estimated using actual data. The quantity and cost data were taken from the municipalities. The price year was 2002.

No statistical analysis of the costs was reported.

No indirect costs were reported.

Chilean pesos (CLP) and US dollars ($). No exchange rate was reported.

Univariate sensitivity analyses were performed to investigate the discount rates for benefits (1, 1.5 and 2%) and costs (4 to 7.5%). The method used to select the range was not reported for health benefits, and for the costs it represented the inflation rates for the period when the programme was implemented (1993 - 1994).

See the 'Effectiveness Results' section. For the baseline results, the estimated benefits appear to have been presented undiscounted.

Considering 90 children, the total cost in education for the reference municipalities during 2 years was CLP 46,452,960 ($69,024). The incremental cost due to the MP for the intervention municipalities was CLP 711,000 ($1,057).

The incremental cost was CLP 75,159 ($112) per boy prevented from alcohol consumption and CLP 103,494 ($154) per boy prevented from tobacco consumption.

The incremental cost was CLP 88,875 ($132) per girl prevented from alcohol consumption and CLP 87,670 ($130) per girl prevented from tobacco consumption.

According to the sensitivity analysis, the best cost-effectiveness ratio was observed in boys in relation to alcohol consumption (Range: CLP 70,224 to 76,692) followed by girls in relation to tobacco consumption (Range: CLP 81,913 to 89,005), girls in relation to alcohol consumption (Range: CLP 83,040 to 90,689), and boys in relation to tobacco consumption (Range: CLP 81,094 to 105,606). The programme caused an additional cost per child, for the city hall, of $11.7 in 2 years.

It is possible to apply health promotion interventions in schools with good cost-effectiveness in the short term.

The choice of the comparator was explicitly justified. The justification given was that no intervention was current practice before the educational health programme was implemented. The fact that no intervention was used as the comparator allowed the impact and effects of the strategy to be evaluated. You should judge whether this strategy is relevant in your setting, or whether other comparators from other health promotion strategies could have been relevant as well.

The analysis was based on a non-randomised, within-group comparison study. It is unclear how appropriate the design was for the study question, particularly as the methods of sample selection and allocation were not described. A power calculation and significance level were stated. The population groups were shown to be comparable at analysis in terms of their age and gender, but not for some other relevant confounding factors such as socioeconomic status. Evidence that the study sample was representative of the study population was not reported, and it could not be inferred from the study since the selection method was not described. These facts introduced uncertainty into the reliability of the internal and external validity of the study.

The estimation of benefits was obtained directly from the effectiveness analysis. The measure used (incremental costs per boy or girl prevented from alcohol consumption or tobacco consumption could be useful in this case, but it limits comparability with other economic evaluations in other health fields.

The authors reported an institutional perspective and referred to the Municipal Government as the decision-maker. In that sense, the indirect costs were appropriately not included. In addition, the programme development and research costs were not included as they were funded from sources external to the municipalities. The exclusion of these costs is unlikely to have significantly affected the authors- conclusions. The resource use quantities and prices were not reported separately. No sensitivity or statistical analyses of the costs were conducted. Even when costs were taken from different sources and years, adjustments were carried out but not reported in sufficient detail. The price year was reported.

The authors compared their findings with those from prior studies and studies from other countries, although they stated that there were very few studies of this type published in the literature and none in Chile or in Latin America. The authors- conclusions reflected the scope of the analysis. The authors did not state any further limitations of the study.

The authors highlighted the fact that treatment costs for those who are addicted to drugs, alcohol, or tobacco are far above the costs of this early intervention for health education and promotion of healthy lifestyles. When evaluating the cost-effectiveness of an educational health programme aimed at supporting behavioural changes against tobacco and alcohol consumption, not only the economic burden of these unhealthy lifestyles should be considered, but also the psychological, familial and social burden for a damaged teenager and the damage that he or she might transmit to other generations.

Partially funded by grants from the Finnish International Development Agency/NGO North Karelia Project and FONDECYT.

Amezcua D, Ramirez A, Mc Alister. A su salud: health promotion in a Mexican-American border community. In: Brach N, editor. Health promotion at the community level. Newbury Park (CA): Sage Publications; 1990. p. 191-5.