This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn.

The study assessed strategies designed to promote population-based behaviour change in milk consumption, namely, to switch from high-fat milk (classified as whole milk or 2%) to low-fat milk (1% or skim). The promotional strategies were combinations of paid advertising, media relations and community-based educational activities. Paid advertising included television, radio and newspaper advertisements. Media relations consisted of events designed to generate coverage in the local news media. Community-based educational activities included events based in grocery stores, schools and workplaces. The four different types of campaign combinations investigated were:

paid advertising, media relations and community education;

paid advertising and media relations;

media relations and community education; and

paid advertising alone.

The intervention was a form of health promotion.

Cost-effectiveness analysis.

The study population comprised residents of four individual West Virginia communities in the USA.

The setting was the community. The economic study was carried out in West Virginia, USA.

The effectiveness and resource use data referred to 1996 to 1998. The price year was not reported.

The effectiveness data were derived from a single study.

The cost data were calculated during the effectiveness study.

The authors did not report power calculations. The method of selecting the participating communities was not reported. Four communities were chosen, one for each strategy under investigation. Their population sizes ranged from 18,000 to 35,000. A matched comparison community was selected as the control for each campaign community. The authors stated that the campaign communities all had similar demographics and similar budgets for the intervention at baseline, and that their media markets did not overlap.

The study had a quasi-experimental population-based case control design, with some aspects implemented using a before-and-after design. Each campaign was started at a different time, using a rolling field experiment design. The media campaigns lasted 6 to 8 weeks and follow-up continued for up to 12 months after the campaign ended, with the exception of one community which had an additional follow-up 2 years after the campaign had ended. Approximately 400 adults in each community were selected for telephone interview using random-digit dialling. Of the adults interviewed at baseline, 68% were re-interviewed at follow-up in Clarksburg, 73% in Wheeling, 67% in Parkersburg and 67% in Beckley.

The analysis of effectiveness included all communities included in the effectiveness study. The primary outcome was the increase in consumption of low-fat milk, as measured by consumption survey self-reports and supermarket milk sales. The primary outcome measure was expressed in terms of effect sizes (r and d), which were interpreted according to Cohen's conventions. Specifically, a small effect size was 0.1<r<0.2 or 0.2<d<0.4; a medium effect was 0.3<r<0.4 or 0.5<d<0.7; and a large effect was r>=0.5 or d>=0.8. The effect size r refers to Pearson's r, the sample correlation coefficient. The effect size d refers to Cohen's d.

The Clarksburg community received the combination of paid advertising, media relations and community education. Low-fat milk sales increased from 18% (as a percentage of total fluid milk sales) pre-campaign to 41% post-campaign, with effect sizes of d=1.67 and r=0.64, (p=0.003).

The Wheeling community received the combination of paid advertising and media relations. Low-fat milk sales increased from 29% pre-campaign to 46% post-campaign, with computed effect sizes of d=1.58 and r=0.62, (p=0.013).

The Parkersburg community received the combination of media relations and community education. Low-fat milk sales increased from 28% pre-campaign to 34% post-campaign, with computed effect sizes of d=0.77 and r=0.36, (p=0.215).

The Beckley community received paid advertising alone. Low-fat milk sales increased from 23% pre-campaign to 28% post-campaign, with computed effect sizes of d=0.02 and r=0.01, (p=0.965).

The percentage of high-fat milk drinkers who switched to low-fat milk, according to self-report, was higher than that in the control group for all four communities, (p<0.001 for Clarksburg, Wheeling and Parkersburg, and p=0.01 for Beckley).

The authors concluded that mass media campaigns can be effective in changing population behaviour.

The measure of benefits used was the number of people switching from high-fat to low-fat milk consumption.

The study included the costs of conducting the campaign, such as the costs of paid advertising, personnel salaries, travel, communications, incentives and meetings. The authors excluded the costs of evaluation, telephone survey and milk sales data collection. It may be assumed that the prices were market prices. Discounting was not relevant for the base-case analysis which had a time horizon of less than 1 year. The date to which the price data referred was not specified.

The costs were treated deterministically. This was appropriate as, for each particular campaign strategy, the sample size was one. Hence, statistics could not be calculated.

The indirect costs were not included in the analysis and were not relevant.

US dollars ($).

A sensitivity analysis was not undertaken.

The number of surveyed individuals who switched consumption from high-fat to low-fat milk was approximately:

83,480 (30%) in Clarksburg (paid advertising, media relations and community education),

75,270 (18%) in Wheeling (paid advertising and media relations),

4,304 (13%) in Parkersburg (media relations and community education), and

32,670 (9%) in Beckley (paid advertising alone).

The measurements were made by comparing self-reported consumption immediately before the campaign to self-reported consumption immediately after the 6- to 8-week campaign.

The total campaign cost was:

$61,000 in Clarksburg (paid advertising, media relations and community education),

$43,000 in Wheeling (paid advertising and media relations),

$51,000 in Parkersburg (media relations and community education), and

$51,000 in Beckley (paid advertising alone).

The costs and benefits were combined to calculate the cost per person who switched consumption from high-fat to low-fat milk.

The cost per person who switched was estimated to be $0.73 in Clarksburg, $0.57 in Wheeling, $11.85 in Parkersburg and $1.56 in Beckley.

The combinations of paid advertising, media relations and community education (Clarksburg) and paid advertising and media relations (Wheeling) may be the most cost-effective means of altering population-based behaviour.

The comparators were selected to represent a combination of possible media strategies. You should consider whether the strategies investigated are relevant in your own setting. Matched control communities were selected to allow the comparison with "no campaign".

The measure of effectiveness was derived from a single study, the design of which was appropriate for the study question. The communities taking part in the study were small, rural cities and the authors stated that they were comparable at analysis. The analysis of effectiveness was handled credibly, but more could have been done to address the issue of loss to follow-up. The authors conducted a complete case analysis that did not account for censoring or missing data.

The measure of benefit was obtained directly from the effectiveness analysis. The authors did not attempt to link an intermediate outcome (switch from high-fat to low-fat milk consumption) to long-term health outcomes, which may make it difficult to interpret the value of the treatment effect. The authors stated that the short length of follow-up prevented them from measuring any direct changes in weight.

The authors did not state the study perspective. They included the direct costs of conducting the campaign, but excluded protocol-driven costs related to trial evaluation. The costs were not reported separately from the quantities. The prices may be assumed to have been based on market prices in the study setting. A statistical analysis of the costs was not undertaken; this might have been due to the limited sample size. The date to which the prices referred was not reported, and it was not clear whether the costs of each campaign related to the same price year.

The authors compared their results with those from other studies. They acknowledged that the generalisability of the results is likely to be limited, mainly because of the small number of communities and the lack of random assignment. The authors highlighted some further limitations, including the fact that their campaigns were implemented in small-city markets, as opposed to large media markets where the promotion issues faced would vary. The results were reported in full and appear to have reflected the scope of the analysis.

The authors did not make any recommendations for further research. They suggested "dieticians and other health educators should consider the combination of paid advertising and media relations as a central dietary change strategy".

Funded in part by a grant from the Office of Epidemiology and Health Promotion of the West Virginia Bureau for Public Health.

Because readers are likely to encounter and assess individual publications, NHS EED abstracts reflect the original publication as it is written, as a stand-alone paper. Where NHS EED abstractors are able to identify positively that a publication is significantly linked to or informed by other publications, these will be referenced in the text of the abstract and their bibliographic details recorded here for information.