This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn.

A programme of financial incentives to promote smoking cessation was examined. The programme, the "Quit and Win" (QW) contest offered a chance to win a cash prize (usually $1,000) for successfully stopping smoking for at least 1 month. The current study reported on 11 QW contests that were carried out in different communities across New York State between 2001 and 2004. In the majority of contests, an easy-to-read newspaper insert was used to advertise the QW contest. The insert was also sent directly to households, and a Spanish version was used when relevant. The insert was also available in local hospitals and clinics, in retail and business outlets, and on websites. The smoking status of the contest winner was based on three criteria:

verbal confirmation that the selected contestant had not smoked any cigarette during the contest month;

validation of the self-reported smoking status of contest winners by asking three individuals, whose names were provided by each contestant, to confirm that the winner had not smoked during the contest month; and

the administration of a carbon monoxide breath test within 24 hours of selection to validate the winner's status as a nonsmoker.

Primary prevention.

Cost-effectiveness analysis.

The study population comprised cigarette smokers aged 18 year or older.

The setting was the community. The economic study was carried out in the USA.

The effectiveness and resource use data were gathered from 2001 to 2004. The price year was not reported.

The effectiveness evidence was derived from a single study.

The costing was carried out prospectively on the same sample of patients as that used in the effectiveness study.

Power calculations, if performed, were not reported. An overall sample of 5,504 individuals participated in the 11 QW contests, ranging from 1,683 in the largest programme to 112 in the smallest. More than 8,000 adults representative of the New York State population were enrolled in the 2003-2004 Adult Tobacco Survey (ATS); these were implicitly regarded as the control group. Data on age, gender and race were collected for both individuals who participated in the contests and those enrolled in the ATS, but these data were only reported graphically and separately by counties.

This was a prospective cohort study with a historical control, which was carried out in New York State. Follow-up was conducted 4 to 6 months after each contest via a combination of mailed paper surveys and telephone interviews. In the counties that enrolled a large number of participants, random samples of individuals were selected for follow-up. Of the initial sample of 5,504 individuals, 2,756 were available at follow-up (50.1% response rate, range: 28 to 91.9). The mean time from enrolment to follow-up interview was 4.8 months (range: 4.1 to 6.1).

The analysis of the clinical study was restricted to individuals who were available at follow-up. However, some results were also reported for the intention to treat population. The primary outcome measure was the quit rate, which was based on self-reported smoking at the time of follow-up. Participants who reported not smoking currently and also reported no cigarettes smoked in the 7 days prior to the interview were defined as having quit. The percentages of individuals who tried to quit during the contest and those who quit during the contest were also reported.

A secondary outcome was programme reach. This calculated in two ways: the ratio of the numbers of smokers enrolled in the contest divided by the estimated number of smokers in the county; and the ratio of the numbers of smokers enrolled in the contest divided by the estimated number of smokers who were likely to be exposed to advertisements in that county. The quit methods used by contestants were also reported. Finally, a regression analysis was performed to identify predictors of smoking cessation.

In terms of the baseline comparability of the study groups, the authors stated that contest participants were similar to smokers from the general populations. However, there was a tendency for contestants to be younger and smoking more cigarettes than individuals in the general population.

In terms of the programme reach, the average proportion of smokers recruited to participate in the contest was 0.55% (range: 0.07 to 1.38) when total number of smokers in the county was used as the denominator.

The average programme reach was 2.18% when the denominator of smokers was limited to those who were likely to be exposed to advertisements.

The programme reach doubled if it was assumed that half of smokers so exposed actually read the promotional material.

The proportion of individuals who tried to quit during the contest ranged from 84.6 to 93.3%.

The proportion of those who quit during the contest ranged from 48.2 to 71.7%.

The quit rate at follow-up (main measure) ranged from 21.7 to 48.8%, with a mean of 31%.

Among contestants who made a quit attempt, the majority said that the contest was a strong reason for quitting.

The proportion of individuals who made a quit attempt in the previous year (as found from the ATS) was 21.5%.

The statistical analysis showed that, in 8 of the 11 QW contests, the quit rate at follow-up was significantly higher than the ATS quit rate.

The median proportions of individuals using specific quit methods in the QW contests and in the ATS were, respectively:

7.4% and 3.5% for clinic,

36.5% and 25.1% for nicotine replacement therapy,

14.8% and 30.5% for drug prescriptions,

4.0% and 1.8% for calls to quitlines,

44.9% and 41.9% for cutting back the number of cigarettes, and

13.0% and 9.8% for switch to lights.

The adjustment of quit rates for factors such as age, gender, race, baseline smoking rate and duration of follow-up did not change the results of the analysis.

Those who reported that they remained abstinent for the duration of the contest were more likely to report not smoking at follow-up than those who reported that they had smoked during the month of the contest (odds ratio 1.3, 95% confidence interval: 1.1 to 1.5).

The effectiveness analysis showed that QW contests were effective in improving smoking quit rates.

The measure of benefit that was combined with the cost was the quit rate. This was derived directly from the effectiveness analysis.

The viewpoint of the analysis was not explicitly stated. The analysis was restricted to programme costs, which included the costs of prizes offered to smokers and paid advertising to promote the contest. Personnel costs associated with organising the contest and validating contest winners were not included. The source of the costs was not clear. Resource use referred to the sample of patients considered in the effectiveness study. The costs in the control group were assumed to have been zero. Discounting was not relevant and was not performed. The price year was not reported but the costs were gathered from 2001 to 2004.

Statistical analyses of the costs were not performed.

The indirect costs were not included in the economic analysis.

US dollars ($).

Sensitivity analyses were not carried out.

See the 'Effectiveness Results' section.

The costs of the programme ranged from $4,345 to $91,444 (mostly depending on the number of individuals who participated in the county), with a median of $25,928.

The cost per contestant ranged from $30.48 to $125.09, with a median of $75.

The cost per participant was slightly correlated to market size, suggesting that larger markets spent more to recruit a contestant than did smaller markets.

Average and incremental cost-effectiveness ratios were calculated in order to combine the costs and benefits of the smoking cessation programme over no intervention (ATS control group).

The average cost per quit ranged from $96.87 to $387.45.

An incremental analysis was performed for those communities where the quit rate among contestants was higher than the estimated quit rate in the general population (8 counties out of 11).

The cost per attributable quit was calculated as the cost of promoting the contest divided by the number of quits attributable to the programme. This ranged from $301.07 to $953.72.

The results of this study support the use of financial incentives to induce more smokers to make quit attempts. Financial incentives could increase the quit rate among smokers for a relatively modest investment of resources.

The choice of the comparator was appropriate as the new intervention was compared with quit rates in the general population. The financial incentives and methods of advertising methods were reported in detail. You should decide whether they are valid comparators in your own setting.

The effectiveness data were derived from a clinical study in which a prospective group of individuals was compared with a historical control group. Such a design was necessary because the study intervention and the comparator were implemented in two different time periods. Since the two groups of patients were not assessed concurrently, factors other than the study interventions might have affected the results of the analysis. The authors pointed out that both geographical differences and temporal trends could have biased the analysis, although a regression analysis took the impact of baseline sociodemographic factors into account.

A large sample of individuals was enrolled, but the appropriateness of the sample size was not explicitly justified through the use of statistical tests. A longer follow-up period would have been helpful in assessing the long-term impact of the intervention. The analysis of effectiveness was mainly based on those individuals who were available at follow-up, but the results of an intention to treat analysis were also reported. The evidence came from several counties in New York State, which means that the study sample should be representative of the general population, although some baseline differences were observed. A potential limitation of the analysis was that follow-up quit rates were not validated using biochemical instruments but relied on self-reported data. These issues should be considered when assessing the validity of the effectiveness analysis.

The use of an intervention-specific benefit measure limits the possibility of making comparisons with the benefits of other health care interventions. However, quit rates are widely used to capture the benefits of smoking cessation programmes.

The analysis of the costs was restricted to the expenses associated with the programme, and it assumed that the costs of the comparators were not relevant. However, the perspective chosen for the analysis was not explicitly stated. Similarly, out-of-pocket expenses, which might be a substantial cost for individuals who attempt to quit smoking, were not considered. Future savings related to smoking cessation were also not considered. The impact of changes in the cost estimates was not investigated and sensitivity analyses were not performed. The costs were presented as a macro-category and a detailed breakdown of the cost items was not provided. Therefore, it is not possible to replicate the analysis in other settings. The period during which the costs and resource use data were gathered was reported, but the price year was not. This will hinder the possibility of reflating the cost results in other time periods. The costs were treated deterministically but the correlation between costs and other factors was investigated.

The authors did not compare the results of their analysis with those from other published studies, stating instead that little evidence was available on the cost-effectiveness of financial incentive programmes for smoking cessation. The issue of the generalisability of the study results was not addressed and no sensitivity analysis was performed. Thus, the external validity of the study appears low and the study results should be considered relevant only for a US context. The key issues of the study appear to have been the lack of a standard control group as in a clinical trial, the short follow-up for evaluating quit rates amongst smokers, and the limited details of the economic analysis.

The authors suggested that future studies should evaluate how to boost participation rates among smokers. The use of a prospective, randomised study would be appropriate to assess the cost-effectiveness of QW contests.

Supported by the New York State Department of Health.

Because readers are likely to encounter and assess individual publications, NHS EED abstracts reflect the original publication as it is written, as a stand-alone paper. Where NHS EED abstractors are able to identify positively that a publication is significantly linked to or informed by other publications, these will be referenced in the text of the abstract and their bibliographic details recorded here for information

Korhonen T, Su S, Korhonen HJ, et al. Evaluation of a national Quit and Win contest: determinants for successful quitting. Prev Med 1997;26:556-64.

Croghan IT, O'Hara MR, Schroeder DR, et al. A community-wide smoking cessation program: Quit and Win 1998 in Olmsted County. Prev Med 2001;33:229-38.

Tillgren P, Haglund BJ, Ainetdin T, Holm LE. Who is a successful quitter? One-year follow-up of a National Tobacco Quit and Win contest in Sweden. Scan J Soc Med 1995;23:193-201.