|
Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunised children in a health maintenance organization |
Lieu T A, Capra A M, Makol J, Black S B, Shinefield H R |
|
|
Record Status This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn. Health technology Sending letters, automated telephone messages, or both to families with underimmunised children aged 20 months, in order to improve immunisation rates.
Economic study type Cost-effectiveness analysis.
Study population The study population included underimmunised children aged 20 months. The definition of underimmunisation status was based on the minimum immunisations that should have been given by 20 months of age. Because some of the children enrolled in the health plan after 42 days would have received prior immunisations, not always recorded in the health plan's computerised tracking system, the rules of underimmunisation were based on the child's age at enrolment in the health programme. The following definitions were used: from birth to 42 days: 4 diphtheria-tetanus-polio (DTPs), 3 oral polio vaccine (OPVs), 1 measles-mumps-rubella (MMR), 1 hepatitis B (HepB), 1 Haemophilus influenza type B (Hib) at 12 months or older; 43 to 98 days: 3 DTPs, 2 OPVs, 1 MMR, 1 HepB, 1 Hib at 12 months or older; 99 to 154 days: 2 DTPs, 1 OPVs, 1 MMR, 1 HepB, 1 Hib at 12 months or older; 155 to 350 days: 1 DTPs, 1 MMR, 1 HepB, 1 Hib at 12 months or older.
Setting The setting was primary care: medical centres of the Kaiser Permanente Medical Care Program of Northern California. The economic evaluation was carried out in the United States.
Dates to which data relate The effectiveness and resource use data were obtained during the period September 1996 to January 1997. The price year was 1997.
Source of effectiveness data The estimate/evidence for final outcomes was derived from a single study.
Link between effectiveness and cost data The costing was not undertaken on the same patient population as that of the effectiveness study. The costing was undertaken retrospectively for the whole patient population enrolled in the health plan. It was estimated that 30,000 children would reach 24 months of age in the health plan in one year: 22% of these would be underimmunised.
Study sample 752 patients were initially randomised using the Kaiser Permanente immunisation tracking system. It was estimated that 160 children in each intervention group would have 80% power to detect a 16% difference in the percentage of children receiving any immunisation during the 4 months after their families were sent the message(s). 162 children were randomised to the Letter intervention, 165 to the Phone, 154 to the Letter-Phone and 167 to the Phone-letter. 67 (9%) of the 752 children were excluded because they had a gap in health plan enrolment between 20 and 24 months of age and their follow-up data may have been incomplete. There were 219 children in the comparison group.
Study design The study design was a randomised controlled trial, with each patient being randomised to one of the four interventions. It was deemed unethical to have a comparison group with no intervention, so a comparison group of historical controls was retrospectively studied. The participating clinics were asked temporarily to suspend all local efforts to telephone or send letters regarding immunisations during the study period. The duration of follow-up was four months. The method of randomisation was not reported. There were no losses to follow-up. The letters and telephone messages were personalised. The telephone messages were sent by the Customer-Activated Appointment Processing Services (CAAPS). The messages were approximately 1 minute long and allowed the listener to choose the language in which the message was delivered (English, Spanish, and Cantonese). Telephone numbers that could not be reached were called again the next evening. Up to six calls were made each evening.
Analysis of effectiveness The analysis of effectiveness was based on intention to treat. The primary health outcomes used in the analysis were the percentage of underimmunised children who received any needed vaccinations by 24 months and the percentage of underimmunised children who were fully immunised by 24 months. The comparability of the intervention groups was not analysed.
Effectiveness results The percentages of underimmunised children who received any needed vaccination by age 24 months were:
44.2% ((95% CI: 36.6 - 51.9) for Phone;
43.8% (95% CI: 36.1 - 51.5) for Letter;
53.3% (95% CI: 45.7 - 60.9) for Phone-letter;
57.8% (95% CI: 49.9 - 65.7) for Letter-phone.
The result for the Letter-phone group was statistically significantly different from the groups receiving single messages only (p<0.05). The phone-letter intervention was also more effective than either message alone, but the differences only approached statistical significance (p=0.09 with letters and p=0.10 with phone messages).
The percentages of underimmunised children who received all needed vaccination by age 24 months were not statistically different (p=0.11) but suggested that a phone message (36.4%, 95% CI: 29 - 43.8) was similar to a letter (37.7%5, 95% CI: 30.1- 45.2) and two messages 1 week apart (47.4%, 95% CI: 39.5 - 55.4, for letter-phone and 45.5%, 95% CI: 37.9 - 53.1, for phone-letter) were better than either a letter or a phone message alone. In the comparison group of no intervention, 35.6% received a needed immunisation by their 24-month birthday.
Clinical conclusions A letter followed by a telephone message was the most effective option to improve the immunisation rates among underimmunised children in this HMO setting. The next most effective methods were a telephone message followed by a letter, a telephone message and finally, a letter.
Modelling A decision tree was used to estimate the cost-effectiveness of different alternatives.
Measure of benefits used in the economic analysis The number of immunised children by 24 months was the measured benefit in the economic analysis.
Direct costs Costs were not discounted due to the short time horizon of the study (four months). Quantities and costs were not reported separately. The direct costs included the start-up costs in software programming amortised over 10 years, costs of mailing and costs of phone calls. The cost/quantity boundary adopted was that of the health maintenance organization (HMO). The estimation of the quantities and costs was based on consultation with Kaiser Permanente's administrators and study personnel. The source of quantity and cost data was not reported. The price year was 1997. Some costs were reflated when necessary using the Medical Services Component of the Consumer Price Index.
Statistical analysis of costs The costs were assessed based on the decision tree model and no statistical analysis was performed.
Indirect Costs No indirect costs were analysed.
Sensitivity analysis A sensitivity analysis was performed for two different strategies, an automated telephone message strategy and sending postcards instead of letters.
Estimated benefits used in the economic analysis Based on the assumption that, under a strategy of no intervention, 35.6% of the 10,248 underimmunised 20 month old children (3,648 children) would receive an immunisation by 24 months, even if there were no intervention, the number of additional children immunised by 24 months relative to no intervention was:
8,392 for Phone;
9,024 for Letter,
16,216 for Letter-Phone.
The Phone-letter strategy results were not reported. The length of follow-up was 4 months. The long term benefits of immunisation were not included in the analysis.
Cost results The total annual costs for 10,154 underimmunised 20 month old children were $8,392 for Phone, $9,024 for Letter and $16,216 for Letter-Phone.
Synthesis of costs and benefits Estimated benefits and costs were combined into cost per child immunised by 24 months relative to no intervention. Relative to no intervention the cost per child immunised by 24 months of age was $9.80 for a telephone message alone, $10.50 for a letter alone and $7 for letter-phone. By adding letters to an existing telephone strategy the incremental cost-effectiveness would be $8.30 per child immunised by 24 months of age. Under the assumptions in the sensitivity analysis, automated telephone messages would cost $2.20 per child immunised under the public immunisation clinic cost assumption and, if postcards were as equally effective as letters, they would cost $6.50 per child immunised.
Authors' conclusions For underimmunised 20 month old children in this HMO setting, letters followed by automated telephone messages were more effective and more cost-effective than either type of message alone. Compared with letters alone, automated telephone messages alone were equally effective and more cost-effective.
CRD COMMENTARY - Selection of comparators The comparators were selected according to the purpose of study and no explicit justification was provided. It is questionable why the cost-effectiveness of each intervention strategy was measured compared to no intervention, when the authors specified that no intervention could not be used as a comparator in the effectiveness study. You, as a user of the database, should decide if these are widely used technologies in your own setting.
Validity of estimate of measure of benefit The analysis was based on a randomised controlled trial, which was appropriate for the study question. The study sample appears to have been representative of the study population. Patient groups were not shown to be comparable at analysis and this is problematic, given that different languages were represented in the study population and that the group who received no intervention was analysed retrospectively from historical data. The method of patient randomisation to the intervention groups was not specified. No comparison of groups was performed. The estimation of benefits was obtained using a decision tree based on the results from the effectiveness analysis.
Validity of estimate of costs Positive aspects of the cost analysis were that all categories of costs relevant to the perspective adopted were included in the analysis, the price year was reported, and, for each category of costs, all relevant costs were included. Some costs were omitted from the analysis, such as immunisation costs, but these would have been common to all interventions. Some negative aspects were that quantities and costs were not reported separately, the specific source of quantities and costs was not reported, and a statistical analysis of costs and quantities was not conducted.
Other issues The authors made appropriate comparisons of their results with findings from other studies and the issue of generalisability to other settings was addressed. The authors do not appear to have presented their results selectively. The study enrolled 20 month old underimmunised children and this was reflected in the authors' conclusions. Limitations reported by the authors included the possibility of including other reminder strategies, and the possibility of modelling the long terms costs and effects of an immunisation programme. As the authors concluded, the cost-effectiveness could vary widely in different populations and the results of this study should be carefully applied.
Implications of the study The authors suggest that the rate of children's immunisations could be improved by sending letters and automated telephone messages.
Source of funding Supported by the Innovation Program of Northern California Kaiser Permanente and by the Vaccine Safety Datalink Project of the Centres for Disease Control and Prevention.
Bibliographic details Lieu T A, Capra A M, Makol J, Black S B, Shinefield H R. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunised children in a health maintenance organization. Pediatrics 1998; 101(4): E31-E37 Indexing Status Subject indexing assigned by NLM MeSH Comparative Study; Cost-Benefit Analysis; Female; Health Maintenance Organizations; Humans; Immunization /economics /utilization; Immunization Programs /economics /utilization; Infant; Information Systems; Male; Patient Compliance; Postal Service /economics; Reminder Systems /economics; Research Support, Non-U.S. Gov't; Research Support, U.S. Gov't, Non-P.H.S.; Telephone /economics AccessionNumber 21998000529 Date bibliographic record published 30/06/2001 Date abstract record published 30/06/2001 |
|
|
|