|
A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes |
Shiffman R N, Freudigman K A, Brandt C A, Liaw Y, Navedo D D |
|
|
Record Status This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn. Health technology A guideline implementation system, using handheld computers, for the office management of asthma exacerbations in paediatric patients was examined. The guidelines of the American Academy of Pediatrics (AAP) recommended the use of physiologic measures (peak expiratory flow rate, PEFR, and oxygen saturation) to better assess exacerbation severity, increased usage (both frequency and dosage) of beta2-agonists, and increased use of corticosteroids and oxygen. The intervention was implemented using a computer with custom-designed software that provided reminders for the fulfilment of recommendations.
Economic study type Cost-effectiveness analysis.
Study population The study population comprised children aged between 5 and 18 years who presented with acute exacerbations of asthma. As the intervention was directed at physicians, the inclusion criteria for paediatricians covered the availability of equipment in their offices for PEFR measurement and for providing supplemental oxygen therapy. Paediatricians in academic practices, physicians in training, and sub-specialists in allergy and pulmonology were excluded.
Setting The setting was community-based care. The economic study was conducted in New Heaven (CT), USA.
Dates to which data relate The effectiveness and resource use data were gathered from September 1996 to October 1998. The price year was not reported.
Source of effectiveness data The effectiveness evidence was derived from a single study.
Link between effectiveness and cost data The costing was conducted prospectively on the same sample of patients as that used in the effectiveness study.
Study sample Power calculations were conducted in the preliminary phase of the study. These suggested that 10 physicians enrolling 10 patients each would have provided an 80% power to detect a 20% difference in adherence rates. Sample size calculations were then performed retrospectively. These showed that the study had a 90% power to detect statistically significant differences in missed school, missed work, and office re-visits with the sample size used in the analysis.
The paediatricians were drawn from a pool of 375 paediatricians listed in the 1996 Fellowship Directory of the AAP. Those listed in Connecticut cities and towns within a 20-mile radius of New Haven, who anticipated seeing 20 eligible patients within the following year and who had equipment available in their offices, were screened for eligibility. Initially, 138 (80 residents, fellows; 58 academic) were found to be ineligible. Of the 237 remaining available, a further 22 were ineligible (mainly because they were not in active practice) and 18 declined. The paediatricians were then randomly selected. Of the final sample of 11 paediatricians, two dropped out (one moved out of state and one had an excessive workload). The remaining 9 paediatricians enrolled 10 consecutive patients each (one enrolled 11 children), resulting in a sample of 91 children in the control group. Subsequently, the same 9 paediatricians enrolled 74 consecutive patients (six enrolled 10 patients each, while three enrolled 8, 5 and 1 patient) for the intervention group. The mean age was 10.3 years in the control group (age range: 5 - 17.4) and 10.8 years in the intervention group (age range: 5 - 17.8).
Study design This was a prospective comparative study with historical controls, which was conducted in several primary care centres. The patients were identified in two different timeframes. A random system based on number tables was used to select paediatricians from the 1996 Fellowship Directory of the AAP. The physicians and patients were blinded to the hypothesis of the study. The patients were followed for 7 to 14 days and one of the study authors contacted them by telephone. Seven patients in the control group and 6 patients in the intervention group were lost to follow-up. The physicians received $200 as partial compensation at the end of the study. Paediatricians were not constrained to follow the guidelines suggested by the computerised system.
Analysis of effectiveness The analysis of effectiveness was limited to those patients who provided complete data. The outcomes used in the analysis were:
the adherence rate, defined as the proportion of visits at which the physicians performed an intervention following guideline recommendations;
the number of PEFR measurements;
the number of oxygen saturation measurements;
the number of nebulisation treatments; and
a series of immediate and intermediate outcomes, such as improved asthma severity after discharge, immediate disposition home or to the emergency department (ED)/direct hospitalisation, missed school or missed work days for caretakers at follow-up, and the number of office re-visits, ED visits, and hospitalisations at follow-up.
The authors did not comment on the comparability of the two groups, but stated that the intervention patients had a more severe disease than the control patients. Accordingly, a covariate analysis was performed, taking this potential confounding factor into consideration.
Effectiveness results The average adherence rate was:
0.86 in the control group and 0.94 in the intervention group for PEFR assessment, (p=0.320);
0.29 (control) versus 0.56 (intervention) for oxygen saturation assessment, (p=0.007);
0.73 (control) versus 0.91 (intervention) for metered-dose inhaler or nebulisation, (p=0.064);
0 (control) versus 0.11 (intervention) for oxygen therapy, (p=0.139); and
0.43 (control) versus 0.57 (intervention) for the prescription of systemic corticosteroids, (p=0.055).
The mean number of PEFR measurements was 1.6 (control) versus 2.2 (intervention), (p=0.001).
The mean number of oxygen saturation measurements was 0.48 (control) versus 1.1 (intervention), (p=0.017).
The average number of nebulisation treatments was 0.77 (control) versus 1.2 (intervention), (p=0.026).
In terms of immediate patient outcomes (from presentation to discharge), there was a reduction in severity of asthma symptoms for 43.3% of the control group patients versus 57.7% of the intervention group patients. This differences approached statistical significance, (p=0.069). However, differences in intermediate outcomes after 1-month follow-up (missed school or missed workdays for caretakers and number of office re-visits, ED visits and hospitalisations) did not reach statistical significance.
Physicians commented that they often disagreed with the recommendations made in the guidelines.
Clinical conclusions The effectiveness study showed that the implementation of guidelines led to an increase in the number of assessments and office treatments. However, no beneficial effects in terms of immediate or intermediate outcome measures were observed.
Measure of benefits used in the economic analysis The health outcomes were left disaggregated and no summary benefit measure was used in the economic analysis. In effect, a cost-consequences analysis was conducted.
Direct costs Discounting was not relevant since the costs per patient were incurred during a short time. The unit costs were not presented separately from the quantities of resources used. The health services included in the economic evaluation were prescriptions, duration and number of visits, tests and treatments performed in the office. The cost/resource boundary of the study was unclear. The costs were based on total physician fees for the visit. Resource use was derived from actual individualised data, which referred to the sample of patients who were considered in the effectiveness study. Data forms were used and resource usage was estimated from 1996 to 1998, but the price year was not reported.
Statistical analysis of costs The t-test was used to test the statistical significance of differences in the estimated fees. An analysis of covariance was conducted to assess the impact of potential confounding factors on the estimated charges.
Indirect Costs The indirect costs were not considered in the economic analysis.
Sensitivity analysis Sensitivity analyses were not conducted.
Estimated benefits used in the economic analysis See the 'Effectiveness Results' section.
Cost results The paediatricians' average fee rose from $103.11 in the control phase to $145.61 in the intervention phase. The increase in charge was mainly due to the higher number of tests and treatments performed after the guidelines had been implemented. The duration of the visit was also significantly longer in the intervention group. The impact of the guidelines was confirmed in the analysis of variance.
Synthesis of costs and benefits The costs and benefits were not combined as a cost-consequences analysis was performed.
Authors' conclusions The implementation of guidelines for the management of asthmatic children presenting with exacerbations to primary care providers, led to an increase in the costs of care, without a significant improvement in immediate and intermediate outcomes.
CRD COMMENTARY - Selection of comparators The rationale for the choice of the comparator was clear and appropriate. Standard care was selected to reflect the treatment practice before the implementation of the new guidelines. You should decide whether it represents a valid comparator in your own setting.
Validity of estimate of measure of effectiveness The analysis of effectiveness used a comparative study with historical controls, which the authors considered to be the most common design for evaluating computer-based, clinical decision support systems. However, as noted in the paper, time-dependent confounding factors could have affected the results of the study since the outcomes were assessed in two different timeframes. The authors performed an analysis of covariance to investigate the impact of potential confounders. The paediatricians were selected using a random procedure, which enhanced the robustness of the analysis conclusions. Consecutive patients were enrolled and the study sample is likely to have been representative of the study population. Power calculations, both in the preliminary phase of the study, and after the size of the sample was defined, were performed to ensure that the study was powered to detect statistically significant differences in several outcome measures. The method of sample selection was described in detail. These issues tend to enhance the validity of the analysis. However, the analysis of effectiveness did not consider those patients who were lost to follow-up.
Validity of estimate of measure of benefit No summary benefit measure was used in the analysis because a cost-consequences analysis was conducted.
Validity of estimate of costs The authors did not explicitly report the perspective adopted in the study. Hence, it was unclear whether all the relevant categories of costs were considered in the economic evaluation. The costs were estimated using fees, which might not have been appropriate to assess the economic impact of the guidelines. Limited information on resource use and unit costs was reported. The price year was not provided. These issues limit the possibility of replicating the study and carrying out reflation exercises in other settings. Statistical tests were conducted when the costs were compared, but no sensitivity analyses were performed. The cost estimates were specific to the study setting.
Other issues The authors compared their findings with those from other studies and stated that similar results were observed in the literature. In terms of the generalisability of the study results to other settings, the authors noted that their sample of physicians was representative of paediatric practice in Connecticut. However, due to practice variations, the results of the analysis should not be extrapolated to settings with different epidemiologic characteristics or treatment patterns. The study referred to children with asthma who presented with exacerbations and this was reflected in the authors' conclusions.
Implications of the study The study results suggested that more research should be conducted before implementing guidelines for the management of asthmatic children. Physicians tended to resist some explicit recommendations, in particular the administration of oxygen therapy. Therefore, caution is required when implementing guidelines that have not yet been validated.
Source of funding Supported by the Robert Wood Johnson Foundation and the National Library of Medicine, grant numbers 1-R29-LM 05552-01A1 and T-15-LM07056.
Bibliographic details Shiffman R N, Freudigman K A, Brandt C A, Liaw Y, Navedo D D. A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes. Pediatrics 2000; 105(4): 767-773 Other publications of related interest American Academy of Pediatrics, Provisional Committee on Quality Improvement. Practice parameter: the office management of acute exacerbations of asthma children. Pediatrics 1994;93:119-26.
Worrall G, Chaulk P, Freake D. The effects of clinical practice guidelines on patient outcomes in primary care: a systematic review. CMAJ 1997;156:1705-12.
Indexing Status Subject indexing assigned by NLM MeSH Adolescent; Ambulatory Care; Asthma /therapy; Child; Child, Preschool; Decision Making, Computer-Assisted; Decision Support Techniques; Guideline Adherence; Humans; Office Management; Practice Guidelines as Topic; Treatment Outcome AccessionNumber 22000000706 Date bibliographic record published 30/09/2004 Date abstract record published 30/09/2004 |
|
|
|