|
Evaluating two implementation strategies for whiplash guidelines in physiotherapy: a cluster-randomised trial |
Rebbeck T, Maher C G, Refshauge K M |
|
|
Record Status This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn. Health technology The study examined two strategies for the implementation of acute whiplash guidelines in physiotherapy. An active implementation strategy, involving education from opinion leaders, was compared with a passive dissemination by mail.
Type of intervention Other: Information and education in health.
Economic study type Cost-effectiveness analysis.
Study population The study population comprised patients presenting to physiotherapy clinics with acute whiplash. The inclusion criteria were a minimum age of 18 years, involvement in a motor vehicle accident within the past 6 weeks, sustained whiplash-associated disorder Grade I - III, and patient prepared to give informed consent.
Setting The setting was physiotherapy clinics in two Australian states, New South Wales (NSW) and Australian Capital Territory (ACT). The economic study was carried out in Australia.
Dates to which data relate The guidelines were released in 2001. The study began patient enrolment in July 2001 and continued until December 2002, although enrolment was initially planned for 12 months. The price year was not reported.
Link between effectiveness and cost data The costs were measured both before and after guideline implementation for both strategies. Pre-trial costs were obtained from the insurer's database and during-trial costs were derived by audit of patient notes.
Study sample A total of 48 clinics were invited to participate, of whom 27 consented to do so. Each clinic provided 1 physiotherapist. The characteristics of the participating and non-participating physiotherapists did not differ except in location (more participants in ACT). Similarly, there were no differences between the implementation group (n=14) and dissemination group (n=13) in billing history or knowledge of the guidelines. Of those recruited, 8 physiotherapists failed to recruit any patients (1 in the implementation group, 7 in the dissemination group); the characteristics of recruiters did not vary from those of non-recruiters. The authors noted that the enrolment period was extended from an initial 12 months by a further 6 months because the target number of patients had not been met; the target was not reported. Of the 103 patients that entered the study, 72 were in the implementation group and 31 in the dissemination group.
Study design The authors conducted a multi-centre, cluster-randomised trial, with physiotherapists being the unit of randomisation. Physiotherapists were first stratified into low- and high-cost providers and members of each stratum were then randomised to the implementation or dissemination groups. Randomisation was done by an insurer, from whom allocation was concealed by coding the interventions. In addition, stratification was concealed from the study centre. The physiotherapists were informed that they were randomised into one of two implementation groups. One physiotherapist in the dissemination group withdrew (reason not given).
The patients were followed up for 12 months following injury. Of 103 patients who entered the study, 4 withdrew because they were referred elsewhere by their general practitioner (GP) (n=1), they became pregnant (n=1), or reason unspecified (n=2). The poorest follow-up was at 6 months (72%). Follow-up at 12 months was 90% when 6 patients were lost due to moving overseas (n=1), no longer residing at the same address (n=3), or non-contactable after 5 attempts (n=2).
Analysis of effectiveness Disability was measured using the Functional Rating Index. Disability due to acute whiplash was measured more specifically using an adapted version of the 7-item Core Outcome Measure for neck pain, which was reduced to 5 items. Clinically important change (Global Perceived Effect), patient satisfaction and physiotherapist satisfaction were measured using Likert scales. Physiotherapist knowledge of the guidelines was measured using a custom-made questionnaire developed for the study. Physiotherapy clinical practice was measured as the percentage of physiotherapists prescribing guideline recommendations (from responses to the questionnaire and patient notes). The analysis was conducted on the basis of treatment completers only. The groups appear to have differed substantially in gender and distribution of whiplash severity. However, the statistical significance of these differences was not tested, nor accounted for in the analysis.
Effectiveness results No significant differences were found among the patient groups at any follow-up point in the measures of disability or clinically important change.
Both groups were equally satisfied with care provided by their GP, (p=0.69), their physiotherapist, (p=0.87), and with the consumer version of the guidelines (p=0.93).
Physiotherapists in the implementation group increased their overall knowledge of the guidelines by 5.5 points (95% confidence interval, CI: 2.5 to 8.4) more than physiotherapists in the dissemination group.
Improvements were also seen in specific areas. For example, self-reported understanding of the guidelines increased by 1.5 points (95% CI: 0.7 to 2.3). In addition, 2 of the 5 guideline recommendations ("reassure patient" and "advise to act as usual") were identified by more physiotherapists in the implementation group than in the dissemination group, (p=0.05 and p=0.02), and were prescribed more during the trial, (p=0.04 and p=0.02).
Physiotherapists were equally satisfied with the guidelines, (p=0.29), and their implementation package, (p=0.07).
Clinical conclusions The authors concluded that the active implementation programme resulted in improved knowledge and clinical practice more consistent with the guidelines. However, patient outcomes did not differ between the groups. The authors suggested that this could have been due to a high baseline quality of treatment, or the possibility that some elements of the guidelines are not essential.
Measure of benefits used in the economic analysis The authors did not derive a summary measure of benefit, although they chose to use a 1-point improvement on the Functional Rating Index to illustrate cost-differences between the groups.
Direct costs The costs were measured before the trial from an insurer database and during the trial from audit of patient notes, thus allowing the authors to compare the increase in costs between implementation strategies. The authors stated that the costs were analysed using linear regression for continuous data, adjusted for before-trial score. Summary total costs were reported for the different patient groups before and after the trial. No further details of resources or unit costs were provided. The price year was not reported.
Statistical analysis of costs The cost of care was analysed using linear regression for continuous data, adjusted for before-trial score. Significance levels were set at p<0.05 throughout the study. The median costs per patient per therapist were reported.
Indirect Costs No productivity costs were included.
Currency Australian dollars (AUD).
Sensitivity analysis A sensitivity analysis was not undertaken.
Estimated benefits used in the economic analysis See the 'Effectiveness Results' section.
Cost results The total median cost per patient per physiotherapist in the implementation group increased from $606 (standard deviation, SD=452) before the trial to $1,092 (SD=1,099) after. This was not significantly different, (p=0.67), from the cost increase for patients in the dissemination group from $627 (SD=489) to $1,408 (SD=1,342). These were costs associated with 12 months of treatment post-injury.
Synthesis of costs and benefits Average cost-effectiveness ratios were calculated despite there being no significant differences in the incremental costs or comparative effectiveness.
The cost per 1-point improvement on the Functional Rating Index was $116 for the implementation group. This was not significantly different from the $189 for the dissemination group, (p=0.55).
Authors' conclusions Patient outcomes and costs of care were not affected by the implementation strategies for the clinical guidelines.
CRD COMMENTARY - Selection of comparators Although no explicit justification was provided, the comparator of passive dissemination of guidelines appears to have represented current practice in the authors' setting. You should decide whether this is an appropriate comparator in your own setting.
Validity of estimate of measure of effectiveness The analysis was based on a cluster-randomised trial design, which was reasonable given the study objective and intervention type (the authors noted their intention to "minimise contamination between patients"). The authors demonstrated that physiotherapists entering the study and patients subsequently recruited were representative. However, some baseline differences in the patient groups might have affected the results, but these were not investigated further. Despite the randomisation, adjustments to deal with confounding might have been necessary. In addition, a treatment-completer rather than intention to treat approach to the data analysis was taken. Appropriate statistical techniques were used to account for any cluster effect. The method of randomisation, length of study and follow-up were reported clearly, thereby clarifying the internal validity of the study. However, since no power calculations were provided it is impossible to verify whether the results are robust. Finally, many of the instruments used to measure the outcomes were adapted from other instruments or were unique to the study and, therefore, are not validated or commonly accepted measures of the outcomes they purported to measure. This casts some doubt on the external validity of the study.
Validity of estimate of measure of benefit No summary benefit measure was employed. In effect, a cost-consequences analysis was performed. However, the authors used a 1-point improvement on the Functional Rating Index scale in a type of illustrative average cost-effectiveness calculation. Incremental cost-effectiveness is a more informative evaluation and caution should be exercised when interpreting the meaning of average ratios.
Validity of estimate of costs It is impossible to judge the completeness, accuracy or validity of the cost estimates as no details were given, apart from their source. Discounting was not required given the length of follow-up (12 months). However, no price dates or price adjustments (for inflation or price proxying) were reported. The cost data were thus insufficient to permit a meaningful critique and any generalisation from the authors' setting will be extremely limited.
Other issues The authors compared their findings with those of other studies and showed them to be in agreement. They did not acknowledge any variation in costs across settings, or evaluate the impact of any variations on the results of the study. The authors do not appear to have presented the results selectively, but it would have been helpful had they explained some of the more unusual results in more detail. The authors' conclusions may be reasonable but, given the issues highlighted, it is still unclear whether the results are robust even in the authors' own setting. The authors did not discuss further limitations to their study.
Implications of the study The authors suggest that the effectiveness of an implementation programme depends upon the current level of clinical practice. They also suggest that there are no barriers to the implementation of guidelines in terms of either patient or physiotherapist satisfaction with the guidelines, or of cost.
Source of funding Funded by the Motor Accidents Authority of New South Wales.
Bibliographic details Rebbeck T, Maher C G, Refshauge K M. Evaluating two implementation strategies for whiplash guidelines in physiotherapy: a cluster-randomised trial. Australian Journal of Physiotherapy 2006; 52(3): 165-174 Indexing Status Subject indexing assigned by NLM MeSH Adult; Attitude of Health Personnel; Cluster Analysis; Consumer Behavior; Cost-Benefit Analysis; Education, Continuing /methods; Female; Guideline Adherence; Humans; Male; Outcome Assessment (Health Care); Physical Therapy Modalities /economics /standards; Physical Therapy Specialty /education; Practice Guidelines as Topic; Whiplash Injuries /rehabilitation AccessionNumber 22006001950 Date bibliographic record published 31/07/2007 Date abstract record published 31/07/2007 |
|
|
|