|The costs and effects of introducing selectively trained radiographers to an A&E reporting service: a retrospective controlled before and after study
|Brealey S, King D G, Hahn S, Godfrey C, Crowe M T, Bloor K, Crane S, Longsworth D
This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn.
The study examined the introduction of selectively trained radiographers reporting accident and emergency (A&E) radiographs of the appendicular skeleton and remaining body area.
Economic study type
The study population comprised patients presenting to A&E and requiring radiographic evaluation.
The setting was a hospital. The economic study was carried out in the UK.
Dates to which data relate
The effectiveness and resource use data were gathered from February 1993 to January 1997. The costs were expressed using 2001/02 values.
Source of effectiveness data
The effectiveness evidence was derived from a single study.
Link between effectiveness and cost data
The costing was not performed on the same sample of patients as that used in the clinical study.
Power calculations were not reported. To select the reports, a random sample of 3 days for each quarter was chosen during a period of 2 years. A sample of 400 A&E plain radiograph reports was selected from a list of all examinations. Of these, 200 were extracted from the hospital database for the period prior to the implementation of the selectively trained radiographer service (February 1993 to January 1995), while 200 were obtained after the implementation of the service (February 1995 to January 1997). Owing to the long timeframe of the analysis, a further sample of general practitioner (GP) plain radiograph reports was selected as a non-equivalent, non-intervention control group, in order to assess the potential impact of historical trends affecting the validity of the primary comparison. The characteristics of the reports were not provided, but the reports were stratified by time and body area.
This was a retrospective comparative study with an historical control that was carried out at a single centre. The Systems and Networks Service Department provided a list of all A&E and GP examinations performed on the selected days. A random sample of 25 A&E and 25 GP reports was selected for each quarter, stratified by body area. The reports were followed up for one year. The arbiter who compared reports for concordance was blinded to who had produced each report.
Analysis of effectiveness
The analysis of effectiveness was restricted to available reports with complete data. The clinical measures used were accuracy (including sensitivity and specificity), impact of A&E reports on subsequent management of patients, and patients' re-attendance, which was used as a proxy for outcome.
In relation to accuracy, a blinded consultant radiologist with 10 years' experience at a different hospital assessed the reports for concordance. A&E radiographers' reports were compared with a reference standard report made by a radiologist with 11 years experience. A normal report was defined as within normal limits, no body injury, or a minor abnormality such as an old fracture. An abnormal report was defined as all clinically relevant abnormalities. Reports were only judged concordant if they agreed on the presence, location and type of abnormality.
In relation to the assessment of management, an A&E specialist registrar decided whether this might have a clinically important effect on patient management compared with the reference standard report. Three possible options were available: no difference in management, clinically unimportant difference in management, or clinically important difference in management.
In relation to the outcome assessment, patient re-attendance to the A&E or the X-ray department indicated that significant pathology was missed at the first visit. Three outcomes were possible: patient did not re-attend, patient re-attended X-ray department for an unrelated examination, or patient re-attended X-ray department for a related examination.
The comparability of the study groups was not discussed.
In the pre-service period, the sensitivity was 50% (95% confidence interval, CI: 36 - 64), the specificity was 97% (95% CI: 94 - 99) and the accuracy was 87% (95% CI: 81 - 91).
In the post-service period, the sensitivity was 51% (95% CI: 36 - 66), the specificity was 94% (95% CI: 89 - 97) and the accuracy was 86% (95% CI: 80 - 90).
Thus, there was no statistically significant difference in the A&E radiograph reporting accuracy. In general, sensitivity and specificity were higher for appendicular skeleton than for the remaining body area, both before and after intervention. For the non-equivalent, non-intervention control group, there were no statistically significant differences before and after the intervention in the sensitivity, specificity and accuracy of GP radiograph reporting.
Fifty-six A&E reports (27 before and 29 after the intervention) were judged to be incorrect. The clinical details of five patients (two before and five after the intervention) were not available and were not considered. The percentages of patients falling within the three management options were identical both before and after the intervention. In particular, there were only two incorrect reports before and after the intervention that might have had an important clinical effect. This led to a 11% reduction of cases that might have had a clinically important effect on patient management. However, this difference did not reach statistical significance.
The same number of patients (4 in each group; 15% and 14%, respectively) re-attended for related X-ray examination and the subsequent report was discordant with the original.
The effectiveness analysis showed that comparable outcomes were observed in the pre- and post-intervention periods.
Measure of benefits used in the economic analysis
No summary benefit measure was used in the economic evaluation because no statistically significant differences between the clinical outcomes were observed. In effect, a cost-minimisation analysis was performed.
The analysis of the costs was performed from the perspective of the X-ray department. The health services included in the economic evaluation were the capital costs of providing radiographers with the equipment and facilities to report, the costs of training radiographers, and the overhead costs of the room used for reporting activity. Consumable costs were not considered since they were common to both professions and were negligible. The unit costs were not presented separately from the quantities of resources used for most items, but time to report A&E radiographs was reported. The source of the costs was not explicitly stated, but it was likely to have been the authors' institution. Resource use was based on the whole number of A&E radiographs of the appendicular skeleton performed during 2001/02 (n=9,713). The time spent reporting was estimated on the basis of the experience of the radiographers and radiologists involved in the study. The costs were expressed using 2001/02 prices. Discounting was not relevant as the costs per patient were incurred during a short timeframe. Capital costs were annualised, using an annual interest rate of 6%, to produce annual equivalent costs.
Statistical analysis of costs
The costs were treated deterministically.
The indirect costs were not considered.
Sensitivity analyses were carried out to test the impact of changes in assumptions about the inclusion of certain costs. A univariate sensitivity analysis appears to have been carried out, and best and worst scenarios were considered. The authors defined alternative assumptions for the cost analysis.
Estimated benefits used in the economic analysis
See the 'Effectiveness Results' section.
In the base-case scenario, the annual cost was 7,137.03 (average cost per radiograph reported 0.73) with the radiographer service and 7,498.17 (average cost per radiograph reported 0.77) with the radiologist service. Thus, the difference in annual costs was -361.14 (average cost-difference per radiograph reported: -0.04).
The sensitivity analysis showed that, if the secretaries typed radiographer reports, the average cost per radiograph would fall from 0.73 to 0.66.
Different results were found in the best and worst scenarios. Thee best-case scenario for the radiographer service would save the X-ray department 4,528 per year. The worst-case scenario for the radiographer service would cost the X-ray department an extra 4,524 per year.
Synthesis of costs and benefits
A synthesis of the costs and benefits was not relevant as a cost-minimisation analysis was performed.
The introduction of reporting by selectively trained radiographers did not have an adverse impact on Accident and Emergency (A&E) radiograph reporting accuracy, patient management, or outcomes. However, cost-savings to the X-ray department were observed. These could increase as radiographers acquire the same experience as the radiologists, and if secretaries type the radiographers' reports.
CRD COMMENTARY - Selection of comparators
The selection of the comparators was appropriate as conventional reporting was compared with reporting by selectively trained radiographers in A&E. You should decide whether they are valid comparators in your own setting.
Validity of estimate of measure of effectiveness
The effectiveness analysis was based on a retrospective study. Patient reports were randomly selected from all reports available in the hospital database. The use of random selection was appropriate to identify patients within each study period. However, the allocation of reports to the study groups was based on the period during which patients were admitted to the A&E department. The two study groups were not assessed concurrently. The authors attempted to control the potential impact of historical trends by observing some clinical outcomes in a third study group, namely, the sample of GP reports, which was used an external non-comparative group. The analysis revealed that no major changes in the hospital institution were observed, thus the impact of factors other than the intervention under examination could be ruled out. However, the baseline comparability of the two groups of reports was not discussed.
Both interventions were evaluated by a single experienced consultant radiologist who defined the 'gold' standard. The potential impact of bias was minimised by using blinded assessment and introducing concordant reports among discordant ones that required further evaluation. No justification for the sample size was provided, thus it was unclear whether statistically significant differences could have been observed with a larger sample of reports. The accuracy of the analysis depended on the quality of case note extraction. The evidence came from a single institution, and caution is therefore required when extrapolating the results of the analysis to other hospitals.
Validity of estimate of measure of benefit
No summary benefit measure was used in the analysis because a cost-minimisation analysis was conducted. Please refer to the comments in the 'Validity of estimate of measure of effectiveness' field (above).
Validity of estimate of costs
The analysis of the costs was consistent with the perspective adopted in the study. A justification for the exclusion of some categories of costs was provided. The source of the costs was unclear, although it might have been the authors' institution. The unit costs were not reported, but some information on resource use was. The authors acknowledged the uncertainty surrounding some resource use and cost estimates, which were varied in the sensitivity analysis, the results of which were clearly reported. It was noted that a detailed cost analysis, including capital costs, was performed. The price year was reported, which makes reflation exercises in other settings possible.
The authors reported the results of a cost analysis that had shown the extra costs associated with the implementation of the radiographers' service. However, the limitations of this analysis were pointed out. The issue of the generalisability of the study results to other settings was not explicitly addressed, but some sensitivity analyses on key cost estimates were performed. The analysis highlighted the fact that, while the reporting of radiographs had a dramatic impact on activities performed by the radiographers, there was a negligible change to the radiologists' work. Further investigation of this issue was limited by the retrospective nature of the data collection. Very different results were found in terms of the costs under the worst- and best-case scenarios.
Implications of the study
The study results suggested that selectively trained radiographers can accurately report A&E plain radiographs at no additional cost.
Brealey S, King D G, Hahn S, Godfrey C, Crowe M T, Bloor K, Crane S, Longsworth D. The costs and effects of introducing selectively trained radiographers to an A&E reporting service: a retrospective controlled before and after study. British Journal of Radiology 2005; 78(930): 499-505
Other publications of related interest
Loughran CF. Reporting of fracture radiographs by radiographers: the impact of a training programme. Br J Radiol 1994;67:945-50.
Brealey S, King DG, Crowe MT, et al. Accident and Emergency and General Practitioner plain radiograph reporting by radiographers and radiologists: a quasi-randomised trial. Br J Radiol 2003;76:57-61.
Piper K, Paterson AM, Ryan C. The implementation of a radiographic reporting service for trauma examinations of the skeletal system, in Four National Health Service Trusts. London: Research & Development Directorate, South Thames Regional Office (NHSE); 1999.
Subject indexing assigned by NLM
Clinical Competence; Education, Continuing; Emergency Service, Hospital /economics /organization & England; Health Services Research; Hospital Costs; Humans; Outcome Assessment (Health Care); Radiography /economics; Radiology /education /organization & Radiology Department, Hospital /economics /organization & Retrospective Studies; Sensitivity and Specificity; administration; administration; administration
Date bibliographic record published
Date abstract record published