This is a critical abstract of an economic evaluation that meets the criteria for inclusion on NHS EED. Each abstract contains a brief summary of the methods, the results and conclusions followed by a detailed critical assessment on the reliability of the study and the conclusions drawn.

The study analysed the effect of different approaches for improving personnel training in ambulance services in Mexico. These were:

in Monterrey, the Pre-Hospital Trauma Life Support (PHTLS) course or Basic Trauma Life Support (BTLS) course, conducted annually or semi-annually; and

in San Pedro, the BTLS course, Advanced Cardiac Life Support (ACLS), plus a locally designed airway management refresher course.

Other: Pre-hospital and trauma care.

Cost-effectiveness analysis.

The study population included trauma patients in large urban areas where the training interventions were implemented in two ambulance services, as well as the population of the control area.

The setting was emergency medical services (EMS) in Monterrey, San Pedro and Santa Catarina, Mexico. The economic study was carried out in Mexico.

The effectiveness evidence referred to October 1994 to June 1995 for Monterrey, and January 2000 to June 2001 for San Pedro and Santa Catarina. The resource use dates were not reported. The price year was not reported.

The effectiveness data were derived from a single study.

The costing method was not reported. Only training costs were considered.

The total numbers of trauma patients treated by each EMS during the study period were 866 (Monterrey), 510 (San Pedro) and 504 (Santa Catarina; control group). In all three sites the patients were primarily young (average of 27 to 32 years old), male (65%) and with blunt mechanisms of injury (76%). The severity of injury scores were assessed using the Prehospital Index (PHI). Most of the patients had a 3-point average score. The patients were divided into approximately equal Before and After groups. Despite some differences between the various sites, the Before and After groups were well matched demographically. The authors did not report the methods of sample selection and matching, nor power calculations.

This was a quasi-experimental before-and-after study with a control group, which included trauma patients in three EMS. No blinding method for the outcome assessment or independent validation of the information recorded was reported. In all of the sites, deaths were recorded if they occurred at the scene of the event or en route to the hospital. In Monterrey, deaths were recorded in the emergency department (ED) if the doctors who transported the deceased patient were present and were informed about the death. In San Pedro and Santa Catarina, follow-up was obtained from medical records at area hospitals for those at high risk of mortality. Access to medical records was provided for patients thought to be at high risk of death. For those patients who died, the site of death was recorded. Access to these records was needed because many of these patients were transported by other ambulance services and deaths en route would otherwise have not been captured.

The effects of the interventions were evaluated in the analysis. These included spinal immobilisation, airway and breathing management, fluid resuscitation, scene time and mortality rates. The groups were comparable at baseline and no adjustments for confounding factors were reported. A statistical analysis was performed, using the unpaired Student's t-test for continuous data and the chi-squared test or Fisher's exact test for categorical data.

The frequency of the use of cervical and thoracic immobilization increased in Monterrey from 39% before to, respectively, 63% and 67% after, (p<0.001). There were no changes in San Pedro. The changes in the control community (Santa Catarina) were non statistically significant.

When considering all patients, the use of oropharyngeal airways, intersuctioning and oxygen administration increased significantly in both the intervention groups (Monterrey and San Pedro), but not so in the control group where the results were statistically non significant.

For all the patients, the administration of intravenous (IV) fluids increased significantly in both of the intervention groups, but not in the control group where the results were statistically not significant.

Scene times (time taken by pre-hospital providers from arrival at scene until transporting patient from scene) did not change between the before and after periods for any of the three study sites. In Monterrey, the mean values for scene times were 5.7 (+/- 4.4) minutes in the before period and 5.9 (+/- 6.8) minutes afterwards. In San Pedro, the times were 9.6 (+/- 7.3) minutes before and 10.9 (+/- 35.6) minutes after. In Santa Catarina, they were 13.8 (+/- 11.5) minutes before and 15.5 (+/- 29.5) minutes after. Although there were differences between the sites, none of the differences were significant.

The overall mortality rate decreased non significantly in Monterrey following training (10.3% before to 7.4% after, (p=0.13). The corresponding rates did not change in San Pedro or in the control group of Santa Catarina. The overall mortality rate included those patients who were in extremis and who died at the scene with or without treatment. For those patients who were transported, the mortality rate in Monterrey declined from 8.2% before to 4.7% after, (p=0.04). Regardless of whether all deaths or only those of patients transported were considered, there were no changes in the differences between the before and after periods for San Pedro or Santa Catarina.

The training modalities evaluated in this study resulted in an increase in the use of pre-hospital interventions, including basic airway manoeuvres and IV fluid resuscitation for patients in shock. Increased training for advanced life support (ALS)-level interventions did increase the provision of endotracheal intubation. However, no improvement in relevant outcomes such as mortality was documented. The mortality rate for trauma victims was decreased only in a sub-group of patients at the one service for which it had been the highest, in the before period and the period in which the simplest and lowest cost training interventions were used. Advanced airway management and other ALS-level interventions may or may not ultimately prove useful for trauma victims in this and other environments.

No summary measure of benefit was used in the economic evaluation. The costs and effects were left disaggregated and the study was therefore considered a cost-consequences analysis.

The costs assessed in the study only included those associated with the training. The potential costs of increased use of supplies and equipment associated with the improved process of care were not assessed. The source of the quantity and cost data was not reported, and the quantities and the costs were not reported separately. Discounting was not reported, and this was not appropriate since many of the interventions recurred over at least 2 years. The price year was not reported.

No statistical analysis of the costs was performed.

No indirect costs were reported.

Mexican pesos ($). These were converted to US dollars ($) but the exchange rates were not reported.

As the interventions in San Pedro required a 3-month period to implement, and the main analysis included this period as part of the before period, a supplemental analysis was performed in which the data from this period were excluded. There were no changes in the major findings of the study, whether or not the 3-month intervention period was considered as part of the before period.

See the 'Effectiveness Results' section.

All of the standardised courses (PHTLS, BTLS and ACLS) cost approximately $150 per student in Monterrey. The local airway course cost $50 per student. In San Pedro, the cost of the intervention (i.e. BTLS, ACLS and the airway course) was $400 per doctor.

The annual cost in Monterrey to organise either a PHTLS or a BTLS course once or twice per year involved the registration of around 10 to 15 medics per year. This represented about 0.5% of the budget of the ambulance service.

Not relevant.

According to the authors, the data presented in this study provided some useful conclusions on the effectiveness of the training interventions conducted. Both components of the study demonstrated an improvement in basic airway management. In San Pedro, after more intensive airway management training, there was also an increase in the use of advanced airway management techniques. Spinal immobilisation application increased only in Monterrey where it had been the lowest during the before period. All of these changes were in comparison with the control group, which remained unchanged. Throughout the study period, the mortality rate decreased only in Monterrey (although not significantly) where it had been the highest during the before period. The more extensive changes in the process of care in Monterrey, as well as the decreased mortality, occurred after the most basic and lower cost interventions of Pre-Hospital Trauma Life Support (PHTLS) were provided.

A justification was given for the comparator used. It reflected standard practice in the authors' setting before implementation of the training intervention. Also, improving EMS is probably one of the most effective means of reducing trauma deaths in pre-hospital care. A limitation could be that some of the courses were not described in detail. You should decide if this represents a widely used intervention in your own setting.

The analysis was based on a quasi-experimental study design which, although not as methodologically sound as a randomised controlled trial, could be helpful in evaluating the effectiveness of the interventions. There may be concerns about sample size and power calculations, which were not reported. These facts should be taken into account when considering the results. The authors reported that statistical analyses were undertaken to take potential biases and confounding factors into account, but not enough details were provided. Given these limitations, the internal validity of the effectiveness results may be quite low.

No summary measure of benefit was derived. Thus, the reader is referred to the comments in the 'Validity of estimate of measure of effectiveness' field (above). The limitations concerning the sample size and power necessary to assure reasonably results are also relevant here.

The perspective of the study was not reported. There was too little detail on the cost estimation and only training costs were considered. Although it was not stated, some relevant costs could have been omitted from the analysis. The cost categories were not reported in detail and their omission might have affected the authors' conclusions. The costs and the quantities were not reported separately, which will not enable the analysis to be easily extrapolated to other settings. Sources of the cost data were not reported. All these factors could affect the robustness of the cost results. A statistical analysis of the costs was not reported. Discounting was necessary for some costs, such as the annual or semi-annual repetition of the courses over a period longer than 2 years. The price year was not reported, which will make any future reflation exercises difficult. Overall, the costing was very limited with no real detail reported.

The authors compared their findings with those from other studies. They did not directly address the issue of the generalisability of the results to other settings, although they implied that larger patient samples across a multi-site setting would increase the generalisability to other developing countries. The authors stated several limitations to the study. First, the time period used for Monterrey was different to that of the other two sites. Second, the data relied on self-reporting by the doctors on the ambulance run sheets. Third, there was a difference in the assessment of death in San Pedro and Santa Catarina compared with Monterrey. Finally, the potential costs of increased use of supplies and equipment associated with the improved process of care were not taken into consideration. These limitations might lead to bias when interpreting the study results.

According to the authors, in the current setting of existing basic ambulance services in Latin America, it seems that the biggest gains are to come from assuring the use and quality of basic life support capabilities and rapid transport. Moreover, the educational approaches that provided this capacity in this study (PHTLS/BTLS) cost less than more intensive interventions (preceding courses plus ACLS, plus an extra airway course). One option for assuring such basic life support capabilities is short, in-service training such as PHTLS/BTLS. The utility of more widespread acquisition of these degrees would be aided by promoting better standardisation of training curricula on a nationwide or regional basis.

Funded by a grant from the Medic One Foundation.