To systematically review the existing evidence for the effectiveness of farm injury prevention interventions.

Relevant studies from peer-reviewed journals, technical and government reports, and unpublished reports were retrieved. The following databases were searched: MEDLINE, EMBASE, ERIC, PsycINFO, Sociofile, NTIS, Expanded Academic ASAP, Dissertation Abstracts, AGRICOLA, and NIOSHTIC.

Additional relevant information sources were identified by checking references, consulting experts in the field, and by reviewing conference proceedings and technical papers of the National Institute for Farm Safety.

Study designs of evaluations included in the review

All study designs were accepted, including those without comparison groups and those with absent or inadequate evaluation methods. Descriptive studies of farm injuries and risk factors were not included unless they involved a safety intervention component. The included studies generally involved pre-and post-test methodology.

Specific interventions included in the review

Any farm injury prevention interventions including farm safety education programmes; multifaceted farm safety interventions that included environmental revisions and/or a farm visit; and farm safety programmes without completed evaluations. Papers concerning the prevention of agricultural health problems other than injuries, such as respiratory disease or pesticide poisoning, were excluded.

The type of farm safety education interventions examined by the included studies were: safety fairs; day camps; certification programmes; workshops; and courses for families, youth and agricultural workers.

Most interventions involved the dissemination of information through lectures, written materials, testimonials, or demonstrations. Many of the interventions covered general safety information on a variety of activities, such as riding a tractor, handling livestock, and working in grain silos. A couple of exceptions focused primarily on one activity. The time commitment of the participants varied from a couple of hours at a fair or evening programme, to 15 days attending a course in the field.

The multifaceted interventions examined by the included studies werefarm safety audits that involved the provision of specific safety recommendations by safety specialists. Farmers made environmental or equipment improvements, or participated in farm safety education programmes, or undertook a combination of these two activities.

Some included studies (without completed valuations) evaluated educational interventions.

Participants included in the review

The authors did not specify any a priori participant inclusion or exclusion criteria. The farm safety education programmes were targeted mainly at families, with some programmes aimed specifically at children and teenagers who live and/or work on farms, and two interventions aimed at agricultural workers. All of the multifaceted farm safety interventions were aimed at farm operators.

Outcomes assessed in the review

The authors did not specify any a priori outcome inclusion or exclusion criteria. The outcomes assessed included: changes in safety attitudes, knowledge, and/or behaviours; changes in the incidence of farm injuries; the number of worker compensation claims; changes in productivity; the number of training sessions provided to hired workers; and the effectiveness of cash incentives.

How were decisions on the relevance of primary studies made?

The titles and abstracts identified in the literature search were reviewed. Studies that potentially met the inclusion criteria were retrieved for further review, and any disagreements regarding their inclusion were resolved by consensus of both authors.

Validity was not formally assessed, although some information relating to study design and methodological quality was provided in the tables and text of the review.

The authors do not state how the data were extracted for the review, or how many of the reviewers performed the data extraction. Data were extracted from the studies for the following categories:author and year; intervention; study design; outcomes (if any), results (if available); and comments about study design and validity.

How were the studies combined?

A narrative synthesis was undertaken, with studies grouped according to the type of intervention. Publication bias was not assessed.

How were differences between studies investigated?

Differences between the studies were investigated in the narrative.

Twenty-five studies were included. The specific study designs were not reported for some of the studies, and the number of participants was not reported for any of the studies.

Farm safety education interventions (11 studies).

The only intervention that compared actual injury rates before and after the intervention found a reduction in injuries from 21 injuries/1000 work days to 12. Many of the studies reported increases in the correct responses of participants to questions on farm safety, more appropriate attitudes about farm safety and/or self-reported (or intended) changes in farm safety behaviours after the interventions. Most of these studies had design limitations, such as a lack of comparison groups and a reliance on self-reported outcomes.

Multifaceted interventions that included environmental revisions and/or a safety audit (5 studies).

All of these studies reported some positive changes in outcomes following the intervention. One study found a 27% decrease in the number of worker compensation claims filed after the intervention. Researchers conducted random inspections of 25% of the farms and found that 95% of hazard corrections had been made as was self-reported. However, the number of worker compensation claims for the intervention group was not compared with that of the control group, to determine if factors other than the intervention were responsible for some or all of the decrease.

In one randomised controlled trial of 200 farmers, injury rates dropped from 33.4 to 20.1 injuries per 100,000 work hours for the intervention group (p<0.05), and improvement in safety behaviours occurred for 66 working routines. No significant changes in these outcomes were observed in the control groups.

Programmes without completed evaluations (9 studies).

Some of the studies had evaluations planned or in progress whilst others did not. Only two study designs included the collection of data on injuries.

The included studies differed in types of interventions, target populations, study designs, outcome measures, and sample sizes.

Methodological problems made it difficult to assess the efficacy of many of the farm interventions reviewed. In addition, the evidence for the effectiveness of farm safety education programmes was weak due to inadequate study designs.

The multifaceted farm safety interventions provided greater evidence of efficacy and, in some cases, had more adequate designs, e.g. randomisation to intervention groups and the use of controls.

There is a need for more rigorous evaluations of farm safety intervention programmes. The study designs could be improved by including the randomisation of study participants when appropriate, the use of control groups, and the objective measurement of outcomes such as behaviour change and injury incidence.

The review question was stated clearly. However, the authors did not specify any inclusion criteria for the participants or outcomes assessed.

The literature search was comprehensive and identified unpublished studies. However, the authors did not report whether any foreign language papers were identified.

The overall methodological quality of the studies was discussed, but no formal validity assessment of the primary studies was undertaken. Some details concerning the primary studies were absent, such as sample size and the specific study design used. The data were pooled appropriately in a narrative synthesis. Some details of the review process were provided, such as how decisions were made on the relevance of the primary studies, whereas other details were not, such as how the data were extracted from the primary studies.

The authors' conclusions follow on from their findings.

Practice: The authors did not state any implications for practice.

Research: The authors recommended a number of methodological improvements in the evaluation of farm safety interventions.

1. A control group should be used when appropriate to account for the effects of extraneous variables, such as the effect of taking the pre-test alone, external events other than the intervention that occur between pre- and post-tests, and the maturation of participants.

2. The participants in the study should be randomised to intervention and control groups, to avoid selection bias.

3. When possible, the focus should be on outcomes such as behaviour change and injury incidence, rather than measuring only knowledge, attitudes, or intended behaviour change.

4. When possible, outcomes should be objectively measured or self-reported outcomes should be verified.

5. When possible, lasting changes in outcomes should be evaluated by conducting post-tests weeks or months following the intervention.

6. Appropriate statistical tests should be used to compare changes in pre- and post-tests for the intervention groups and controls.

After determining which types of interventions work, research should focus on the best ways to implement the programmes in the farm community.

Farm safety programmes and evaluations for hired farm workers should be considered due to the lack of published studies pertaining to this group.

National Institutes of Health, National Institute of Environmental Health Sciences, grant number 5 T 32ES07262; National Institute of Occupational Safety and Health, Division of Safety Research.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.