The authors concluded that behaviour-based safety interventions were associated with a statistically significant reduction in workplace accidents or injuries, but the poor quality of the studies means that these results should be interpreted with caution. There were limitations to this review but, overall, the authors' conclusion appears to reflect limited evidence from poor-quality studies.

To evaluate the effectiveness of behaviour-based safety (BBS) interventions in reducing accidents and injuries in the workplace.

MEDLINE, Ergonomics, Expanded Academic Index, Compendex, Health and Safety Science Abstracts, the Social Sciences Citation Index and PsycINFO were searched for peer-reviewed studies published in the English language up to January 2004; the search terms were reported. In addition, other studies by the most prominent authors in the field were tracked and reference lists in all related reports and reviews were screened.

Study designs of evaluations included in the review

Inclusion criteria were not specified in terms of the study design. All but one of the included studies were before-and-after studies; the other study was a non-randomised controlled study.

Specific interventions included in the review

Studies that evaluated BBS interventions in the workplace were eligible for inclusion. Studies that evaluated off-the-job safety interventions, interventions set in laboratories under simulated conditions and training-only interventions were excluded. All of the studies included safety training. In addition, all but one study used some form of feedback; other intervention components included goal setting, tokens and poster campaigns. The duration of the interventions ranged from a mean of 9 weeks to 12 years.

Participants included in the review

Inclusion criteria were not specified in terms of the participants. The participants in the included studies were involved in a number of diverse industries. The most common was manufacturing; other industries were shipyard or marine engineering, shoe distribution centres, city vehicle maintenance divisions, construction, mining and a bakery. All of the studies focused on blue-collar workers; two studies also included white-collar workers.

Outcomes assessed in the review

Studies that reported sufficient data to enable the calculation of an effect size for accidents or injuries were eligible for inclusion. In the review, the terms 'accidents' and 'injuries' were treated as the same. The included studies reported measures of accidents or injuries in different ways; these included the absolute number of events, number of events leading to varying amounts of lost time, average number of OSHA (not defined in the review) recordable injuries and the accident rate (per month, per specified hours of work or per week).

How were decisions on the relevance of primary studies made?

The authors did not state how the papers were selected for the review, or how many reviewers performed the selection.

Two reviewers independently assessed validity using the Epidemiological Appraisal Instrument (see Other Publications of Related Interest). This contained 43 items in the following five fields: study description, selection of participants, observation quality, data analysis and generalisability of the results. Scores of between 0 and 2 were assigned to each item. Any disagreements between the reviewers were resolved through consensus.

The authors did not state how the data were extracted for the review, or how many reviewers performed the data extraction.

For each study, the numbers of accidents or injuries were extracted, calculated from available data, or estimated from graphs. The reviewers reported details of the methods used. One standardised accident/injury rate (SAR) with 95% confidence interval (CI) was calculated for each study (number of observed events divided by the number of expected events using pre-intervention data).

How were the studies combined?

An overall SAR (metaSAR) with 95% CI was calculated (total number of observed accidents or injuries divided by the total number of expected accidents or injuries). The random-effects DerSimonian and Laird model was used in the presence of significant heterogeneity. For the one study with a control group, only data from the intervention group were included in the meta-analysis.

How were differences between studies investigated?

Statistical heterogeneity was assessed using the Q statistic. Differences between the studies were discussed in the text.

Thirteen studies were included (number of participants not calculable). There was one study with a control group (maximum n=140) and twelve before-and-after studies. The sample size, where reported, ranged from 10 to 1,000.

The studies were of poor to 'marginal' quality (validity scores ranged from 0.19 to 0.46). Methodological problems included lack of the following: description of losses to follow-up; accounting for confounders; reporting of measures of variability of results data; reporting of reliability and validity of outcome measures; and adjustment for duration of follow-up.

All included studies showed a decrease in accidents or injuries post BBS intervention, with the SARs for each study ranging from 0.13 to 0.98; the decrease was statistically significant in eight of these studies.

The studies were statistically heterogeneous (Q=278.4, d.f.=12). The overall metaSAR showed a statistically significant reduction in accidents or injuries post-intervention; the metaSAR using a random-effects model was 0.61 (95% CI: 0.72, 0.97).

BBS interventions were associated with a statistically significant reduction in workplace accidents or injuries, but the poor quality of the studies means that these results should be interpreted with caution. Further good-quality studies are required.

The review addressed a clear question that was defined in terms of the intervention and outcomes; inclusion criteria were not defined for the participants or study design. Several relevant sources were searched but no attempts were made to minimise either publication or language bias; this could have resulted in the omission of other relevant studies. Methods were used to minimise reviewer error and bias in the validity assessment, but it was not clear whether similar steps were taken in the study selection and data extraction processes. Validity was assessed and the results reported, and validity was taken into account when summarising the evidence.

The finding of significant heterogeneity suggests that meta-analysis might not have been an appropriate method of summarising these studies. There were limitations to this review (such as incomplete reporting of review methods) but, overall, the authors' conclusion appears to reflect the limited evidence obtained from poor-quality studies.

Practice: The authors did not state any implications for practice.

Research: The authors stated that the quality of future studies of behaviour-based interventions could be improved by prior assessment of the needs of the workplace and designing an appropriate intervention; use of a comparable control group; adequate reporting of losses to follow-up, sample size calculation, participation rates, eligibility criteria and characteristics of the participants; use of randomisation; accounting for confounders in the analysis; statistically testing intervention effects; and an evaluation of the validity and reliability of the outcome measures.

Genaidy A, LeMasters GK. Epidemiological appraisal instrument (EAI). Theor Issues Ergon Sci 2006;7:187-9.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.