|
The effectiveness of occupational health and safety management system interventions: a systematic review |
Robson L S, Clarke J A, Cullen K, Bielecky A, Severin C, Bigelow PL, Irvin E, Culyer A, Mahood Q |
|
|
CRD summary The authors concluded that there was insufficient evidence to recommend for or against specific occupational health and safety management system interventions. This was generally a well-conducted review and the authors' conclusions reflect the limitations of the included studies.
Authors' objectives To evaluate the effects of mandatory and voluntary occupational health and safety management system (OHSMS) interventions on employee health and safety.
Searching MEDLINE, EMBASE, PsycINFO, Sociological Abstracts, CCINFOweb (including NIOSHTIC-2, HSELINE and OSHLINE), Safety Science and Risk Abstracts, EconLit and ABI/INFORM were searched from inception to July 2004. Some details of the search strategy were reported; full details were stated as being available from the reviewers. In addition, reference lists of all eligible studies and of reviews published in books or journals, bibliographies requested from experts in the field, and the reviewers' personal files were screened. Only peer-reviewed journal articles were eligible for inclusion. It was unclear whether any language restrictions were applied.
Study selection Study designs of evaluations included in the reviewThe included studies were randomised controlled trials (RCTs), time series, before-and-after studies, non-randomised studies, cross-sectional studies and after-only studies. Only data from studies rated as moderate or high during the quality assessment were extracted and used in the review.
Specific interventions included in the reviewStudies that evaluated extra-workplace or workplace OHSMS interventions were eligible for inclusion. Studies had to compare OHSMS with no intervention or compare different intensities of OHSMS. In the review, OHSMS interventions could be defined directly using terms synonymous with OHSMS or by the mention of specific types of OHSMS, or indirectly by mentioning OHSMS legislation, other OHSMS initiatives, or by including one management element and at least one other system element from the system elements specified by Redinger and Levine (see Other Publications of Related Interest) in the description of the OHSMS. Multifaceted management system interventions that included an occupational health and/or safety element were included. Management system interventions that focused on disability or health services and OHSMS interventions that focused on isolated aspects or particular risks were excluded.
The included mandatory OHSMS interventions were interventions dealing with the Norwegian Internal Control legislation introduced in 1992 and the Canadian provincial legislatures in the late 1970s. All of the included voluntary OHSMS interventions were set in English-speaking countries and some were set in just one workplace. The included studies were set in an a variety of organisations including public administration, manufacturing, transportation, health care, retail and motor vehicle repair services.
Participants included in the reviewStudies in workplaces located anywhere in the world were eligible for inclusion.
Outcomes assessed in the reviewStudies that assessed OHSMS implementation or effectiveness were eligible for inclusion. Studies assessing effectiveness had to report a quantitative measure of at least one of the following: intermediate OHS outcomes (including proxy outcomes such as employee knowledge, beliefs, values or perceptions, behaviours or OHS hazards) or final OHS outcomes (such as employee injury or illness). Studies assessing implementation had to report a quantitative change in the level or intensity of the OHSMS.
How were decisions on the relevance of primary studies made?Two reviewers independently screened titles and abstracts. Pairs of reviewers selected studies and resolved any disagreements on inclusions through discussion, with recourse to a third author if required.
Assessment of study quality Internal validity for the assessment of effectiveness in each study was assessed by considering potential sources of bias in relation to sample selection, confounders, measurement methods, any other sources of bias and the appropriateness of statistical tests. The quality of each study was then rated as very low, low, moderate or high. Two reviewers independently assessed validity and resolved any disagreements through discussion, with recourse to a third author if required.
Data extraction Pairs of reviewers extracted data on the results and statistical analysis from studies with a quality rating of moderate or high. Any disagreements were resolved by consensus. For each study, intervention effects were reported in tables as being either positive (statistical effect was not necessarily determined) or not statistically significant (p>0.05).
Methods of synthesis How were the studies combined?The studies were grouped by outcome measure and combined in a narrative. The results were discussed separately for voluntary and mandatory OHSMS.
How were differences between studies investigated?Differences between the studies were discussed.
Results of the review Thirteen studies of moderate or high quality were included in the review: 1 RCT, 6 time series (including one with a comparison group), 2 before-and-after studies, 1 non-randomised study, 2 cross-sectional studies and 1 after-only study. The number of workplaces analysed in each study ranged from 1 to 2,029.
Methodological limitations of the studies related to limitations in study design; lack of accounting for potential confounders; inadequate description of the methods used to measure outcomes; the potential for bias in the measurement of outcomes; and generalisability of findings limited by high rates of refusal to participate and evaluations of single sites. One study, the RCT, was judged to be of a high quality. Voluntary OHSMS (5 studies; 2 additional studies that only appeared to evaluate economic outcomes are reported under 'Costs' below).
All 5 studies showed positive effects. One study also had some null findings, but there were no negative findings. Positive effects included increased OHSMS implementation over time, intermediate effects (including better safety climate, increased hazard reporting, more organisational action on OH issues) and decreases in injury rate. The only high-quality study assessed implementation and did not report data on outcomes or economic costs.
Mandatory OHSMS (6 studies).
The studies showed positive effects although there were also null findings on some of the final OHS outcomes. Mandatory interventions resulted in increased OHSMS implementation over time, improved intermediate effects (including increased health environment and safety (HES) awareness, improved perceptions of the physical and psychosocial environments, increased workers' participation in HES activities), decreased lost-time injury rates and increased workplace productivity.
Cost information Studies that assessed economic outcomes were also eligible for inclusion in the review.
One study of voluntary OHSMS reported a 13% decrease in worker's compensation cost per employee, while 2 other studies reported decreases of 25% and 52% in premium rates. The reviewers stated that other concomitant changes may have contributed to the positive results.
Authors' conclusions There was insufficient evidence to recommend for or against specific OHSMS.
CRD commentary The review addressed a clear question that was defined in terms of the intervention, outcomes and study design. Several relevant sources were searched but no attempts were made to minimise publication bias; the reviewers acknowledged and discussed this limitation. Two reviewers independently selected the studies, assessed validity and extracted the data, thus reducing the potential for reviewer bias and error. Validity was assessed using specified criteria and the results of the assessment were reported. However, it might have been more appropriate to have assessed quality using established criteria for each type of study design. In view of the differences between the studies, a narrative synthesis which highlighted the methodological limitations of the included studies was appropriate. This was generally a well-conducted review and the authors' conclusions reflect the limitations of the included studies.
Implications of the review for practice and research The authors did not state any implications for practice or further research.
Funding Ontario Workplace Safety and Insurance Board.
Bibliographic details Robson L S, Clarke J A, Cullen K, Bielecky A, Severin C, Bigelow PL, Irvin E, Culyer A, Mahood Q. The effectiveness of occupational health and safety management system interventions: a systematic review. Safety Science 2007; 45(3): 329-353 Other publications of related interest Redinger CF, Levine SP. Development and evaluation of the Michigan occupational health and safety management system assessment instrument: a universal OHSMS performance measurement tool. Am Ind Hyg Assoc J 1998;59:572-81.
Indexing Status Subject indexing assigned by CRD MeSH Occupational Diseases /prevention & Occupational Health; Occupational Health Services /methods /organization & Safety Management /organization & Workplace; administration; administration; control AccessionNumber 12007000258 Date bibliographic record published 31/03/2008 Date abstract record published 31/03/2008 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|