PROSPERO International prospective register of systematic reviews
International law’s effects on health and its social determinants: a systematic review, meta-analysis and meta-regression
Steven Hoffman, Harkanwal Randhawa, Matthew Hughsam, John-Arne Rottingen, Sally Davies, Colleen Flood, Julio Frenk, Gordon Guyatt, Marie-Paule Kieny, John Lavis
Steven Hoffman, Harkanwal Randhawa, Matthew Hughsam, John-Arne Rottingen, Sally Davies, Colleen Flood, Julio Frenk, Gordon Guyatt, Marie-Paule Kieny, John Lavis. International law’s effects on health and its social determinants: a systematic review, meta-analysis and meta-regression.
Available from http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015019830
1) What is the effect of international law on health outcomes?
2) What is the effect of international law on the social determinants of health?
3) Upon what factors are effects of international law conditional?
Sources: MEDLINE, Global Health, CINAHL, PAIS INTERNATIONAL, Worldwide Politican Science Abstracts, International Politican Science Abstracts, International Bibliography of Social Sciences, Social Sciences Abstracts, Applied Social Sciences Index and Abstracts, Social Sciences Citation Index.
No language or date restrictions applied.
Types of study to be included
All quantitative impact evaluations, including experiments (e.g., randomized controlled trials), quasi-experiments (e.g., interrupted time-series analyses), and observational designs (e.g., pooled time-series cross-sectional analyses, event histories, survival analyses), will be included.
Condition or domain being studied
The value, feasibility and applicability of international law for health and related challenges.
Given that international law only imposes obligations on states (with only a small number of exceptions such as crimes against humanity for which individuals can also be convicted), participants will include all current and former countries, formally recognized as states by the international community which is best indicated by membership in the United Nations or a similar multilateral organization.
Any implementation of international law related to health or its social determinants (e.g., armed conflict, human rights, trade, finance, employment, empowerment and the environment) between at least three parties. Commonly referred to as agreements, charters, conventions, declarations, exchange of notes, memorandums of understanding, modus vivendi, protocols, treaties and at least 30 other names, international law is defined as the “rules and principles of general application dealing with the conduct of states and of international organizations and with their relations inter se, as well as with some of their relations with persons.” (Source: Restatement of the Law (Third), the Foreign Relations of the United States. Section ?101. Washington: American Law Institute; 1987.)
Control interventions are not necessary for studies to meet eligibility for inclusion in this review. Examples of possible control interventions include, but are not limited to, the absence of an international law, or a local state law.
To be included, a study has to measure the effect of international law on an objective and quantifiable outcome related to health or its social determinants, which, according to the WHO Commission on the Social Determinants of Health (2008), include armed conflict, human rights, trade, finance, employment, empowerment and the environment, among other factors.
Data extraction, (selection and coding)
Before beginning this process, calibration exercises will be conducted to ensure there is consistency between reviewers. A powerful online data abstraction program (Distiller SR) will be used to maximize efficiency – it will be customized for this task using the criteria of a standardized form. Reviewers will extract data independently from eligible studies. Data extracted will include general study information, methodology, intervention details and outcome data. Disagreements will be documented and resolved by discussion to achieve consensus.
Risk of bias (quality) assessment
Studies that are consistent with the inclusion criteria will be assessed for bias using the Cochrane or Campbell Collaboration’s checklists (for randomized experiments) and the criteria developed by Hombrados and Waddington (2012) for social experiments and quasi-experiments. Note that, although the kinds of biases – namely selection bias, confounding, group equivalence, spill-overs and reporting biases – are broadly equivalent for both types of studies, there are important differences in operationalization of the evaluation criteria. Reviewers will resolve disagreement by discussion.
In presenting the evidence, the GRADE (Grading of Recommendations Assessment, Development and Evaluation) system of rating the quality of evidence will be also used, as recommended by the Cochrane Collaboration. Our methodology should receive an AMSTAR rating of at least 8/11 (indicating high quality) if not greater.
Strategy for data synthesis
Three types of analysis will be conducted, resulting in at least three sets of results: 1) the impact of international laws in each field on any outcome; 2) the impact of international laws from any field on particular outcomes; 3) the conditions mediating the impact of international laws. The meta-analysis will facilitate the first two lines of inquiry, while the meta-regression will help answer the third.
Meta-analysis: In our preliminary survey of 90 studies obtained thus far, we have found that they are mostly time-series cross-sectional analyses that provide quantitative data on continuous variables suitable for pooling. However, they evaluate different laws across different countries, years and outcome variables. In a more detailed review, we will determine the extent to which it is plausible that across the range of the populations, the interventions, and the outcomes, it is plausible that effects are similar. For questions in which it is plausible, we will conduct meta-analyses.
In principle, our aim is to be liberal in this judgment: that is, we are prepared to pool results from a relatively broad range of populations, interventions, and outcomes. Having done that, we can examine the variability in results to determine the extent to which the data support the assumption regarding similar effects across populations, interventions, and outcomes. We anticipate substantial variability, and will address this through meta-regression (see below).
Since we know that the studies will measure outcomes of interest in different units, the effect size will be calculated using the difference in mean values divided by the pooled standard deviation of the two groups (i.e., intervention versus non-intervention). This calculation will result in a unit-less measure of the effect known as the standardized mean difference (SMD). This can then be used for comparison and pooling across studies. To facilitate the interpretation of the pooled SMD in a meaningful way, it will be expressed as a ratio of means. Confidence intervals for all effect measures will be also calculated. Random effects meta-analysis will be used as this approach is conservative in that it considers both within and between study differences in calculating the error term. We will examine the extent of heterogeneity statistically using chi-square tests of heterogeneity and of inconsistency using the I-squared measure.
Meta-regression: Meta-regression will be used to systematically explore the reasons for different effect sizes and outcomes across studies. This will allow us to test existing theories in international law and international relations about when law matters and to provide global policymakers with evidence-informed guidance on when different types of international laws may be most helpful (and when they may even be harmful). Specifically, informed by a new theoretical framework we have developed, we will examine the following a priori specified possible determinants of heterogeneity, including postulated directions of effect: we hypothesize larger and more beneficial effects in laws that deregulate rather than regulate activity; laws that target countries’ foreign rather than domestic policies; laws that target low- and middle-income countries rather than high-income countries; laws assessed in the last decade versus prior decades; and laws adopted to improve national security or the economy rather than social well-being or the environment.
Analysis of subgroups or subsets
Subgroup analyses will be conducted by the following attributes to ascertain effects of different types of international laws under different circumstances:
a) By policy domain: This allows for comparison between laws operating in different policy domains in terms of magnitude of effects.
b) By enforcement mechanism: Level or robustness of enforcement of a treaty may impact magnitude of effects on law/social determinants.
c) By impact/indicator measure: Categorization of studies by impact measure (e.g. population health, foreign trade inflow, economic status) allows for easier pooling of results and comparison of the effects of international treaties on the same indicators.
d) By scope (small club of countries vs. universal): Can provide insight into the relationship between number of countries involved in a treaty and impact on health/social determinants.
e) By study design: Robustness of analytical methods may impact the magnitude of effects determined in each study, and categorization by study design will be necessary when we attempt to pool results.
f) By time period: Can give insight into whether there was enough time to measure an impact and if the study was able to evaluate the effects after.
g) By forum: Can provide information on whether laws have differing effects depending on the process through which they are created and the organization that may have hosted its negotiation and enforcement (e.g., UN vs. WTO).
Contact details for further information
1280 Main Street West, MML-417
Hamilton, ON, Canada, L8S 4L6
Organisational affiliation of the review
Global Strategy Lab @uOttawa
Professor Steven Hoffman, University of Ottawa Mr Harkanwal Randhawa, McMaster University Mr Matthew Hughsam, University of Ottawa Mr John-Arne Rottingen, Norwegian Institute of Public Health Ms Sally Davies, Department of Health London Ms Colleen Flood, University of Ottawa Mr Julio Frenk, Harvard School of Public Health (Massachusetts) Mr Gordon Guyatt, McMaster University Ms Marie-Paule Kieny, World Health Organization (Switzerland) Mr John Lavis, McMaster University
Details of any existing review of the same topic by the same authors
A recent scoping review conducted by three of the investigators (SJH, JAR & JF) that is published in AJPH attempted to delineate these effects of international law by qualitatively summarizing 90 empirical impact evaluations of different laws. This review, which is the most comprehensive to date, found that the effects of international laws were mixed and depended on the nature of the instrument, intended outcome and mediating factors. However, this scoping review was unable to produce a comprehensive analysis, point estimates or an authoritative list of considerations as it: 1) did not use a systematic search or analysis protocol; 2) only reported on the direction of effects and did not report any measure of extent of impact; 3) did not employ standard techniques to reduce bias; and 4) did not disaggregate or analyze findings according to the type of international law or outcome measures studied.
Hoffman SJ and Røttingen JA. Assessing the Expected Impact of Global Health Treaties: Evidence From 90 Quantitative Evaluations. American Journal of Public Health 2014; e1-e14.
Anticipated or actual start date
01 September 2014
Anticipated completion date
01 October 2015
No funding sources have been acquired presently.
Conflicts of interest
Subject index terms status
Subject indexing assigned by CRD
Subject index terms
Health; Humans; Jurisprudence; Sociology
Stage of review
Date of registration in PROSPERO
08 May 2015
Date of publication of this revision
08 May 2015
Stage of review at time of this submission
Piloting of the study selection process
Formal screening of search results against eligibility criteria
Risk of bias (quality) assessment
PROSPERO This information has been provided by the named contact for this review. CRD has accepted this information in good faith and registered the review in PROSPERO. CRD bears no responsibility or liability for the content of this registration record, any associated files or external websites.