This review found that compliant and persistent postmenopausal women from observational studies and clinical practice benefited with reduced fractures on bisphosphonate treatment to a similar extent as those in randomised controlled trials. The results of the review should be interpreted with a substantial degree of caution because of the risk of a number of biases and methodological flaws.

To evaluate bisphosphonate therapy for the reduction of fracture risk in women with postmenopausal osteoporosis and determine whether the use of this intervention differs between randomised controlled trials (RCT) and clinical practice.

MEDLINE, the Cochrane Library and other unspecified Internet-resident information resources were searched from November 2007 to November 2008; search terms were reported. Reference lists of retrieved studies, prior meta-analyses and systematic reviews were scanned to identify additional studies.

Fully published placebo-controlled RCTs and observational studies (of more than 200 women) that evaluated bisphosphonate therapy in women with postmenopausal osteoporosis were eligible for inclusion. Studies had to provide data for clinically diagnosed fractures. In the observational studies, compliant patients (defined as those with a medication possession ratio of 80 to 90%) or highly persistent patients (defined as those continuously using medication without gaps) had to be compared with less compliant or persistent patients. The concomitant treatment with other antiresorptive medications (other than bisphosphonates) was permitted in observational studies.

In included studies, the median age of the women was 68 years (interquartile ranges from 65 to 72 years for RCTs and 64 to 72 years observational studies). In most RCTs, trial duration was either two or three years (range one to four years); in the observational studies, study durations ranged from one to five years. Some included women had a history of previous fractures. The single bisphosphonate that was most often investigated in the RCTs was alendronate at a range of doses; other bisphosphonates investigated included clodronate, etidronate, ibandronate, risedronate, tiludronate and zoledronic acid at various doses and frequencies.

Two reviewers performed the study selection; any disagreements were resolved by discussion

The authors did not state that they formally assessed methodological quality, but some information on study completion was provided in the review.

Two reviewers independently extracted or calculated treatment odds ratios (OR) or hazard ratios (HR) and corresponding 95% confidence intervals (CI) using the Fisher's exact method for the outcomes of fracture occurrence and occurrence of specific fracture types. Any disagreements between the reviewers were resolved by discussion.

Pooled odds ratios and 95% confidence intervals were calculated using a random-effects model. The Cochran's Q test was used to evaluate statistical heterogeneity. Meta-regression was used to compare the treatment effects observed in the RCTs with those found in the observational studies. Sensitivity analyses were undertaken on the basis of fracture type (non-vertebral fractures, vertebral fractures and hip fractures). Publication bias was investigated using regression of standardised effect compared with precision.

Twenty RCTs (n=44,878 women, sample size range 327 to 9,331 women) and 11 retrospective observational studies (n=331,256 women, sample size range 2,613 to 101,038 women) were included in the review. The median proportion of women completing the studies was 79% in the RCTs (interquartile range 66 to 88%).The authors stated that the included studies were uniformly well designed.

All fractures: There were statistically significant benefits associated with bisphosphonate treatment observed in seven RCTs that evaluated the occurrence of fractures (OR 0.762, 95% CI 0.680 to 0.855; n=23,769 women). Pooled data from the six observational studies also showed significant benefits of bisphosphonate treatment on fracture occurrence (OR 0.797, 95% CI 0.748 to 0.850; n=113,459 women). Across all 13 studies, a statistically significant 22% reduction in the occurrence of all fractures was observed (OR 0.776, 95% CI 0.729 to 0.826). Results of the sensitivity analyses showed similar significant treatment effects for different bisphosphonate types, study periods (one to two years or three to five years), effect size measures (OR or HR), and geographic regions (USA or other and multiple sites).

Single fracture types: There were significantly lower risks observed in the RCTs with bisphosphonate treatment of non-vertebral fractures (OR 0.796, 95% CI 0.739 to 0.858; 17 RCTs; n=37,460 women), vertebral fractures (OR 0.413, 95% CI 0.279 to 0.612; four RCTs; n=14,865 women), hip fractures (OR 0.711, 95% CI 0.616 to 0.820; 10 RCTs; n=34,415 women). There were no differences in the occurrence of wrist fractures for both RCTs and observational studies.

The was no evidence of statistical heterogeneity or publication bias for the results for all fracture types across the RCTs.

Compliant and persistent postmenopausal women from observational studies/clinical practice benefited with bisphosphonate treatment to a similar extent as observed in randomised controlled trials.

The review addressed a clear question and criteria for the inclusion of studies were stipulated. Two appropriate electronic databases were searched over a short period; the additional Internet-based resources searched were not specified, so it would be difficult to replicate the search for studies. Although the review was restricted to published studies, there was no evidence of publication bias .There were no language restrictions stipulated, so the risk of language bias was unclear. Steps were taken to minimise reviewer error and bias for study selection and data extraction.

The authors did not formally assess methodological quality of the included studies, so the robustness of the findings of the review was unclear. The results of observational studies were subject to a number of potential biases, so it was not appropriate to combine the results of these studies statistically with the results of RCTs. The therapeutic regimens differed between the two groups of study designs, in that single agents were used in the RCTs, and multiple bisphosphonates were used in the observational studies; it may not have been appropriate to compare across the different study designs. Little data was presented on compliance and persistence by included women. The authors acknowledged the limitations of observational studies. One author was an employee of Novartis (manufacturers of bisphosphonates and funders of the review).

The results of the review should be interpreted with a substantial degree of caution because of the risk of a number of biases and methodological flaws.

Practice: The authors stated that women would gain optimal benefit from bisphosphonate treatment if there were improvements in compliance and persistence of treatment, otherwise bisphosphonate therapy is no more effective than placebo treatment.

Research: The authors did not state any implications for research.

Supported by an unrestricted grant from Novartis Pharmaceuticals Corporation (manufacturers of bisphosphonates that were evaluated in the review).

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.