Seventy studies (from 88 papers) were included in the review. There were 82 relevant comparisons in which 71 compared a decision support system with a control group and 11 provided a direct comparison of systems. Amongst the studies, there were approximately 6,000 clinicians caring for 130,000 patients.
Decision support systems improved clinical practice in 68% of the studies. In the univariate analysis, 6 system features were relevant to intervention success. Differences in the success rates were reported for studies of interventions with and without the specific feature. For those reaching statistical significance (P=0.05), 75% of interventions were successful when decision support was provided as part of routine clinical practice (RD 75%, 95% CI: 37, 84) and 73% were successful when integrated with charting or order entry systems (RD 37%, 95% CI: 6, 61). Seventy-six per cent of the interventions were successful when a recommendation (rather than assessment alone) was provided (RD 35%, 95% CI: 8, 58). All interventions were successful when reasons for non-compliance with recommendations were requested from the clinician (RD 41%, 95% CI: 19, 54). Seventy-six per cent of the interventions were also successful when computer-based systems were included (RD 26%, 95% CI: 2, 49). A further feature (although marginally short of the significance level) was the provision of support at the time and location of decision-making (RD 48%, 95% CI: 0, 70).
In the multivariate regression analysis, four of the above features were confirmed as independent predictors of successful interventions. Of the 32 systems including all 4 features, 30 (94%) were noted to improve clinical practice. These were the provision of support as part of the clinical workflow (adjusted OR 112.1, 95% CI: 12.9, infinity, P<0.00001); the provision of decision support at the time and location of decision making (adjusted OR 15.4, 95% CI: 1.3, 300.6, P=0.0263); the provision of a recommendation rather than assessment alone (adjusted OR 7.1, 95% CI: 1.3, 45.6, P=0.0187); and the use of computer-based systems (adjusted OR 6.3, 95% CI: 1.2, 45.0, P=0.0294). The subgroup analysis of computer-based versus non-electronic systems was consistent with these findings.
The survey of direct experimental evidence confirmed the importance of providing decision support at the time and location of decision-making and the requesting of documented reasons for non-compliance. In addition, success was enhanced when clinicians were provided with periodic feedback about their compliance with recommendations, and when decision support results were conveyed to both clinicians and patients.
The analysis of inter-rater agreement showed favourable results for the study selection (range: 66 to 96%) and data extraction (91 and 94%).