A total of 68 studies were included (28 studies previously reviewed and 40 new studies).
Inter-rater reliability for coding of identified studies as potentially relevant or not Kappa = 58% (95%CI: 42%, 75%). Agreement on eligibility of studies Kappa = 86% (95%CI: 76%, 97%). Chance-corrected agreement between reviewers on validity Kappa = 82% (95%CI: 67%, 97%).
The majority of trials were randomised with 9 studies (13%) using quasi-random allocation or selected concurrent controls. The 28 studies in the original review had validity scores ranging from 2 to 9 (mean 6.4) with 7 (25%) scoring between 8 and 10. Validity scores for the new studies ranged from 4 to 10 (mean 7.7) with 21 (53%) scoring 8 to 10. Linear regression of validity vs year of publication estimated Beta = 0.14 (95%CI: 0.07, 0.20).
65 studies evaluating CDSSs on clinicians behaviour with 43 (66%) of these finding at least some benefit. These included 9 of 15 studies evaluating drug dosing; 1 of 5 studies evaluating diagnostic aids; 14 of 19 studies evaluating preventative care: and 19 of 26 studies evaluating other medical care.
14 studies evaluated the effect of CDSSs on patient outcomes with 6 (43%) documenting a benefit. 5 (62%) of trials finding no benefit had a power of less than 80% to detect a moderate or clinically important benefit.
Drug dosing: Benefit was found with the use of CDSSs in achieving or maintaining therapeutic theophylline (4 studies) or lidocaine hydrochloride levels (1 study) or improving anticoagulation control with heparin (1 study). 7 studies evaluating different outcomes in studies of warfarin control reported inconsistent results.
Diagnostic aids: the one study finding a benefit identified patients at high risk of developing respiratory complications postoperatively.
Prevention: all studies evaluated process of care with 14 (74%) of studies finding benefit for at least one care process. 1 study reported patient outcomes (blood pressure control) and noted no change.
Other medical care: inconsistent results were reported for hypertension care (2 studies: one reporting benefit); compliance with recommendations for diabetic care (4 studies: 2 reported improvement); compliance with recommendations for general medical problems (5 studies: all reported benefit). 7 studies reported patient outcomes with no improvement being reported for blood pressure (3 studies) and beneficial effects noted for changes in weight and quality of life (2 studies).