|
Effect of clinical decision-support systems: a systematic review |
Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D |
|
|
CRD summary The authors concluded that commercially and locally developed clinical decision-support systems improved health care process measures across diverse settings. Evidence for clinical, economic, workload, and efficiency outcomes remained sparse. These conclusions reflect the paucity of data for some outcomes. Uncertainties around suitability of comparators and meta-analyses mean that the conclusion on health care processes cannot be considered reliable. Authors' objectives To evaluate the effect of clinical decision-support systems on clinical outcomes, health care processes, workload and efficiency, patient satisfaction, cost, and provider use and implementation. Searching Four electronic databases (including PubMed and CINAHL) were searched up to January, 2011 (five databases in the Agency for Healthcare Research and Quality report; see Other Publications of Related Interest). Search terms were reported separately online. Bibliographies of included articles and relevant review articles were handsearched for further studies. Study selection Eligible studies were randomised trials of electronic clinical decision-support systems, implemented in real clinical settings, and used by health professionals to aid decision-making at the point of care or for a specific care situation. At least one of the following types of outcomes had to be reported: clinical; health care process; user workload and efficiency; relationship-centred; economic; or use and implementation by a health professional. Trials not fully published in English were excluded, as were trials with fewer than fifty participants. Most included trials were conducted in academic and/or community settings, mainly in the USA or Europe. Most trials were performed in environments without established health information technology; most targeted specific conditions. Settings included in-patient, out-patient, long-term facilities, and emergency departments. System objectives included pharmacotherapy, diagnosis, chronic disease management, laboratory test ordering, initiating discussion with patients, and other objectives. Systems formats were often integrated, stand alone, fax or computer printouts, or online. Most systems had been developed locally. Comparators included usual care, no clinical decision-support system, the same clinical decision-support system targeted at a different condition or with additional features. Clinical outcomes included length of stay, morbidity (hospitalisations, surgical site infections, cardiovascular events), mortality, health-related quality of life, and adverse events. Healthcare process outcomes included rates for recommended preventive care services or clinical studies ordered or completed, or rates for treatment ordered or prescribed. Two or three reviewers independently screened studies for inclusion; discrepancies were resolved by consensus with another reviewer. Assessment of study quality The quality of individual trials was assessed as good, fair, or poor using criteria specified by the Agency for Healthcare Research and Quality (AHRQ). The overall strength of evidence for each outcome was judged as being high, moderate, low, or insufficient. Evidence applicability was evaluated by identification of issues on trial setting, interventions, and outcomes. Two reviewers performed the quality assessment. Discrepancies were resolved by consensus or referral to a third reviewer. Data extraction Data considered appropriate for meta-analysis were extracted to calculate relative risks (RRs) and 95% confidence intervals (CIs) for morbidity and adverse events, or odds ratios (ORs) and 95% confidence intervals for mortality and health care process measures. Data not appropriate for meta-analysis were extracted and reported in the Agency for Healthcare Research and Quality report linked to the review (see Other Publications of Related Interest). Data were extracted by one reviewer and confirmed by another. Discrepancies were resolved by consensus or referral to a third reviewer. Methods of synthesis Outcome-specific meta-analyses of data from trials that measured the outcome in the same manner were conducted to estimate summary estimates (relative risks or odds ratios depending on the outcome), with 95% confidence intervals, using the DerSimonian and Laird random-effects model. Meta-analysis was only performed where four or more trials reported on the outcome. All other data were presented within the narrative synthesis, according to the outcome represented. Publication bias was assessed using funnel plots. Results of the review One hundred and forty-eight randomised controlled trials (RCTs) were included in the review; the exact number of participants was unclear. The quality of most trials was rated as good or fair. Strength of evidence was rated as being low for all but one of the clinical outcomes, and high or moderate for the health care process measures. The strength of evidence for all other outcomes was rated as insufficient or low, with the exception of moderate ratings for the cost and health care provider satisfaction outcomes. Trial duration ranged from less than six months to over three years (where reported). Clinical outcomes: Locally and commercially developed clinical decision-support systems improved morbidity (RR 0.88, 95% CI 0.80 to 0.96; 16 of 22 trials). Clinical decision-support systems did not demonstrate a statistically significant effect on the clinical outcomes of mortality (OR 0.79, 95% CI 0.54 to 1.15, six of seven trials) or adverse events (RR 1.01, 95% CI 0.90 to 1.14; five trials). The generalisability of the results was limited (fully reported in the review). Health care process measures: Clinical decision-support systems significantly improved health care process measures for performance of preventive services (OR 1.42, 95% CI 1.27 to 1.58; 25 of 43 trials), ordering or completion of clinical studies (OR 1.72, 95% CI 1.47 to 2.00; 20 of 29 trials), and prescription of appropriate therapies (OR 1.57, 95% CI 1.35 to 1.82; 46 of 67 trials). Use and implementation: Twelve trials investigated provider satisfaction with decision-support systems. Four trials showed that satisfaction was statistically significantly higher among intervention providers than controls. Six trials reported provider dissatisfaction with decision-support systems. Evidence demonstrated low use of decision-support systems among clinicians (17 trials). Further outcomes with insufficient evidence were reported. All meta-analyses were conducted in heterogeneous trials. Cost information Both locally and commercially developed clinical decision-support systems had modestly lower treatment costs, total costs, and reduced costs, compared with control groups and interventions without clinical decision-support systems (22 trials). Six trials assessing the cost-effectiveness of decision-support systems reported conflicting findings. Three trials were cost-effective; three were not. Authors' conclusions Commercially and locally developed clinical decision-support systems were effective in improving health care process measures across diverse settings. Evidence for clinical, economic, workload, and efficiency outcomes remained sparse. CRD commentary The review question was clear and inclusion criteria appeared sufficiently replicable. Relevant electronic databases were searched. The English language search restriction meant that some relevant trials may have been missed. Efforts were made to minimise error and bias throughout the review process. Trial quality varied. Appropriateness of data synthesis methods employed was unclear due to lack of clarity on the weighting of individual trials and the unexplained large number of trial arms within trials for some outcomes. It was unclear whether this may have introduced double counting of patients. It was also noted that only a small proportion of the 148 trials reported outcomes. Some trials were underpowered or had short follow-up periods. The authors highlighted that contamination within control groups was possible. It was unclear how suitable the comparators were. The authors' conclusion reflects the paucity of data for some outcomes, but given the limitations of the evidence, the conclusion on health care process measures cannot be considered reliable. Implications of the review for practice and research Practice: The authors stated that the role of clinical decision-support systems can and should play in reshaping health care delivery; this needs to be communicated objectively. Research: The authors stated that larger studies with longer evaluation duration were required to understand: how clinical decision-support systems might be expanded to accommodate multiple comorbid conditions simultaneously; how to determine which care team members should receive decision support; what their effect was on clinical and economic outcomes; how they could be integrated into workflow across diverse settings; ways in which they might be incorporated into workflow tools. Funding Agency for Healthcare Research and Quality, USA. Bibliographic details Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of clinical decision-support systems: a systematic review. Annals of Internal Medicine 2012; 157: 29-43 Other publications of related interest Lobach D, Sanders GD, Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux R, Samsa G, Hasselblad V, Williams JW, Wing L, Musty M, Kendrick AS. Enabling health care decisionmaking through clinical decision support and knowledge management. Rockville, MD, USA: Agency for Healthcare Research and Quality. Evidence Report/Technology Assessment Number 203. 2012. Indexing Status Subject indexing assigned by NLM MeSH Cost-Benefit Analysis; Decision Support Systems, Clinical /economics /standards; Humans; Randomized Controlled Trials as Topic; Treatment Outcome AccessionNumber 12012030528 Date bibliographic record published 10/07/2012 Date abstract record published 17/07/2012 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|