|
Computerized clinical decision support systems for acute care management: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes |
Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB, CCDSS Systematic Review Team |
|
|
CRD summary The review concluded that most computerised clinical decision support systems improved process of care, but patient outcomes were rarely assessed. Systems have not matured sufficiently to enable application in acute care settings. Wide variability in studies and limitations surrounding the determination of study significance and the use of vote counting methods mean the conclusions cannot be considered reliable. Authors' objectives To assess the effects of computerised clinical decision support systems on process of care and patients outcomes for acute medical care. Searching MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, DARE, ACP Journal Club, Inspec and other unnamed evidence-based medicine review databases were searched between 2004 and January 2010 without language restrictions. Search terms were reported in Haynes 2010 (see Other Publications of Related Interest). Studies in the reference list of the authors’ previous systematic review were reassessed. Study selection Randomised controlled trials (RCTs) that compared the effects of patients care with clinical decision support systems versus no clinical decision support systems in acute medical care (defined as episodic health conditions with potential for cure or stabilisation in less than six months) were eligible for inclusion. Eligible studies were required to assess effects in healthcare professionals, provide patient-specific information in terms of management options or probabilities and measure clinical performance (a measure of process of care) or patient outcomes. Studies were excluded if they reported feedback on groups of patients without individual assessment, or only provided computer aided instruction. Most included studies were conducted in the USA and included over 121 different clinics at over 106 sites. Computerised clinical decision support systems were mainly standalone, but some were integrated with other systems and were categorised as treatment management assistants (alerts and reminders), management systems (guidelines and algorithms), diagnostic assistants and medication dosing assistants. The most common delivery method was computer; a range of other methods were also employed. Over 75% of system users were physicians, the rest represented a range of health-related professionals. Patient outcomes varied considerably. Treatment comparisons included usual care, standard decision-making, questionnaires and various diagnostic assessment tools. Two reviewers independently screened studies for inclusion, with disagreements resolved through consensus or referral to a third reviewer. Assessment of study quality Study quality was assessed using a modified version of the Jadad scale, which measured five sources of bias and awarded a maximum score of 10. The authors did not state how many reviewers performed the quality assessment. Data extraction Two reviewers extracted statistical data on the primary, main or most relevant, or available outcome. A series of decision rules were applied to determine which outcomes were evaluated and how that translated into a judgment of whether the study was considered to have been statistically significantly positive (+) or negative (-), or no effect (0). Where statistical data were not reported, effect was denoted as not evaluated. Primary authors were contacted for clarification on data, where necessary. Two reviewers independently extracted data; discrepancies were resolved through consensus or referral to a third reviewer. Methods of synthesis Meta-analysis was not reported due to significant heterogeneity; data were presented as a narrative synthesis. Data were reported separately for type of treatment categorisation. Sensitivity analysis was performed to assess the impact of studies with mismatch between the unit of allocation (for example clinician) and the unit of analysis (such as individual patients without adjustment for clustering). Results of the review Thirty-six RCTs (37 publications) were included in the review; 3,417 healthcare professionals and 202,491 patients. The mean quality score was 6.4 (range 2 to 10); 21 RCTs reported adequate allocation concealment, 10 RCTs went some way to employ cluster randomisation. Twenty-two of 35 RCTs reported improvements in process of care outcomes; three of 20 RCTs reported improvements in patient outcomes. Management assistants – alerts and reminders (11 RCTs): Nine of the 11 RCTs that assessed process of care outcomes reported improvements; none of the four RCTs that assessed patient outcomes reported improvements. Management assistants – guidelines and algorithms (nine RCTs): Three of eight RCTs that assessed process of care outcomes showed improvements while none of the four RCTs that assessed patient outcomes reported improvements. Diagnostic assistants (three RCTs): Two of the three RCTs showed improvements in process of care, neither of the two RCTs that assessed patient outcomes showed an improvement. Medication dosing assistants (14 RCTs): Nine of the 14 RCTs that assessed process of care outcomes demonstrated improvements. Three of 10 RCTs that assessed patient outcomes reported positive effects while one reported negative effects. Sensitivity analysis did not identify any significant impact from studies with mismatch on process of care outcomes or patient outcomes. Cost information Four RCTs reported costs relating to costs of installing computer systems and associated equipment and cost saving benefits (as reported in the review). Authors' conclusions Most computerised clinical decision support systems showed improvements in process of care, but patient outcomes were rarely assessed and were much less likely to show benefit. Computerised clinical decision support systems for acute medical care have not matured sufficiently to enable clinical decision makers to embrace the technology for clinical application. CRD commentary The review question and supporting inclusion criteria were broadly stated. Several relevant sources were searched for relevant data without language restrictions. Although publication bias was not formally assessed, the authors suggested that bias was likely to exist. Study quality was assessed using appropriate criteria and results were fully reported. It was unclear whether quality assessment was performed in duplicate, which meant that reviewer error and bias could not be ruled out. Methods employed in data extraction meant that in some instances studies could be evaluated as statistically significantly positive even when no pre-specified outcomes were reported. The synthesis used a vote-counting method to evaluate the evidence base which meant that study size and quality were not taken into account in considering the impact of a study on the overall result. The authors acknowledged multiple limitations of the synthesis presented and the included studies which contributed to it. The authors' conclusions may have reflected the evidence but the wide variability in studies and limitations that surrounded the determination of study significance as well as the use of vote counting methods meant that the conclusions could not be regarded as reliable. Implications of the review for practice and research Practice: The authors stated that computerised clinical decision support systems for acute medical care have not matured sufficiently to enable clinical decision makers to embrace the technology for clinical application. Research: The authors stated that high quality studies were needed to rigorously evaluate the effect of computerised clinical decision support systems on patient outcomes. Funding Canadian Institutes of Health Research Synthesis. Bibliographic details Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB, CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implementation Science 2011; 6(91) Other publications of related interest Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Hanes RB. Effects of computerised clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005; 293(10):1223-1238. Haynes RB, Wilcznski N, the Computerised Clinical Decision Support System (CCDSS) Systematic Review Team. Effects of computerised clinical decision support systems on practitioner performance and patient outcomes: Methods of a decision-maker-researcher partnership systematic review. Implementation Science 2010; 5:12. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, Garg AX, Haynes RB. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013; 346: f657. Indexing Status Subject indexing assigned by NLM MeSH Acute Disease; Algorithms; Biomedical Research; Cooperative Behavior; Decision Making; Decision Support Systems, Clinical; Humans; Monitoring, Physiologic /methods; Patient Care; Treatment Outcome AccessionNumber 12011006655 Date bibliographic record published 21/12/2011 Date abstract record published 11/06/2012 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|