|
Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review |
Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB; CCDSS Systematic Review Team |
|
|
CRD summary The review assessed the effects of computerised clinical decision support systems for drug therapy management on process of care and patient outcomes. The authors concluded that results showed inconsistent improvement. Due the basic synthesis adopted, the reliability of the author's conclusions is uncertain. Authors' objectives To assess the effects of computerised clinical decision support systems for drug therapy management on process of care and patient outcomes, and to identify system and study characteristics that predicted benefit. Searching MEDLINE, EMBASE and EBM Reviews were searched between 2004 and January 2010 without language restrictions. Search terms were available from Haynes, 2010 (see Other publications of related interest). Reference lists of included studies and relevant reviews were searched, as were two evidence update websites and the Inspec Bibliographic Database. This was an update of a previous review by the authors published in 2005 (see Other publications of related interest). Study selection Eligible studies were randomised controlled trials (RCTs) that reported outcomes for a group of providers or patients using a computerised clinical decision support system compared with care without such a system. Independent providers or postgraduate trainee providers were primary users. The system had to provide patient-specific output in the form of assessments, management options, or recommendations to the clinical user. Studies were excluded if the system was used solely by students, only provided summaries of information for patients, only provided feedback on groups of patients without feedback about individual patients, only provided computer-aided instruction, or were used for image analysis. Around half the included studies were of drug therapy-only systems, with the remainder studying multi-faceted systems (which gave drug advice as part of a more complex intervention). Most systems were designed to optimise drug therapy; others aimed to prevent adverse drug events or focused on cost management. Around two-thirds of the systems were integrated with an electronic medical record. Systems were mostly used by fully trained physicians. Comparators were most commonly described as being usual care. Just over half the studies were published after 2004 and around two-thirds were conducted in the USA. Most studies were in an out-patient setting and most populations were adults or were unspecified. Two reviewers independently screened studies for inclusion. Disagreements were resolved through referral to a third reviewer. Assessment of study quality Study quality was evaluated using a 10 point scale that assessed the appropriateness or adequacy of the following: concealment of allocation; unit of allocation; adjustment for baseline differences; follow-up period; and outcome assessment. The authors did not state how many reviewers performed the quality assessment. Data extraction Two reviewers independently extracted data on process of care and patient outcomes; discrepancies were resolved through consensus or referral to a third reviewer. For multi-armed trials data from the most sophisticated support system was extracted. Authors were contacted to clarify data where necessary. Methods of synthesis Meta-analysis was not reported due to clinical heterogeneity, so results were presented as a vote-counting narrative synthesis based on whether at least 50% of relevant outcomes showed a statistically significant difference. Tests of association between study and support system factors and improved outcomes were tested using the univariate Fisher's exact test. Multivariate analyses were conducted using multinomial logistic regression. Sensitivity analyses were conducted to determine if the class of outcome selected to judge improvement affected results, and whether successful trial results were affected by the type of analysis used in cluster randomised trials (matched and mismatched units of allocation and units of analysis). Results of the review Sixty-five RCTs were included. The total number of practitioners was 8,932, and the total number of patients was 1,246,686 (range 30 to 400,000). The median study quality score was 8 out of 10. Forty-four studies were cluster randomised, 41 reported adequate allocation concealment and all but one measured an objective outcome or blinded the outcome assessor. Thirty-seven of 59 trials showed improvement in process of care outcomes due to computerised clinical decision support systems use; no significant difference in studies showing improvement was found between drug therapy-only (improvement in 23 of 33 studies) and multi-faceted (improvement in 14 of 26 studies) support systems. Six out of 28 studies that measured a patient outcome showed improvement with a support system when compared to usual care without a support system. All studies showing improved patient outcomes also showed improvement in process of care outcomes. The proportion of successful trials was not significantly different between cluster trials where units of allocation were mismatched with units of analyses, compared with non-cluster trials, or cluster trials with an appropriately adjusted analysis. Support systems not integrated with an electronic medical record were more likely to improve process of care outcomes (p=0.03), and patient outcomes (p=0.017), than systems with electronic medical record linkage. The association between electronic medical record integration and support systems failure was not statistically significant via multivariate regression. Improvement in process of care or patient outcomes was not affected by integration with computerised provider order entry, timing or method of decision support delivery, or method of data entry and did not vary by country, provider type and out-patient versus other settings of care. Cost information Six trials observed significantly decreased costs, five trials found no significant difference in costs and one trial found significantly increased costs associated with computerised clinical decision support systems. Only one trial performed a formal cost-effectiveness analysis. Authors' conclusions Computerised clinical decision support systems inconsistently improved process of care measures and seldom improved patient outcomes. Lack of clear patient benefit and lack of data on harms and costs preclude a recommendation to adopt computerised clinical decision support systems for drug therapy management. CRD commentary The review question was clear and the supporting eligibility criteria were reproducible. Several relevant sources were searched for relevant studies. The full search strategy was reported elsewhere (see Other Publications of Related Interest). There were no language restrictions. Publication bias could not be formally assessed. It was unclear whether quality assessment was performed in duplicate (duplicate processes were used for study selection and data extraction), so reviewer error and bias could not be ruled out. The narrative synthesis used a vote-counting method whereby study size was not taken into account (when considering the impact of individual studies on the overall result). The authors acknowledged the limitations of this type of synthesis. Although predictors of support system success were explored, including the effect of unit allocation, the effect of study quality scores or allocation concealment was not included. Comprehensive study details were provided as appendices. Due to the basic synthesis adopted, the reliability of the author's conclusions is uncertain. Implications of the review for practice and research Practice: The authors could not recommend the adoption of computerised clinical decision support systems for drug therapy management. Research: The authors stated that future research should explicitly address patient outcomes, including potential harms, and costs and adverse clinician impacts of support systems. Future studies may wish to incorporate non-computerised approaches as active comparators. Funding Canadian Institutes of Health Research Synthesis Grant. Bibliographic details Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB; CCDSS Systematic Review Team. Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review. Implementation Science 2011; 6:89 Other publications of related interest Haynes RB, Wilczynski NL, Computerized Clinical Decision Support System (CCDSS) Systematic Review Team: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: methods of a decision-maker-researcher partnership systematic review. Implementation Science 2010; 5:12. Roshanov PS, Misra S, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB; CCDSS Systematic Review Team. Computerized clinical decision support systems for chronic disease management: a decision-maker-researcher partnership systematic review. Implementation Science 2011; 6:92. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Hanes RB. Effects of computerised clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005; 293(10):1223-1238. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, Garg AX, Haynes RB. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013; 346: f657. Indexing Status Subject indexing assigned by NLM MeSH Algorithms; Biomedical Research; Consumer Behavior; Cooperative Behavior; Decision Support Systems, Clinical /instrumentation; Disease Management; Drug-Related Side Effects and Adverse Reactions; Evidence-Based Medicine /instrumentation /methods; Global Health; Humans; Monitoring, Physiologic; Practice Patterns, Physicians' /statistics & Prescription Drugs; United States; User-Computer Interface; numerical data AccessionNumber 12011006670 Date bibliographic record published 21/12/2011 Date abstract record published 26/10/2012 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|