|
Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review |
Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB; CCDSS Systematic Review Team |
|
|
CRD summary This review concluded that some computerised clinical decision support systems improved practitioners' diagnostic test ordering behaviour, but the determinants of success and failure remained unclear. This conclusion seems broadly appropriate, but the review did not provide information about the overall size of effect or impact of specific systems in specific settings. Authors' objectives To assess whether computerised clinical decision support systems improved health professionals' diagnostic test ordering behaviour. Searching MEDLINE, EMBASE, Evidence-Based Medicine Reviews and Inspec electronic databases were searched up to 6 January 2010, supplemented with handsearching of reference lists of included articles and relevant systematic reviews. Search terms were reported in Haynes 2010 (see Other Publications of Related Interest). Study selection Randomised controlled trials (RCTs) were eligible for inclusion if they evaluated the effect of computerised clinical decision support systems that gave direct recommendations to health professionals to order or not order a diagnostic test, or presented testing options, and measured impact on diagnostic processes. Participant populations in the identified studies included people with diabetes, coronary artery disease, chronic obstructive pulmonary disease, asthma, angina, obesity, renal disease, HIV, dementia, children with fever without apparent source, chronic diseases and general primary care. Computerised clinical decision support systems related to ordering of tests for diagnosis (including screening), monitoring disease or monitoring treatment. The systems were primarily evaluated in primary care and also in emergency department and outpatient settings. Most computer systems gave advice (reminders, alerts, recommendations) at the time of care, were integrated with electronic medical records (sometimes with computerised physician order entry systems) and were often delivered via desktop or laptop computer (among other methods). Two reviewers independently selected studies for inclusion and disagreements were resolved by a third. Assessment of study quality Study validity was rated on a 10-point scale for five potential sources of bias: concealment of allocation, unit of allocation, adjustment for baseline differences, blinding of assessment and follow-up. It appeared that two reviewers independently assessed study quality with disagreements resolved by consensus or recourse to a third reviewer. Data extraction Two reviewers independently extracted data on key study characteristics. Disagreements were resolved by consensus or recourse to a third reviewer. Effectiveness assessment was limited to the impact on test ordering behaviour. A system was considered effective if it changed (in the intended direction) a prespecified primary outcome that measured use of diagnostic tests (two-tailed p<0.05). Where multiple outcomes were reported in the absence of a relevant primary outcome, a change in 50% or more outcomes was considered effective. Where a study had multiple arms, the system was considered effective any arm showed a benefit over a control arm. Methods of synthesis Studies were combined in a narrative synthesis grouped by objective (disease monitoring, treatment monitoring, diagnosis, other). Results of the review Thirty-five RCTs of 4,212 practitioners and 626,382 patients (range 164 to 400,000 patients) were included in the review, 33 of which reported statistical measures of the effects of computer systems on diagnostic test ordering. Most trials received a good methodological quality score (median score 8 out of 10, range 2 to 10). Eighteen of 33 studies showed improved testing behaviour with computerised clinical decision support systems, including five out of six studies for diagnosis, five out of eight for treatment monitoring, six out of 17 for disease monitoring and three out of three for other purposes. All four systems that aimed at reduced rates of test ordering showed significant effects. Authors' conclusions Some computerised clinical decision support systems improved practitioners' diagnostic test ordering behaviour, but determinants of success and failure remained unclear. CRD commentary This review addressed a broad research question supported by appropriate selection criteria. Evidence was identified from multiple sources. The full search strategy was reported in Haynes 2010 (see Other Publications of Related Interest). It was unclear whether the search was restricted by publication status; the authors stated that they could not rule out the possibility of publication bias and selective outcome reporting. Attempts were made to minimise potential for errors and bias in the review process. The decision not to combine studies in a meta-analysis due to obvious heterogeneity appeared appropriate. The alternative approach of vote counting had several limitations. The authors conclusions followed from the evidence as presented and seem broadly appropriate. The evidence was a simple summary of the number of statistically significant reports of one outcome across a diverse group of computerised clinical decision support systems, so the results did not give information about the overall size of effects or the impact of specific systems in specific settings. Implications of the review for practice and research Practice: The authors did not state any implications for practice. Research: The authors stated that more trials with more detailed description of system features and implementation were needed. Such studies should report adverse effects of systems, their impact on user workflow and satisfaction and the costs of design, development and implementation. Funding Canadian Institutes for Health Research Synthesis Grant. Bibliographic details Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB; CCDSS Systematic Review Team. Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implementation Science 2011; 6(88) Other publications of related interest Haynes RB, Wilczynski NL for the Computerized Clinical Decision Support System (CCDSS) Systematic Review Team. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: methods of a decision-maker-researcher partnership systematic review. Implementation Sci 2010; 5:12. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, Garg AX, Haynes RB. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013; 346: f657. Indexing Status Subject indexing assigned by NLM MeSH Consumer Behavior; Cooperative Behavior; Decision Making; Decision Support Systems, Clinical; Diagnostic Tests, Routine /methods /statistics & Global Health; Humans; Monitoring, Physiologic; Practice Patterns, Physicians' /statistics & United States; numerical data; numerical data /utilization AccessionNumber 12011006671 Date bibliographic record published 21/12/2011 Date abstract record published 22/05/2012 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|