|
Computer-based guideline implementation systems: a systematic review of functionality and effectiveness |
Shiffman R N, Liaw Y, Brandt C A, Corb G J |
|
|
Authors' objectives To analyse the functionality provided by recent computer-based guideline implementation systems and review the effectiveness of the computer-based interventions in influencing clinicians' behaviours and changing patient outcomes. Only information relating to the effectiveness of the computer-based interventions is presented in the following abstract.
Searching MEDLINE and CINAHL were searched from 1992 through January 1998. The search terms included the following MeSH terms: 'algorithm', 'computer-assisted decision making', 'computer-assisted therapy', 'consensus statement', 'guideline adherence', 'health planning guidelines', 'health service research', 'medical audit', 'practice guideline', 'process and outcome evaluation', 'quality assurance', 'quality of health care' and 'reference standard'; and the following textwords: 'remind$', 'alert$', 'guideline$', 'implement$' and 'computer$'. Books and bibliographies of primary studies and review articles were also scanned. No language restrictions were reported.
Study selection Study designs of evaluations included in the reviewThe review only included studies published in or after 1992 and that included an evaluation component that objectively studied the effectiveness of the system in a clinical setting. No further a-priori inclusion criteria with regards to study design were reported. The methodologies used by included studies were randomised controlled trials, controlled trials, and time-series studies (none of which incorporated external controls; one applied a switchback design).
Specific interventions included in the reviewThe use of computers as part of an implementation strategy for clinical practice guidelines. Articles were included if the authors indicated an intention to implement guideline recommendations and the interventions were intended to influence health care providers. Included studies were reported to have used national guidelines (which were locally modified in several cases) or locally developed guidelines. Thirteen of the guideline implementation addressed management issues and therapy, including one system that provided pre-authorisation for surgical procedures and another system that was intended to improve discussion of advance directives. The other seven systems provided maintenance activities. All systems provided patients-specific recommendations and with one exception, and all systems provided recommendations concurrently with care. The type of intervention used as control was not reported.
Participants included in the reviewHealth care professionals and patients in any practice setting. Type of medical speciality reported by studies varied considerably and included surgical procedures, screening, low back pain, advanced directives, pharmacotherapy in psychiatry, diabetes, common paediatric problems, preventative service, HIV management, lipid management programme, occupational exposure and pressure ulcers.
Outcomes assessed in the reviewAdherence and other process measures (such as documentation, and user satisfaction) and patient outcome measures when available. The specific outcome variables that were measured by included studies varied considerably.
How were decisions on the relevance of primary studies made?The authors do not state how the papers were selected for the review, or how many of the reviewers performed the selection.
Assessment of study quality The authors do not report a method of assessing the validity of included studies as such, but did report having used the classification system reported by Grimshaw and Russell (see Other Publications of Related Interest no.1) to determine the type of study used. The authors do not state how the papers were assessed for validity, or how many of the reviewers performed the validity assessment.
Data extraction Two of the authors conducted data extraction and disagreements were settled by discussion. Data were extracted for the following categories: study details, domain, system, site, source of guideline, components of the guideline implementation system (recommendation, explanation, documentation, registration, communication, calculation, presentation, aggregation), study design, and outcome assessment (documentation, adherence, satisfaction, patient outcomes and cost).
Methods of synthesis How were the studies combined?The studies were combined in a narrative summary.
How were differences between studies investigated?It was reported that, because of the wide variety of study types and outcome variables, no quantitative meta-analysis of the results was possible.
Results of the review There were 20 studies evaluating the effectiveness of guideline implementation which included nine randomised controlled trials, one controlled trial, and ten time-series studies. It was not reported how many participants or health care professionals were included.
Four studies looked at documentation and found improvement in each case. Eighteen of the 20 studies evaluated provider adherence to the guidelines. In 14 of the 18, some level of improved adherence was described. Clinician satisfaction was addressed in four studies, two of which found that users were satisfied with computer-based guidelines. The remaining two studies reported problems with data entry which was found to be tedious or difficult. Eight studies examined patient outcomes (three of which included cost). Three studies reported improvement in patient outcomes. Use of lipid tracking system was associated with improvements in patients' cholesterol and lipid fractions. A system for prevention of pressure ulcers was associated with a decreased incidence of decubiti and one study reported a significant improvement in the completion of advance directives using a computer-based reminder system. An intervention to substitute appropriate antihypertensives for calcium channel blockers did not have any effect on patients' blood pressure, and alerts about appropriate HIV management did not change admission rates, emergency department visits, survival or pneumocystis admissions.
Cost information Yes, three of the included studies looked at costs. A study of an intervention for low-back pain found no effect on cost, whereas costs increased in both system for management of health care workers exposed to body fluids and another pre-authorised surgery.
Authors' conclusions In the studies described here, different types of guidelines, different settings, and different system implications make conclusions about the relationship between information management services and outcomes difficult. In addition, a component of publication bias is likely to be present, in that the generally favourable results may represent a biased subset of system implications.
CRD commentary The review included a clearly stated objective, although the authors did not specify clear inclusion/exclusion criteria. MEDLINE and CINAHL were the only databases searched and no attempt was made to look for unpublished data. The authors noted that a component of publication bias is likely to be present, in that the general favourable results may represent a biased subset of system implications. The review only included studies published in or after 1992, but the authors report their reasons for doing this. However, due to the search strategy used some important information may have been missed and publication bias cannot be ruled out. Information relating to the primary studies was clearly reported and relevant details were presented in table format. The quality of included studies was not systematically assessed.
The decision not to undertake a meta-analysis was appropriate and the authors' conclusions follow from the results.
Implications of the review for practice and research Practice: The authors did not report any implications for practice.
Research: The authors note that to adequately evaluate the effect of those services on the success or failure of a computer-based guideline implementation, more of the confounding variables need to be controlled.
Funding National Library of Medicine, grant numbers 1-R29-LM-05552 and T-15-LM-07056.
Bibliographic details Shiffman R N, Liaw Y, Brandt C A, Corb G J. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. Journal of the American Medical Informatics Association 1999; 6(2): 104-114 Other publications of related interest 1. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993;342:1317-22.
Indexing Status Subject indexing assigned by NLM MeSH Decision Making, Computer-Assisted; Decision Support Systems, Clinical; Guideline Adherence /organization & Humans; Information Management; Practice Guidelines as Topic; Practice Patterns, Physicians'; administration AccessionNumber 11999005337 Date bibliographic record published 31/03/2001 Date abstract record published 31/03/2001 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|