|Changing physician performance: a systematic review of the effect of continuing medical education strategies
|Davis D A, Thomson M A, Oxman A D, Haynes R B
To review the literature relating to the effectiveness of education strategies designed to change physician performance and health care outcomes.
From 1975 to 1994: MEDLINE, ERIC, NTIS, CINAHL, Healthline, EMBASE to identify trials in the broad domain of continuing health education. These results were combined with the Cochrane search strategy for identifying RCTs. Also, manual searches of journals and the bibliographies of retrieved articles were made, and experts in the field were consulted. (The resultant data source is the Research and Development Resource Base in Continuing Medical Education, RDRB-CME).
Study designs of evaluations included in the review
Randomised controlled trials (RCTs) or pseudo-randomised trials (e.g. with alternating allocation) were included.
Specific interventions included in the review
Replicable educational interventions directed at changing clinical behaviour or health outcomes. For example: educational materials; formal continuing medical education programmes (e.g. conferences, seminars, workshops, small group sessions, traineeships); outreach visits, including academic detailing; opinion leader strategies; patient-mediated methods; audit with feedback; physician reminders.
Participants included in the review
Health professionals, the majority of whom were either practising physicians (predominantly internists of family physicians) or medical residents, were included.
Outcomes assessed in the review
Physician performance or health care outcomes were assessed.
How were decisions on the relevance of primary studies made?
The inclusion criteria were pilot tested by all authors in five studies. At least two authors then independently applied the criteria to all of the remaining studies.
Assessment of study quality
The authors do not state that they assessed validity.
The outcomes were assessed as positive, negative, inconclusive or mixed. The data were extracted by one author and checked by at least one other.
Methods of synthesis
How were the studies combined?
A narrative synthesis was undertaken
How were differences between studies investigated?
The authors do not state how differences between the studies were investigated.
Results of the review
Ninety-nine studies, evaluating 160 separate interventions (apart from placebo or non-intervention controls), were included. Fifty-one studies included internists, 35 family physicians or general practitioners, and 15 did not specify physician type or background. Few studies focused on specialists and many included more than one category of physician. Residents were included in one third of the studies.
Of the 160 interventions, 99 (62%) showed an improvement in at least one major outcome measure. 53 (33%) failed to demonstrated a change and 8 (5%) showed mixed results. 145 interventions focused on physician performance and 70% of these demonstrated change. In comparison 48% of the 46 interventions targeted at changing health care outcomes succeeded. Effect sizes were small to moderate in the positive studies. 81 single intervention strategies were used in the trials, 49 (60%) of which demonstrated change in at least one major outcome, whilst 30 (37%) failed to demonstrate such a change, and 2 effected change in physician performance but not health care outcomes. Effective strategies were: outreach visits, opinion leaders, patient-mediated interventions and physician reminders. More variable results were found for audit with feedback, and short formal continuing medical education programmes generally found no change. 39 interventions used two educational methods. 25 (64%) of these were positive, 12 (31%) were negative or inconclusive and 3 gave mixed results. The combination of effective single interventions in pairs generally resulted in positive changes in contrast to combinations of less effective single interventions. 39 interventions used three or more educational strategies, 31 (79%) of these were positive (2 of which included positive changes in health care outcome), 5 (13%) were negative or inconclusive and 3 displayed mixed results. Results are presented by the degree to which primary study authors considered or defined a clinical need, performance gap or barrier to performance change. Greater attention to needs/barriers was associated with greater positive change. Results are presented by domain of behaviour change targeted; approximately one third of interventions were in the area of health promotion and disease screening, and two thirds in clinical disease management. Subcategories within these groups are identified.
Physician performance may be altered by many educational interventions and, to a lesser extent, so may health care outcomes. These alterations are most often small, less often moderate, and rarely large. The effect of the interventions is not consistent across practitioners, settings or behaviours. Change occurred relatively frequently when barriers to change were addressed or gaps were demonstrated and resources deployed to help the learner. Widely-used continuing medical education delivery methods, such as conferences, have little direct impact on improving professional practice. More effective methods such as systematic practice-based interventions and outreach visits are seldom used by continuing medical education providers.
This review builds on previous work of the authors (see Other Publications of Related Interest for details). Little information on individual studies is given, but this would be difficult given the large number of studies identified and the even larger number of interventions evaluated. The authors provide a thorough review and highlight potential limitations of the work. In particular, it was difficult to identify all relevant studies, while inadequate reporting of original studies made complete data extraction difficult.
Davis D A, Thomson M A, Oxman A D, Haynes R B. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995; 274(9): 700-705
Other publications of related interest
1.Haynes RB, Davis DA, McKibbon A, Tugwell P. A critical appraisal of the efficacy of continuing medical education, JAMA 1984;251:61-64. 2.Davis DA, Thomson MA, Oxman AD, Haynes RB. Evidence for the effectiveness of CME: a review of 50 randomized controlled trials. JAMA 1992;268:1111-117.
Subject indexing assigned by NLM
Decision Making; Education, Medical, Continuing /standards; Outcome Assessment (Health Care); Practice Patterns, Physicians' /trends; Program Evaluation; Quality of Health Care /trends; United States
Date bibliographic record published
Date abstract record published
This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.