|
Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? |
Davis D, Thomson O'Brien M A, Freemantle N, Wolf F M, Mazmanian P, Taylor-Vaisey A |
|
|
Authors' objectives To review, collate and interpret the effect of formal Continuing Medical Education (CME) interventions on physician performance and health care outcomes.
Searching The authors searched the complete Research and Development Resource Base in Continuing Medical Education and the Specialised Register of the Cochrane Effective Practice and Organisation of Care Group and supplemented these searches with a search of MEDLINE (January 1993 to January 31, 1999). The search terms used were: 'randomized controlled trials' and 'random allocation' with 'randomized controlled trial' as both publication type and textwords. Other search terms added were: 'education', 'medical' and 'continuing'; 'education', and 'continuing'; and 'education' as well as variations of the text words: 'lectures', 'rounds', 'seminars', 'meetings', 'symposia', 'conferences', 'courses', 'workshops' and 'small groups'.
Study selection Study designs of evaluations included in the reviewRandomised controlled trials (RCT) with at least 50% of study participants being active physicians.
Specific interventions included in the reviewFormal didactic and/or interactive CME interventions (conferences, courses, rounds, meetings, symposia, lectures and other formats) using only educational activities meant to be persuasive rather than those that coerced or provided incentives to the learner- participant. Interventions that deployed post-course reminders, audit, and feedback or other measures to change physician performance were excluded.
Participants included in the reviewA priori criteria for participants were only stated as requiring that more than 50% of a study's participants should be physicians. Participants actually included were American, Canadian or French physicians (general practitioners, general pediatricians, family physicians, and internists).
Outcomes assessed in the reviewPhysician performance and health care outcomes (including patient behaviour outcomes such as adherence to medication or smoking cessation rates). Outcomes were classed positive if one or more primary outcome measure related to physician performance or patient health care demonstrated a statistically significant change and negative if no such change occurred.
How were decisions on the relevance of primary studies made?Studies were selected independently by three of the authors.
Assessment of study quality The authors assessed validity using criteria previously published by the Cochrane Collaboration (Other Publications of Related Interest no.1). The authors did not state how the papers were assessed for validity.
Data extraction Two of three reviewers independently performed the data extraction. Data were extracted for the categories of: study identification and year of publication, participants and topic, intervention, outcomes, and effect size with 95% confidence interval for each individual study.
A standardised effect size was estimated for each individual study group.
Methods of synthesis How were the studies combined?Only seven studies provided data for the quantitative pooling. For those studies, standardised effect sizes (SES) were calculated with 95% confidence intervals (CIs) using a random-effects model as described by Smith et al. (Other Publications of Related Interest no.2).
The remaining studies were discussed in a narrative review.
How were differences between studies investigated?The authors do not state how differences between the studies were investigated.
Results of the review Fourteen RCTs with 17 interventions were included in the review with 1,294 participants.
Only one of the fourteen trials described adequate concealment of allocation to randomisation indicated in the published report and ten trials documented adequate follow-up. Outcomes were assessed blindly in only seven studies.
Within the fourteen studies, 24 separate interventions were tested and, of which, the outcomes of seventeen were reported.
Nine interventions generated positive changes in professional practice, and three of four interventions altered health care outcomes in one or more measures.
In seven studies, sufficient data were available for effect sizes to be calculated. Overall, no significant effect of these educational methods was detected (SES 0.34, 95% CI: -0.22, 0.97).
However, interactive and mixed educational sessions were associated with a significant effect on practice (SES 0.67, 95% CI: 0.01, 1.45).
Authors' conclusions The authors state that the data show some evidence that interactive CME sessions that enhance participant activity and provide the opportunity to practice skills can effect change in professional practice and, on occasion, health care outcomes. Based on a small number of well-conducted trials, didactic sessions do not appear to be effective in changing physician performance.
CRD commentary The authors have clearly stated the research question and inclusion and exclusion criteria. The literature search appears to be thorough but it is not stated whether there were language restrictions. The authors do, however, include searches from topic-specific databases. The quality of the included studies was formally assessed and the authors have reported how the articles were selected, and who performed the selection and data extraction. The authors state that a critical limitation of the review was the small number of studies included and the doubt that the included participants were not representative of the physician population as a whole.
The data extraction is reported in tables and discussed in the text of the review. The studies were combined in a statistical meta-analysis where data were available and a narrative review of the remaining data was undertaken. A random-effects model was used without any report of heterogeneity. If significant heterogeneity was found, it is possible that these studies should not have been pooled.
Implications of the review for practice and research Research: The authors state that further research is needed.
1. To investigate the changing demographics of the physician population and gender mix, the increasing numbers of graduates from problem-based schools and primary care training, and the inclination towards and skills required for self-directed lifelong learning.
2. To investigate those factors that accelerate or impede translation from one domain to another.
The authors state that this research has several implications in practice.
1. Medical license boards need to rethink the value of the CME credit system. 2. Medical schools, specialty societies and other providers of CME must reconsider the value of the credit they provide, as well as the type and duration of learning activities they produce.
3. Quality monitoring bodies must evaluate the services they provide to a large, complex and expensive CME enterprise that values the production of single-session, teacher-centred activities over learner achievement.
4. Physicians must reflect on the value of attending ineffective didactic sessions instead of interactive, challenging and sequenced activities that increase the potential for positively affecting their performance and the health of their patients.
Funding National Institutes of Health Fogarty International Center, grant number 1 FO6 TW 02123; National Health Service Research and Development Program
Bibliographic details Davis D, Thomson O'Brien M A, Freemantle N, Wolf F M, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999; 282(9): 867-874 Other publications of related interest 1. Bero LA, Grilli R, Grimshaw J, Oxman AD, Zwarenstein M, editors. Cochrane Effective Practice and Organisation of Care Group. In: The Cochrane Library, Issue 2, 1999. Oxford: Update Software. 2. Smith TC, Spiegelhalter DJ, Thomas A. Bayesian approaches to random-effects meta-analysis: a comparative study. Stat Med 1995;14:2685-99.
This additional published commentary may also be of interest. Holm HA. Review: interactive, but not didactic, continuing medical education is effective in changing physician performance. Evid Based Med 2000;5:64.
Indexing Status Subject indexing assigned by NLM MeSH Clinical Competence; Congresses as Topic; Education, Medical, Continuing /methods; Outcome Assessment (Health Care) AccessionNumber 11999009676 Date bibliographic record published 31/08/2001 Date abstract record published 31/08/2001 Record Status This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn. |
|
|
|