|A systematic review: the effect of clinical supervision on patient and residency education outcomes
|Farnan JM, Petty LA, Georgitis E, Martin S, Chiu E, Prochaska M, Arora VM
This review concluded that enhanced clinical supervision of graduate medical education trainees was associated with improved patient and educational outcomes. Relevant studies might have been missed, and the included studies were of limited quality, so these conclusions should be treated with caution.
To assess the effect of clinical supervision programmes on patient and educational outcomes.
PubMed and JSTOR were searched up to November 2010, for studies in English. Search terms were reported. Relevant journals that were not electronically indexed between 1966 and 2010 (including Medical Education) were handsearched. Reference lists of retrieved studies were searched. Proceedings from medical education meetings (including the Association of American Medical Colleges, Annual Meeting, and the Accreditation Council for Graduate Medical Education, Annual Meeting) were searched from 2007 to 2010.
Randomised controlled trials (RCTs), concurrent controlled studies, and before-and-after studies that evaluated the implementation of a supervision intervention for graduate medical trainees, in any field of practice and any clinical setting, were eligible for inclusion. The eligible studies had to report patient or education outcomes, and they had to be conducted in the USA or Canada.
The included studies were of the following specialties: psychiatry, emergency medicine, surgery and subspecialties, obstetrics and gynaecology, internal medicine, anaesthesia, paediatrics, and radiology. Most included studies evaluated direct supervision or oversight interventions, while some assessed indirect models of supervision. Most studies assessed patient-related outcomes; a few evaluated education-related outcomes. Most studies were conducted at one site.
All reviewers assessed all studies for inclusion, but it was unclear how decisions were made.
Assessment of study quality
The quality of studies was assessed using the validated Medical Education Research Study Quality Instrument (MERSQI), with a maximum score of 18. This instrument evaluated six domains of study quality: design, sampling, type of data, validity, data analysis, and outcomes.
At least one reviewer assessed study quality, and any disagreements were resolved by consensus.
The data were extracted, as they were reported in the original studies, by four reviewers (each abstracting a quarter of the studies), and checked by two other reviewers.
Methods of synthesis
The studies were combined in a narrative synthesis, grouped by the type of outcome evaluated.
Results of the review
Twenty-four studies were included; three were RCTs, six were before-and-after studies, eight were retrospective cohort studies, six were prospective cohort studies, and one was a case-control study. Some studies had small samples. Their quality scores ranged from 8.5 to 16, with a mean of 11.8.
Patient outcomes: There were 21 studies reporting patient outcomes. Eight assessed the impact on complications of supervision during an invasive diagnostic or therapeutic procedure. These studies showed that direct or indirect resident supervision of trainees, in the operating theatre, had either a positive or no effect on complication rates, mortality, operating time, and postoperative pain.
Four studies assessed supervisor-initiated changes for missed or inappropriate diagnoses. Two found generally similar interpretations of patient cases (abdominal computed tomography scans or trauma cases) between trainees and their supervisors. Two assessing oversight or direct supervision provided by attending physicians, showed that attending physicians frequently modified the diagnosis or treatment plans of the resident physician.
Six studies evaluated resident practice patterns, in terms of resource use and compliance with protocols. These studies showed that attending supervision had a positive effect on resident adherence to quality-of-care guidelines, but did not significantly affect resource use.
The results for other outcomes, such as educational outcomes, physicians perception of patient illness severity, and performance on standardised tests were reported.
Enhanced clinical supervision of trainees was associated with improved patient and educational outcomes.
This review's inclusion criteria were clear. Only two relevant databases were searched, so relevant studies might have been missed. Some efforts were made to find unpublished studies to minimise publication bias. The searches were restricted to studies in English, which may have introduced language bias. It appears that attempts were made to minimise reviewer error and bias in the review process. Appropriate criteria were used to assess study quality and most of them were of low-to-moderate quality. Given the high level of clinical variation between the included studies, a narrative synthesis was appropriate. The included studies were conducted in the USA and Canada, which limits the generalisability of the findings to other countries.
In view of the potential for missed relevant studies and limited quality of the included studies, the authors' conclusions should be treated with caution.
Implications of the review for practice and research
Practice: The authors stated that there was insufficient evidence to require continuous on-site attending supervision, as recommended by the Institute of Medicine in the USA.
Research: The authors stated that further studies were required to establish the effect of enhanced on-site supervision, at any level, particularly during nights and weekends. Research should develop validated measures of the effects of clinical supervision.
Funded by the Department of Medicine, University of Chicago.
Farnan JM, Petty LA, Georgitis E, Martin S, Chiu E, Prochaska M, Arora VM. A systematic review: the effect of clinical supervision on patient and residency education outcomes. Academic Medicine 2012; 87(4): 428-442
Subject indexing assigned by NLM
Accreditation; Clinical Competence; Humans; Internship and Residency /organization & Job Satisfaction; Medical Errors /prevention & Outcome Assessment (Health Care); Patient Safety; Physician's Practice Patterns; Professional Autonomy; United States; administration /standards; control
Date bibliographic record published
Date abstract record published
This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.