|Investigating the effectiveness of critical care outreach services: a systematic review
|Esmonde L, McDonnell A, Ball C, Waskett C, Morgan R, Rashidian A, Bray K, Adam S, Harvey S
The authors of this review concluded that, although some benefits of critical care outreach services were reported, there was insufficient evidence to draw firm conclusions about their effectiveness and further research is required. This was a well-conducted review and the authors’ conclusions are likely to be reliable.
To evaluate the effects of critical care outreach services (CCOS) on patient and service outcomes.
The following sources were searched from 1996 to 2004 for published and unpublished studies, using the reported search terms: MEDLINE, EMBASE, CINAHL, PsycINFO, HMIC, the National Research Register, Dissertation Abstracts, the Cochrane Database of Systematic Reviews, the Cochrane CENTRAL Register, DARE, NHS EED, HTA, Science Citation Index, Social Sciences Citation Index, OMNI and TRIP. In addition, papers on MEDLINE and the Web of Science were searched for other studies conducted by key authors in the field, the reference lists of key publications, key reports and reviews were screened, and five named key journals were handsearched. Authors of relevant studies mentioned on the NHS Modernisation Agency website and experts in the field were contacted. Only studies written in the English language were included. Two studies that were published after the final search date were also included.
Study designs of evaluations included in the review
Studies that used concurrent or historical control groups were eligible for inclusion in the review.
Specific interventions included in the review
Studies that evaluated the introduction of CCOS or any outreach activity delivered by critical care staff (as defined by the NHS Modernisation Agency) were eligible for inclusion. The included studies evaluated different services delivered by different types of outreach teams, and were conducted in hospitals that varied in type and size. Most interventions incorporated an early warning/track-and-trigger tool and some studies used multi-component interventions. Most of the studies were conducted in England.
Participants included in the review
Studies of adults (aged over 18 years) were eligible for inclusion in the review.
Outcomes assessed in the review
Studies that assessed any measure of patient health or professional performance were eligible for inclusion. The review assessed mortality, length of stay, cardiac arrest, unplanned admissions and readmission to the critical care unit. Studies varied in the timing of the outcome assessment (ranging from immediately after the introduction of the service to 1 year after implementation); some studies included a ‘run-in’ period.
How were decisions on the relevance of primary studies made?
Two reviewers independently selected studies for inclusion.
Assessment of study quality
Two reviewers independently assessed validity using the criteria described by Thomas for randomised and non-randomised studies. The criteria assessed were selection bias, allocation bias, confounders, blinding, methods of data collection, withdrawals and drop-outs, analysis and intervention integrity. Each criterion was rated as strong, moderate or weak.
One reviewer extracted the data, which a second reviewer then checked. For each study, the outcome data were tabulated, usually with either the relevant 95% confidence interval (CI) or the level of statistical significance.
Methods of synthesis
How were the studies combined?
The studies were grouped by outcome and combined in a narrative that took account of study quality.
How were differences between studies investigated?
Differences between the studies were discussed in the text. The influence of the 6 unpublished studies was also discussed.
Results of the review
Twenty-three studies were included in the review (including two reports of the same original study). These comprised 2 randomised controlled trials (RCTs), 3 quasi-experimental studies, 1 controlled before-and-after study, 1 post-only controlled before-and-after study and 16 uncontrolled before-and-after studies. The sample size ranged from 15 to 180,000 patients.
Overall, the studies were of a poor quality. Only 9 studies were rated as ‘strong’ on at least three of the six quality criteria.
Mortality (22 studies): 7 studies reported a significant reduction in mortality associated with the intervention. These included an RCT, the highest quality non-randomised study and 5 uncontrolled before-and-after studies. One additional study reported a reduction in mortality associated with the intervention, but neither the statistical significance nor the study design was reported.
Length of stay (11 studies): 4 studies reported a significant reduction in length of stay associated with the intervention. These included 1 non-randomised study, 1 quasi-experimental study and 2 uncontrolled before-and-after studies.
Cardiac arrest (12 studies): 4 studies reported a significant reduction in cardiac arrest rates associated with the intervention. These included 1 RCT, 1 quasi-experimental study and 2 uncontrolled before-and-after studies.
Unplanned admissions (8 studies): 3 studies reported a significant reduction in unplanned admission to the critical care unit associated with the intervention. These included 2 uncontrolled before-and-after studies and 1 post-only study.
Readmission to the critical care unit (6 studies): 2 uncontrolled before-and-after studies reported a significant reduction in readmissions to the critical care unit associated with the intervention.
Although some benefits of CCOS were reported, there was insufficient evidence to draw firm conclusions about their effectiveness and further research is required.
The review addressed a clear research question. The use of broad inclusion criteria appears appropriate given the field of the review. The extensive search included attempts to locate unpublished studies, but no attempts were made to minimise language bias. Methods were used to minimise reviewer error and bias in the study selection, validity assessment and data extraction processes. There appear to be some inconsistencies in the referencing of studies. Validity was assessed using appropriate criteria. In view of the differences between the studies, a narrative synthesis that took account of study quality was appropriate. This was generally a well-conducted review and the authors’ conclusions are likely to be reliable.
Implications of the review for practice and research
Practice: The authors did not state any implications for practice.
Research: The authors stated the need for a thorough evaluation of the effect of CCOS in the UK on patient and service outcomes and service costs. Studies should evaluate different types of outreach services; consider the most appropriate time to evaluate services; select appropriate outcome measures; use multicentre studies; take account of different mixes of patients and providers; and include qualitative as well as quantitative research.
NHS R&D Service Delivery and Organisation Programme.
Esmonde L, McDonnell A, Ball C, Waskett C, Morgan R, Rashidian A, Bray K, Adam S, Harvey S. Investigating the effectiveness of critical care outreach services: a systematic review. Intensive Care Medicine 2006; 32(11): 1713-1721
Subject indexing assigned by NLM
Emergency Medical Services /organization & administration; Great Britain; Humans; Outcome Assessment (Health Care); Quality Assurance, Health Care /methods; Quality Indicators, Health Care
Database entry date
This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.