The authors concluded that health status data suggested primary care complementary therapy services improved health outcomes, but data on the impact on NHS costs were scarcer and inconclusive. The authors' cautious conclusions appeared to reflect the poor quality evidence, however, due to the lack of validity assessment and the reporting of some data and review methods their reliability is uncertain.

To determine the impact of primary care complementary therapy services on improving health outcomes and reducing NHS costs.

PubCAM and AMED databases were searched for published and unpublished articles from November 2003 to April 2008. Search terms were reported. Several journals were handsearched (included Complementary Therapies, Homeopathy and Acupuncture in Medicine). Google Scholar and the database of registered users for SF36 (Medical Outcomes Study 36-Item Short-Form Health Survey) and MYMOP (Measure Yourself Medical Outcome Profile) questionnaires were searched. Experts in the field were contacted. Bibliographies of previously collected articles were scanned.

Studies that evaluated primary care complementary therapy services conducted in England or Wales and delivered by National Health Service (NHS) clinicians or professional therapists were eligible for inclusion. To be included studies had to report outcomes using data from SF36 or MYMOP tools and/or NHS cost data from medical records. Studies were excluded if they only reported throughput, described the setting-up of the service, were operated by a charity or privately or if the service was part of an acute hospital department.

Most studies were conducted in a primary care setting; one study was conducted in an out-patient department. The range of complementary therapy services offered included acupuncture, homeopathy, chiropractic, osteopathy, massage, counselling, spiritual healing, reflexology, shiatsu and medical osteopathy and herbal medicine. Control groups were used in only two studies and included waiting list controls and non-service users.

The authors did not state how many reviewers selected studies for inclusion.

The authors did not state that they assessed validity.

Baseline and follow-up data for health status scores, rates and costs together with confidence intervals (CI) were extracted. Missing CIs were calculated where possible.

The authors did not report how many reviewers extracted data.

Studies were combined in a narrative synthesis with additional data presented in tables.

Fourteen studies (in 21 articles) were included in the review (number of matched records 1,614): two controlled studies and 12 uncontrolled pre-post intervention studies. Follow-up (where reported) ranged from eight weeks to six months after start of treatment.

Complementary therapy services had a statistically significant positive effect on service users for health status SF36 scores for role physical, social functioning, pain and vitality; results were mixed for role emotional, mental health and general health (six uncontrolled studies). One controlled study reported a moderate to strong improvement in the complementary therapy group compared to waiting list control for SF36 domains of role physical, role emotional, social functioning, pain, vitality, mental health and general health. Complementary therapy services had a statistically significant positive effect on service users for MYMOP health status scores for first symptom (five studies) and second symptom (four studies). One study reported no significant differences for activity scores.

Five studies reported that complementary therapy services had a positive effect on MYMOP health status scores for the first symptom identified by service users.

Three out of six studies reported that prescriptions were substantially reduced for service users (range 39% to 57% reduction), one study reported no change and one study reported an increase in costs. Three out of six studies reported a reduction in GP (general practitioner) consultation rates (range 31% to 71%) from complementary therapy service users, one study reported no change and one study reported an increase of £6.26 per patient per month. One randomised controlled trial reported no differences in overall costs for conditions suffered by osteopathy service users and non-users (average total costs £22, 95% CI -£159 to £142), but costs relating to spinal pain (which included costs of medical osteopathy consultations) were higher in the medical osteopathy group (average cost difference £65, 95% CI £32 to £155).

Uncontrolled health status data suggested that primary care complementary therapy services improved health outcome scores. Data on the impact of these services on NHS costs were scarcer and inconclusive. The overall quality of these evaluations was poor.

The review question was clear and had appropriate inclusion criteria. Several relevant sources were searched. Efforts were made to reduce the potential for publication bias. It was unclear whether efforts were made to reduce reviewer error and bias throughout the review process. Validity was not assessed and it was, therefore, difficult to judge the quality of the included studies. The authors stated that study quality was poor overall. Most of the included studies appeared to be uncontrolled pre-post studies and used health service data, both of which were prone to various biases. A narrative synthesis was appropriate given the differences between studies. However, limiting studies in the review to those conducted in primary care settings in England and Wales meant that the results may not have been generalisable to other settings. Some relevant data were reported separately in tables. There were no data on characteristics of participants, which would have aided interpretation of outcome data. The authors' cautious conclusions appeared to reflect the poor quality evidence, however, due to the lack of validity assessment and the reporting of some data and review methods their reliability is uncertain.

The authors did not state any implications for practice.

Research: The authors stated that further robust research was required that utilised standardised health outcome tools and used appropriate statistical analysis. Further discussion within the NHS healthcare community was required to determine optimum ways of collecting and reporting NHS cost data for the evaluation of complementary therapy services within a primary care setting.

National Co-ordinating Centre for Research Capacity Development.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.