This review evaluated the effects of chemotherapy on cognitive function. The authors concluded that the meta-analysis supported the hypothesis that chemotherapy can have a negative impact on cognitive function, but most deficits ranged from small to moderate and were non significant. Inadequate reporting and/or shortcomings in the conduct of the review mean that the conclusions should be regarded with caution.

To estimate the effects of chemotherapy on specified domains of cognitive function.

PubMed, PsycINFO, CogNet, CINAHL and the Cochrane Database of Systematic Reviews were searched from inception to December 2004 for articles published in the English language; the search terms were reported. In addition, the references of relevant studies and reviews were screened. Unpublished studies might have been excluded as the authors stated that unpublished sources were not considered.

Study designs of evaluations included in the review

The studies had to report original data to be eligible for inclusion. Reviews and meta-analyses, commentaries and case reports were excluded.

Specific interventions included in the review

Studies of patients who had or were currently receiving chemotherapy were eligible for inclusion. Patients receiving cancer treatments known to cause cognitive deficits, such as brain irradiation or biologics, were excluded. Most of the included studies were in patients receiving standard doses of chemotherapy; other studies were in patients who had undergone bone marrow transplants or had received high-dose chemotherapy. A quarter of the studies included patients who currently received chemotherapy; in the other studies, the time since treatment varied between less than one and more than 5 years.

Participants included in the review

Studies of (exclusively) adult cancer patients were eligible for inclusion. Studies that only included patients with primary or metastatic central nervous system tumours were excluded. In the included studies, the mean age of the patients ranged from 35.9 to 62 years, the majority (84%) were female, and the most common type of cancer was breast cancer. Control groups, where these existed, were cancer patients who had received local treatment or healthy individuals matched for age and education.

Outcomes assessed in the review

The studies had to report cognitive function (quantitative measurement or inferential statistics) measured using reliable, valid and sensitive neuropsychological tests with published standardised administration procedures to be eligible. The studies also had to report sufficient data to enable the calculation of an effect size (ES). The review assessed the following eight cognitive domains: attention and concentration; executive function; information processing speed; language; motor function; visuospatial skill; verbal memory; and visual memory. The included studies each provided data for between three and eight domains.

How were decisions on the relevance of primary studies made?

The authors did not state how the papers were selected for the review, or how many reviewers performed the selection.

The reviewers developed a study quality scoring tool that assessed 15 study elements, including research question, study design, sample selection, description of neuropsychiatric tests findings, data analysis and results. The possible scores ranged from 0 to 30. The authors did not state how the quality assessment was performed.

The authors did not state how the data were extracted for the review, or how many reviewers performed the data extraction.

The data were extracted using a standardised form. The test data used in individual studies were assigned to the eight cognitive domains (details were reported of which tests were assigned to specific domains). ESs (Cohen's d) were calculated from standardised mean differences using the reported mean and standard deviation (SD); where these were not reported, these were derived from other statistics. Where individual studies used more than one test to measure the domain, a mean ES was calculated; this was also the case for tests that reported more than one score. Significance levels of 0.05 were inferred when the 95% confidence interval (CI) did not cross zero. An ES of less than 0.20 was considered negligible; an ES of 0.20 to 0.50 was considered small; an ES of 0.50 to 0.80 was considered medium; and an ES greater than 0.80 was considered large.

How were the studies combined?

The reviewer computed pooled ESs and 99% CIs, weighted by sample size, and pooled the variance for each cognitive domain.

How were differences between studies investigated?

Study details were tabulated and some differences were referred to in the text. The treatment effects were presented separately for studies that compared the treatment group data with a control group, with published normative data or with baseline data.

Sixteen studies (n=996) were included in the review: 12 cross-sectional studies and 4 longitudinal studies (no further details of study design were reported).

The study quality ranged from 19 to the maximum possible score of 30 (mean 23.5, SD 3.03).

Over all types of controls, only the moderate ES for visual memory was interpreted as statistically significant (-0.51, 99% CI: -1.01, -0.01; 10 studies). The pooled ES for other cognitive domains were negligible or small: -0.17 (99% CI: -0.62, 0.27; 12 studies) for attention and concentration, -0.26 (99% CI: -0.74, 0.20; 16 studies) for executive function, -0.44 (99% CI: -0.96, 0.07; 11 studies) for information and processing speed, -0.33 (99% CI: -0.78, 0.13; 11 studies) for language, -0.36 (99% CI: -0.80, 0.10; 11 studies) for motor function, -0.11 (99% CI: -0.57, 0.34; 11 studies) for visuospatial skills and -0.37 (99% CI: -0.83, 0.09; 13 studies) for verbal memory.

When comparing the test scores of chemotherapy patients with normative data, there were significant effects for executive function, information processing speed, verbal memory and visual memory. When comparing the test scores of chemotherapy patients and healthy controls, there were significant but small effects for language and verbal memory.

This meta-analysis supported the hypothesis that chemotherapy can have a negative impact on cognitive function, but most deficits ranged from small to moderate and were non significant.

The review stated a clear question and inclusion criteria. Several relevant sources were searched but no specific attempts were made to search for unpublished data; this can introduce publication bias into the review and it is also possible that relevant studies might have been missed. Details of how the review was undertaken were lacking, so it is unclear whether any attempts were made to reduce errors and bias in the study selection, quality assessment and data extraction processes. This is especially unfortunate as the data extraction involved, for example, a sophisticated assignment of the diverse neuropsychological tests to specific cognitive domains. The authors assessed the quality of the included studies using a checklist but only the composite score was presented; this makes it difficult for the reader to follow the assessment. In addition, details of the design of the included studies were not reported, so it is difficult to assess the level of evidence on which this review is based. Individual ESs appeared to vary considerably among studies but the heterogeneity of the results between individual studies was not formally assessed. The conclusions should be regarded with caution given the lack of information and/or shortcomings in the review procedure.

Practice: The authors did not state any implications for practice.

Research: The authors stated that more research on chemotherapy-induced impairments is needed to investigate impairment characteristics and to determine the clinical significance of cognitive deficits. In addition, it is necessary to identify valid, reliable, sensitive and specific tests for detecting short-term and persistent chemotherapy-induced cognitive impairments.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.