This review found fourfold higher rates of stroke after closure treatment for patent foramen ovale in one randomised trial compared to observational studies. The randomised and observational evidence provided different conclusions and more research was needed. Lack of detail about the literature searches and the large contribution of observational evidence mean the reviews conclusions may not be reliable.

To review percutaneous closure and antiplatelet or anticoagulant therapy used for secondary stroke prevention in patients with patent foramen ovale and cryptogenic stroke.

Searches from previous systematic reviews were updated to August 2010 using MEDLINE. Reference lists of included papers and selected reviews were searched. Search terms were not reported. Only studies published in English were included.

Longitudinal randomised, non-randomised or single-arm studies that included at least 10 patients with presumed paradoxical embolism (including cryptogenic stroke and transient ischaemic attack) that evaluated medical treatment or percutaneous patent foramen ovale closure were eligible for inclusion. Studies had to report the incidence of recurrent stroke or transient ischaemic events over a minimum average follow-up period of 12 months.

Patient populations in the medical treatment and closure studies were fairly similar apart from medical treatment studies had a higher proportion of smokers (median 33% versus 22%) and those with stroke as the index event (median 76% versus 63%). The mean patient age was around 47 years and the median proportion of men was between 53% and 58%. Median proportions with atrial septal aneurysms were 23% to 31%, hypertension 23% to 24%, diabetes 6% and with hyperlipidaemia 18% to 25%. Medical treatments evaluated were antiplatelets (aspirin, clopidogrel or ticlopidine) or anticoagulants (warfarin or heparinoids) and the proportions of patients who received these treatments varied. For closure studies, 70% used multiple closure devices.

The authors did not report how many reviewers performed study selection.

Study design (retrospective or prospective), use of imaging modalities for detecting patent foramen ovale, duration of follow-up and methods for outcome assessment (use of a structured screening instrument for stroke detection, whether recurrent events were ascertained by a neurologist and use of neuroimaging) were used as measures of study quality.

The authors did not report how many reviewers performed the assessment.

Numbers of recurrent events (stroke, transient ischaemic attack or all ischaemic cerebrovascular events) and total follow-up duration in person-years were extracted or calculated from raw data or survival curves. Data were also extracted on peri-procedural adverse events for studies of percutaneous closure. Only data from patients with patent foramen ovale were extracted from studies with mixed populations.

Data were extracted by two reviewers. Discrepancies were resolved by consensus.

Incidence rates with 95% confidence intervals were calculated for each treatment arm (medical or closure). Meta-analysis was performed using random-effects Poisson regression models for single-arm studies and those comparing medical treatment and closure. Pooled incidence rates or incidence rate ratios, with 95% confidence intervals, were calculated.

An additional analysis using the same modelling method combined all data from single-arm and comparative studies. Sensitivity analysis was performed separately for studies that adjusted their analysis for confounders. Subgroup analyses were performed to investigate the effect of patient age (less than 60 years) and for medical treatment only (antiplatelets versus anticoagulants).

Meta-regression was used to explore the effects of atrial septal aneurysm, hypertension, diabetes, smoking, hyperlipidaemia and stroke (compared with transient ischaemic event) and trial design (observational studies versus the CLOSURE 1 trial of closure compared with aspirin and/or warfarin) and their interaction with the treatment effect.

Publication bias was assessed by calculating Spearman rank correlation coefficients between sample sizes and incidence rates.

Fifty-seven studies were included: seven comparative and 50 single-arm studies providing a total of 66 treatment arms. There were 17 arms of medical treatment (1,903 patients) and 49 of percutaneous closure (7,013 patients). Around 75% of the studies were prospective. More of the medical treatment studies used a structured screening instrument for recurrent stroke detection (88% versus 47% for closure studies), confirmation by a neurologist (82% versus 55%) and neuroimaging documentation (59% versus 45%).

Single-arm studies: Twenty-six out of the 49 closure studies reported no recurrent strokes and overall pooled incidence rate was 0.36 (95% CI 0.24 to 0.56) strokes per 100 person-years. The pooled incidence rate from the 18 medical treatment studies was 2.53 (95% CI 1.91 to 3.35) strokes per 100 person-years. Both estimates showed significant statistical heterogeneity. Pooled incidence rates for studies that enrolled patients younger than 60 years were 0.45 (95% CI 0.13 to 1.54) from nine closure studies and 2.30 (95% CI 1.43 to 3.68) from six medical studies. None of the factors assessed in meta-regression were associated with the incidence rates. There was a statistically significant positive correlation between the sample sizes of the closure studies and their incidence rates, which indicated possible publication bias.

Treatment comparisons: The incidence rate ratio of recurrent stroke for closure compared to medical treatment was 0.19 (95% CI 0.07 to 0.54; seven studies). Including single-arm studies in the analysis led to a similar estimate and also significant results that favoured closure for total events and transient ischaemic attacks. Anticoagulants significantly reduced incidence of recurrent stroke compared with antiplatelet treatment (incidence rate ratio 0.42, 95% CI 0.18 to 0.98; nine studies). The difference in recurrent stroke incidence was not statistically significant for the seven studies that compared these two treatments directly, but significant benefits were seen for anticoagulation for transient ischaemic attack and total events.

Comparison of observational and randomised controlled trial evidence: Patients in the closure arm of the trial were more likely to have had a stroke as their outcome event and to have experienced an atrial septal aneurysm than closure patients in the observational studies. In the trial no significant difference was seen between closure and medical treatment in the time to recurrent stroke. The incident rate for recurrent stroke for closure patients in the trial was significantly larger (1.34 events per 100 person-years, 95% CI 0.69 to 2.34) than for patients in the observational studies (0.36 events per 100 person-years, 95% CI 0.24 to 0.56).

More results were reported in the paper.

The authors appeared to conclude that outcome rates of ischaemic events among patients who received mechanical closure in a recent randomised controlled trial were around four times those seen in observational evidence. However, observational and randomised evidence provided different conclusions about the efficacy of closure and further randomised trials were needed.

This review had clear and reproducible inclusion criteria. The search strategy was unclear as it involved updating previous searches and details were not reported, only one database was searched and the search was limited to papers published in English so the search may not have been adequate and may have been at risk of bias. Data were extracted by two reviewers independently to reduce error and bias; it was not reported whether study selection and quality assessment were performed in the same way.

The statistical methods seemed appropriate. Methods to assess statistical heterogeneity were unclear. The authors’ conclusions focused on a comparison between one randomised trial and observational evidence, were hampered by potential bias in the observational evidence and the trial itself did not find any significant between-group differences.

Lack of detail about the literature searches and the large contribution of observational evidence mean the reviews conclusions may not be reliable.

Practice: The authors stated that based on the CLOSURE I trial results, closure could not be considered as the treatment of choice for presumed paradoxical embolism.

Research: The authors stated a need for more results from randomised trials that compared medical treatment and closure. Randomisation of patients into ongoing trials should be encouraged.

National Centre for Research Resources, USA; National Institute for Neurological Disorders and Stroke, USA.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.