|Accuracy and impact of risk assessment in the primary prevention of cardiovascular disease: a systematic review
|Brindle P, Beswick A, Fahey T, Ebrahim S
The authors concluded that the Framingham risk scores appear to overestimate the risk of cardiovascular disease (CVD) in low-risk populations and underestimate it in high-risk populations. There is no good evidence that CVD risk scoring by a clinician is effective for primary prevention. The review was well-conducted in most respects and these conclusions appear reliable.
To assess the accuracy of Framingham-based cardiovascular disease (CVD) risk-scoring methods among different populations, and to determine their impact on clinical outcomes in the primary prevention of CVD.
The Cochrane CENTRAL Register, MEDLINE, EMBASE, CINAHL, PsycINFO, ISI Proceedings and Zetoc were searched from inception to September 2004; the search terms were reported. The reference lists of retrieved articles were checked, as were key journals. The search was restricted to published studies. No language restrictions were applied.
Studies of the accuracy or clinical impact of Framingham-based risk scoring were eligible for inclusion. Studies of accuracy were required to compare the risk of coronary heart disease (CHD) or CVD in a test population with the risk predicted by the Framingham equation. Studies of risk scores not in current clinical practice, or which reported only fatal outcomes, were excluded. Studies of clinical impact were required to be randomised controlled trials (RCTs) comparing the use of a Framingham-based risk scoring method by a health care professional versus usual health professional care. At least 80% of the participants were required to be free from clinically established CVD. These studies were required to report one or more of the following outcomes: fatal or nonfatal cardiovascular or coronary events, risk factors, absolute CVD or CHD risk, prescription of risk-reducing drugs, and/or changes in health-related behaviour.
Studies included in the review of accuracy were conducted among 27 population groups aged from 30 to 80 years. The groups included representative samples of men and/or women with diabetes, high cholesterol, treated hypertension, a family history of CVD and/or no CHD on angiography. They used recent Framingham risk scoring methods (see Other Publications of Related Interest nos.1-2). The studies used differing methods to ascertain cases and to define events and, consequently, the review reported broad outcomes: i.e. combined fatal and nonfatal CVD or CHD. The length of follow-up varied from 1 to 20 years.
The participants in studies included in the review of clinical impact were male and female general practice patients or out-patients with pre-diagnosed hypertension or diabetes. Their ages ranged from 19 to 80 years, where reported. Patient risk was communicated to the doctor verbally, by documenting it prominently on the notes, or by a clinical decision support system. Risk scores were based on one of two CVD risk scores (see Other Publications of Related Interest nos.1 and 3). The outcomes included absolute risk, treatment, referral and changes in risk levels. The length of follow-up was 8 weeks to 21 months, where stated.
Two reviewers independently selected articles for inclusion.
Assessment of study quality
The following quality criteria were considered for studies included in the review of clinical impact: randomisation method, allocation concealment, baseline comparison between the groups and blind assessment of the outcomes.
Two reviewers independently conducted the quality assessment, with any disagreements resolved by discussion with each other or the project advisory panel.
For the review of accuracy, risk ratios were calculated from the numbers of predicted versus the number of observed events over 10 years in each study, with 95% confidence intervals (CIs). For the review of clinical impact, event rates or mean changes in the intervention and control groups were reported in a table with 95% CIs or standard deviations, respectively, and p-values where there was a significant difference between the groups.
Two reviewers independently extracted the data, with any disagreements resolved by discussion with each other or with the project advisory panel. Study authors were contacted for more information if necessary.
Methods of synthesis
The results were combined in a narrative. In addition, study findings on clinical impact were tabulated and findings on accuracy were presented in a forest plot, ordered by the level of observed risk . Statistical heterogeneity was assessed using the χ2 and I2 statistics.
Results of the review
Thirty-one studies on CVD or CHD risk scoring were included: 4 RCTs and 23 controlled studies on accuracy (n=71,727) and 4 RCTs on clinical impact (n=3,762).
Among the RCTs of clinical impact, methodology was poorly reported. Three used individual or cluster randomisation, but the fourth used alternate allocation. None reported blind outcome assessment. Losses to follow-up ranged from 17% at 8 weeks to 14% at 12 months, where reported. Only one study used a power calculation.
Accuracy: observed versus predicted 10-year risk (27 studies).
None of the studies were pooled, owing to highly significant heterogeneity (p<0.00001). For CHD, the ratio of predicted to observed risk ranged from 0.43 to 2.87. Under-prediction was associated with higher risk populations (e.g. patients with a family history of premature CHD) and over-prediction with lower risk populations. Most of the CVD studies conformed to a similar pattern whereby increasing risk was associated with under-prediction. However, there were fewer studies and the range of risk was narrower.
Effectiveness: risk scoring versus usual care (4 RCTs).
One RCT (n=323) found no difference for any outcome between diabetic patients whose CVD risk score was documented on the notes and those whose score was not. However, in a high-risk subgroup (>20% 5-year risk), medication was more likely to be prescribed for the intervention group (p=0.01). A second RCT (n=332) found that hypertensive men randomised to a cardiovascular risk chart group had significantly lower systolic blood-pressure and were more likely to be prescribed cardiovascular drugs than controls receiving usual care. The other 2 RCTs (n=3,107) reported no clinically significant difference between the groups for any outcome.
The Framingham risk scores appear to overestimate the risk of CVD in low-risk populations and underestimate it in high-risk populations. There is no good evidence that CVD risk scoring by a clinician is effective for primary prevention.
The review objectives and inclusion criteria were clear and relevant sources were searched, though the restriction to published studies may have meant that some studies were missed. Publication bias does not appear to have been assessed. Steps were taken to minimise error and bias in the review process by having more than one reviewer make decisions about the study selection, quality assessment and data extraction. Relevant quality criteria were used to assess the RCTs of clinical effectiveness, but the quality of the other included studies was not systematically assessed and relevant details (such as follow-up rates) were not reported. The authors used suitable methods to investigate statistical heterogeneity between the studies, and appropriately decided to combine both groups of studies in narratives, rather than report summary measures. Although the studies on accuracy do not appear to have been systematically evaluated for quality, they provided a large body of evidence that consistently supported the authors’ interpretation of the data. The review appears to have been well-conducted in most respects and the authors’ conclusions appear likely to be reliable.
Implications of the review for practice and research
Practice: The authors did not state any implications for practice.
Research: The authors stated that absolute CVD risk assessment requires work to make it effective as a clinical tool (e.g. a variable representing social deprivation, or recalibration for background risk by geographical area and/or ethnic group).
UK Department of Health, Policy Research Programme, project number RDD/030/064; Wellcome Trust.
Brindle P, Beswick A, Fahey T, Ebrahim S. Accuracy and impact of risk assessment in the primary prevention of cardiovascular disease: a systematic review. Heart 2006; 92(12): 1752-1759
Other publications of related interest
1. Anderson KM, Odell PM, Wilson PW, Kannel WB. Cardiovascular disease risk profiles. Am Heart J 1991;121:293–8.
2. Wilson PW, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation 1998;97:1837–47.
3. Hetlevik I, Holmen J, Kruger O. Implementing clinical guidelines in the treatment of hypertension in general practice: evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care 1999;17:35–40.
This additional published commentary may also be of interest.
Havranek EP. Review: Framingham risk scores have variable accuracy in predicting CVD events in different patient populations. Evid Based Med 2007;12:87.
Subject indexing assigned by NLM
Adult; Aged; Cardiovascular Diseases /prevention & control; Female; Humans; Male; Middle Aged; Prognosis; Randomized Controlled Trials as Topic; Reproducibility of Results; Risk Assessment /standards; Sensitivity and Specificity
Database entry date
This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.