RESEARCH BMJ: first published as 10.1136/bmj.39261.471806.55 on 5 July 2007. Downloaded from 8 Bergenstal RM, Gavin III JR. The role of self-monitoring of blood practices and recommendations from the NIH Behavior Change glucose in the care of people with : report of a global Consortium. Health Psychol 2004;23:443-51. consensus conference. Am J Med 2005;118(9, suppl 1):1-6. 14 Davidson MB, Castellanos M, Kain D, Duran P. The effect of self 9 Gerich JE. Clinicians can help their patients control postprandial monitoring of blood glucose concentrations on glycated hemoglobin hyperglycemia as a means of reducing cardiovascular risk. Diabetes levels in diabetic patients not taking insulin: a blinded, randomized Educator 2006;32:513-22. trial. Am J Med 2005;118:422-5. 10 Farmer A, Wade A, French DP, Goyder E, Kinmonth AL, Neil A. The DiGEM trial protocol—a randomised controlled trial to determine the 15 Guerci B, Drouin P, Grange V, Bougneres P, Fontaine P, Kerlan V, et al. effect on glycaemic control of different strategies of blood glucose Self-monitoring of blood glucose significantly improves metabolic self-monitoring in people with type 2 diabetes [ISRCTN47464659]. control in patients with type 2 diabetes mellitus: the auto- BMC Fam Pract 2005;6:25. surveillance intervention active (ASIA) study. Diabetes Metab 11 Owens DR, Barnett AH, Pickup J, Kerr D, Bushby P, Hicks D, et al. 2003;29:587-94. Blood glucose self-monitoring in type 1 and type 2 diabetes: 16 Schwedes U, Siebolds M, Mertes G. Meal-related structured self- reaching a multi-disciplinary consensus. Diabetes Primary Care monitoring of blood glucose: effect on diabetes control in non- 2004;6:398-402. insulin-treated type 2 diabetic patients. Diabetes Care 12 American Diabetes Association. Standards of medical care in diabetes-2006. Diabetes Care 2006;29(suppl 1):S4-42. 2002;25:1928-32. 13 Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best Accepted: 13 June 2007

Derivation and validation of QRISK, a new risk score for the United Kingdom: prospective open cohort study

Julia Hippisley-Cox,1 Carol Coupland,1 Yana Vinogradova,1 John Robson,2 Margaret May,3 Peter Brindle4

EDITORIAL by Bonneux ABSTRACT was better calibrated to the UK population than either the Objective To derive a new cardiovascular disease risk Framingham or ASSIGN models. With QRISK 8.5% of 1Tower Building, University Park, score (QRISK) for the United Kingdom and to validate its patients aged 35-74 are at high risk (≥20% risk over Nottingham NG2 7RD performance against the established Framingham 10 years) compared with 13% when using the 2 Centre for Health Sciences, cardiovascular disease algorithm and a newly developed Framingham algorithm and 14% when using ASSIGN. With Queen Mary’s School of Medicine http://www.bmj.com/ and Dentistry, London Scottish score (ASSIGN). QRISK 34% of women and 73% of men aged 64-75 would 3Department of Social Medicine, Design Prospective open cohort study using routinely be at high risk compared with 24% and 86% according to University of Bristol collected data from general practice. the Framingham algorithm. UK estimates for 2005 based 4Avon Primary Care Research Setting UK practices contributing to the QRESEARCH on QRISK give 3.2 million patients aged 35-74 at high risk, Collaborative, Bristol Primary Care database. with the Framingham algorithm predicting 4.7 million and Trust Participants The derivation cohort consisted of 1.28 ASSIGN 5.1 million. Overall, 53 668 patients in the Correspondence to: J Hippisley-Cox julia.hippisley-cox@nottingham. million patients, aged 35-74 years, registered at 318 validation dataset (9% of the total) would be reclassified ac.uk practices between 1 January 1995 and 1 April 2007 and from high to low risk or vice versa using QRISK compared on 30 September 2021 by guest. Protected copyright. who were free of diabetes and existing cardiovascular BMJ 2007;335:136-41 with the Framingham algorithm. doi:10.1136/bmj.39261.471806.55 disease. The validation cohort consisted of 0.61 million Conclusion QRISK performed at least as well as the patients from 160 practices. Framingham model for discrimination and was better Main outcome measures First recorded diagnosis of calibrated to the UK population than either the cardiovascular disease (incident diagnosis between 1 Framingham model or ASSIGN. QRISK is likely to provide January 1995 and 1 April 2007): myocardial infarction, more appropriate risk estimates to help identify high risk coronary heart disease, stroke, and transient ischaemic patients on the basis of age, sex, and social deprivation. It attacks. Risk factors were age, sex, smoking status, is therefore likely to be a more equitable tool to inform systolic blood pressure, ratio of total serum to management decisions and help ensure treatments are high density lipoprotein, body mass index, family history directed towards those most likely to benefit. It includes of coronary heart disease in first degree relative aged less additional variables which improve risk estimates for than 60, area measure of deprivation, and existing patients with a positive family history or those on treatment with antihypertensive agent. antihypertensive treatment. However, since the validation Results A cardiovascular disease risk algorithm (QRISK) was performed in a similar population to the population was developed in the derivation cohort. In the validation from which the algorithm was derived, it potentially has a cohort the observed 10 year risk of a cardiovascular event “home advantage.” Further validation in other populations was 6.60% (95% confidence interval 6.48% to 6.72%) in This article is an abridged version is therefore required. women and 9.28% (9.14% to 9.43%) in men. Overall the of a paper that was published on bmj.com on 5 July 2007. Cite this Framingham algorithm over-predicted cardiovascular version as: BMJ 5 July 2007, doi: disease risk at 10 years by 35%, ASSIGN by 36%, and INTRODUCTION 10.1136/bmj.39261.471806.55 (abridged text, in print: BMJ QRISK by 0.4%. Measures of discrimination tended to be Risk equations for cardiovascular disease derived from 2007;335:136-41). higher for QRISK than for the Framingham algorithm and it the American Framingham cohort study are the most

136 BMJ | 21 JULY 2007 | VOLUME 335 RESEARCH BMJ: first published as 10.1136/bmj.39261.471806.55 on 5 July 2007. Downloaded from widely used in the United Kingdom1 but have major of death data from the Office for National Statistics limitations (see bmj.com). For example, the algorithm enabling sensitivity analyses including certified cause does not include factors such as social deprivation. of death. The UK National Institute for Health and Clinical Excellence has lowered the threshold for primary pre- Practices and cohorts vention with from a 10 year cardiovascular dis- We included contributing practices once they had 23 ease risk of 40% to 20%. We refer to a 10 year risk of been using EMIS for at least a year. We randomly allo- 20% or more as high. It is important that this pro- cated two thirds to the derivation dataset (used to gramme includes social deprivation to reduce inequal- derive an equation for estimating cardiovascular risk) ities in cardiovascular disease and to target those at and one third to the validation dataset. greatest risk.4 Scotland has adopted ASSIGN equa- We identified an open cohort of patients aged 35-74, tions that include social deprivation.5 registered with eligible practices during the study per- We derived and validated a new cardiovascular dis- iod 1 January 1995 to 1 April 2007 (see bmj.com for ease risk score for the UK population and compared exclusions). this with the Framingham1 and ASSIGN equations.6 For each patient we determined an entry date to the METHODS analysis, which was the latest of the following: 35th Our prospective cohort study was carried out within a birthday, date of registration with practice, date when large UK primary care population. We used version 14 practice computer system was installed, and start of of QRESEARCH, a validated database containing the study period. We also included patients only when ’ records of 10 million patients from 529 general prac- they had at least one year s complete data in their med- tices using the EMIS computer system. It also contains ical record. area measures of ethnicity and deprivation evaluated at For each patient we determined the right censor date output area on the basis of the 2001 census. The Town- —the earliest date of the dates on which they devel- send score7 is an area measure of material deprivation. oped the outcome of interest, the study ended, date of Most recently QRESEARCH has been linked to cause death, date of deregistration, or date of last upload of

Table 1 |Ratio of predicted to observed risks of cardiovascular disease event at 10 years across tenths of estimated risks

QRISK Framingham ASSIGN http://www.bmj.com/ Predicted Observed Predicted Observed Predicted Observed Tenth* risk (%) risk (%) Ratio risk (%) risk (%) Ratio risk (%) risk (%) Ratio Women: First 0.71 0.44 1.61 0.59 0.53 1.12 1.31 0.34 3.81 Second 1.06 0.95 1.12 1.17 0.96 1.22 1.83 0.91 2.00 Third 1.49 1.45 1.03 1.95 1.59 1.22 2.41 1.49 1.61 Fourth 2.11 2.11 1 3.02 2.19 1.38 3.18 2.14 1.48 on 30 September 2021 by guest. Protected copyright. Fifth 2.97 3.06 0.97 4.41 3.02 1.46 4.23 3.08 1.37 Sixth 4.22 4.5 0.94 6.15 4.63 1.33 5.76 4.17 1.38 Seventh 6.06 6.16 0.99 8.21 6.97 1.18 8.05 6.60 1.22 Eighth 8.9 9.65 0.92 10.76 10.29 1.05 11.61 9.79 1.19 Ninth 13.36 13.95 0.96 14.15 14.63 0.97 17.28 13.86 1.25 Tenth 22.52 20.19 1.12 22.47 16.86 1.33 29.29 19.35 1.51 Overall 6.34 6.25 1.02 7.28 6.16 1.18 8.49 6.17 1.38 Men: First 1.64 0.93 1.77 2.86 1.03 2.79 3.37 0.92 3.65 Second 2.24 1.6 1.39 4.19 1.73 2.43 4.36 1.63 2.68 Third 2.92 2.6 1.12 5.54 2.53 2.19 5.31 2.56 2.07 Fourth 3.81 3.86 0.99 7.04 3.71 1.90 6.43 3.39 1.90 Fifth 5 4.86 1.03 8.86 5.43 1.63 7.83 5.22 1.50 Sixth 6.61 6.67 0.99 11.13 7.38 1.51 9.65 7.16 1.35 Seventh 8.88 9.88 0.9 14.00 10.11 1.38 12.16 10.19 1.19 Eighth 12.24 14.24 0.86 17.78 13.90 1.28 15.76 13.97 1.13 Ninth 17.4 18.46 0.94 23.19 19.05 1.22 21.30 18.79 1.13 Tenth 27.86 25.72 1.08 35.24 23.42 1.51 33.34 24.49 1.36 Overall 8.86 8.88 1.00 12.96 8.81 1.47 11.95 8.83 1.35 All patients 7.59 7.56 1.004 10.10 7.48 1.35 10.21 7.49 1.36 Comparison between QRISK, Framingham, and ASSIGN in validation cohort in patients aged 35-74. *First tenth is lowest tenth of risk.

BMJ | 21 JULY 2007 | VOLUME 335 137 RESEARCH BMJ: first published as 10.1136/bmj.39261.471806.55 on 5 July 2007. Downloaded from data. We determined the person years at risk and used variables and the outcome. We used fractional poly- this as the denominator term for incidence rates. nomials to model non-linear risk relations with contin- uous variables when appropriate.89 We tested for Cardiovascular disease outcomes and risk factors interactions between systolic blood pressure and anti- The primary outcome was the first recorded diagnosis hypertensive treatment and between smoking and of cardiovascular disease on the general practice’s deprivation. computer system before or at death during the study Initially our models were fitted using patients with (see bmj.com for definitions). complete data, and the fractional polynomial terms We used the doctor’s diagnosis of cardiovascular were obtained from the complete case analysis using disease for our main analysis. Patient level data for a multivariable approach. However, because such cause of death became available from the Office for patients might have a health status and risk different National Statistics for 2002-7 and we linked these from those with missing data, we fitted our principal within the general practices’ computer systems and models on the basis of multiple imputed datasets extracted the data on to QRESEARCH. We therefore using Rubin’s rules.910 identified patients who had a primary or underlying We took the log of the hazard ratio for each risk fac- certified cause of death as coronary heart disease, tor from the final model and used this as weight for the stroke, or transient ischaemic attack. equation. We then estimated each patient’s probability The risk factors we included were age, sex, smoking of experiencing a cardiovascular event within 10 years status, systolic blood pressure, ratio of total serum cho- by combining these weights with the patient’s charac- lesterol to high density lipoprotein levels, left ventricu- teristics and using the baseline survivor function (see lar hypertrophy, body mass index, family history of bmj.com) for all participants.11 cardiovascular disease in a first degree relative aged less than 60, Townsend deprivation score, percentage Validation of the new cardiovascular disease risk equation of South Asian residents at output areas, and current We tested the performance of the risk score in the vali- prescription of at least one antihypertensive (see dation dataset. We calculated the 10 year estimated bmj.com). cardiovascular disease risk for each patient in the vali- dation dataset, replacing missing values for continuous Model derivation and development variables with mean values obtained from the deriva- We used the Cox proportional hazards model in the tion dataset by five year age-sex bands and assuming derivation dataset to estimate the coefficients asso- patients were non-smokers when smoking status was

ciated with each potential risk factor for the first not recorded. http://www.bmj.com/ recorded diagnosis of cardiovascular disease for either To assess calibration we calculated the mean pre- sex. We specified the variables to include on the basis dicted and observed risks of cardiovascular disease at of traditional risk scores. We compared models using 10 years obtained using the Kaplan-Meier estimate. the Bayes information criterion, where lower values We then compared the ratio of the predicted to indicate better fit. We examined the strength of the observed risk for patients in the validation cohort in association between one unit increases in each contin- each tenth of predicted risk. We also compared pre-

uous variable and compared categories for other vari- dicted and observed risks overall for men and women on 30 September 2021 by guest. Protected copyright. ables. We checked the assumptions of the proportional by age band and fifth of Townsend score. We calcu- hazards model for each variable and tested for any non- lated the area under the receiver operating curve to linear relation between continuous independent assess discrimination. We also calculated the D statistic12 and an R2 statistic,13 measures of discrimina- tion and explained variation for survival models. Table 2 |Summary statistics comparing QRISK equation for 10 year risk of cardiovascular disease to predictions based on Framingham and ASSIGN equations applied to validation cohort in Comparison with the Framingham and ASSIGN equations patients aged 35-74 We compared the performance of QRISK against the Summary statistics QRISK Framingham ASSIGN risk estimates derived from Framingham equations.1 Women: We used an equation for cardiovascular disease risk Receiver operating curve 0.7879 0.7744 0.7841 that was computed by summing coronary and stroke statistic* risks. D statistic* (SE) 1.549 (0.014) 1.393 (0.014) 1.472 (0.014) We calculated the area under the receiver operating 2 R statistic† (SE) 36.4%(0.43) 31.7%(0.44) 34.1% (0.43) curve and the D statistic for the Framingham estimates Men: in the validation dataset and compared them with the Receiver operating curve 0.7674 0.7598 0.7644 corresponding QRISK values. We also compared the statistic* performance of QRISK with the ASSIGN equation.6 D statistic* (SE) 1.447 (0.013) 1.31 (0.012) 1.357 (0.012) Finally, we calculated the proportion of patients in R2 statistic† (SE) 33.3% (0.39) 29.1% (0.38%) 30.5% (0.38) the validation sample who have a 20% or more risk of Although QRISK validation sample was drawn from independent practices, all use same clinical computer system (EMIS). Therefore score needs to be validated in practices using other clinical systems. cardiovascular disease at 10 years, by age, sex, and *Higher scores indicate better discrimination. deprivation according to QRISK and the Framingham †Indicates percentage of variation explained. and ASSIGN equations. We determined the

138 BMJ | 21 JULY 2007 | VOLUME 335 RESEARCH BMJ: first published as 10.1136/bmj.39261.471806.55 on 5 July 2007. Downloaded from proportion of patients who would be reclassified into a cholesterol to high density lipoprotein levels, systolic higher or lower risk category using the new equations. blood pressure, body mass index, family history of pre- We estimated the number of people in the United mature cardiovascular disease, smoking status, Town- Kingdom with a 10 year cardiovascular disease risk send deprivation score, and use of at least one blood of 20% or more for each score, using population esti- pressure treatment. The final model also included an mates for 2005. interaction term between systolic blood pressure and blood pressure treatment. Left ventricular hyper- RESULTS trophy and the area measure of ethnicity were omitted Overall, 478 UK practices were included: 318 were from the final model (see bmj.com). randomised to the derivation dataset and 160 to the Hazard ratios for the final model show that the risk of validation dataset. a cardiovascular disease event increased with increas- The derivation cohort contained 3 374 617 patients. ing age, body mass index, and Townsend deprivation Of these, 1 283 174 were aged 35-74 years (50.4% were score. The risk was higher in patients who smoked, had women) and were free of cardiovascular disease and a family history of premature cardiovascular disease, diabetes at baseline. They were included in the study. and were receiving blood pressure treatment at base- During the study 65 671 incident cases of cardio- line. Risk increased more steeply with systolic blood vascular disease from 8.2 million person years of pressure in those not receiving treatment for blood observation occurred (crude incidence rate 7.96 per pressure. The hazard ratio associated with the ratio 1000 person years). Incidence rates were higher in for total serum cholesterol to high density lipoprotein men, increasing steeply with age (see bmj.com). The level was just above and close to one for both sexes. computer recorded incidence of cardiovascular dis- ease was also significantly higher among people from Calibration and discrimination of QRISK v Framingham v deprived areas, the effect being more noticeable for ASSIGN women. Overall, 306 259 patients were followed up The Framingham equation over-predicted risk at for at least 10 years. The 10 year observed risk of a 10 years by 35%, ASSIGN by 36%, and QRISK by cardiovascular event in women aged 35-74 was 0.4% (table 1). The equations tended to over-predict 6.69% (95% confidence interval 6.61% to 6.78%) and risk in the lowest three tenths for both sexes. The in men was 9.46% (9.36% to 9.56%). most noticeable over-prediction was with ASSIGN, In the validation dataset 614 553 patients were aged followed by the Framingham equation and then 35-74 (50.3% were women). Incidence rates were simi- QRISK. The Framingham equation, however, consis-

lar to those in the derivation cohort (see bmj.com). The tently over-predicted risk in men across every tenth, http://www.bmj.com/ 10 year observed risk of a cardiovascular event in with a 51% over-prediction in the top tenth compared women aged 35-74 was 6.60% (6.48% to 6.72%) and with 36% for ASSIGN and 8% for QRISK. in men was 9.28% (9.14% to 9.43%). The receiver operating curve statistic indicates that the final QRISK score has at least as good as, if not Outcome comparison with data from the Office for National slightly better, discrimination than the Framingham Statistics and ASSIGN equations (table 2). 2 A subgroup analysis was undertaken for the period The R statistic (standard error) for QRISK in on 30 September 2021 by guest. Protected copyright. 2002-7 for practices with linked data on death to deter- women was 36.4% (0.43%), higher than the 31.7% mine the degree of under-ascertainment of deaths from (0.44%) with the Framingham equation. Similarly, the cardiovascular disease on the general practice’s com- R2 statistic for the model for men was higher for puter record (see bmj.com). QRISK than for the Framingham equation. The D sta- tistic for QRISK was 1.45 for men and 1.55 for women, Baseline characteristics of derivation cohort both higher than those for the Framingham equations Values by age for most variables were similar to those (1.31 and 1.39), indicating better discrimination for in the health survey for England, suggesting that the QRISK. cohort is representative of the general population (see figure on bmj.com). The exception was smoking: sur- Predictions with age, sex, and social deprivation vey rates were slightly higher for women but lower for See bmj.com for the proportion of patients with a men. In general patients with missing data had signifi- cardiovascular disease risk score of ≥20% (high risk) cantly different survival rates (see bmj.com). by Townsend fifths and sex according to each of the three risk prediction scores. Model derivation QRISK predicts 9.9% of women aged 35-74 from the See bmj.com for the results of the Cox regression ana- most deprived fifth to be at high risk compared with lysis for the final model. A log transformation was used 3.0% of women from the most affluent fifth. For the for age but otherwise variables were fitted as linear Framingham equation the values are 6.3% (most terms as this best fitted the data according to the frac- deprived) and 4.6% (most affluent). tional polynomial analysis. Three main models were QRISK predicts 12.6% of men from the most compared for fit and interpretability. The final model deprived areas to be at high risk compared with 9.6% included the logarithm of age, ratio of serum from the most affluent areas. For the Framingham

BMJ | 21 JULY 2007 | VOLUME 335 139 RESEARCH BMJ: first published as 10.1136/bmj.39261.471806.55 on 5 July 2007. Downloaded from equation the values were 19.5% (most deprived) and compared with the Framingham and ASSIGN cohorts. 20.5% (most affluent). This could introduce some imprecision in survival esti- Overall QRISK predicted 8.5% of patients aged 35- mates. We therefore restricted the analysis to the 74 to be at high risk compared with 12.8% for the Fra- closed cohort and found minimal difference in the mingham equation and 14.0% for ASSIGN (see baseline survivor function. The cohort is socially, eth- bmj.com). With QRISK 34.5% of women and 72.9% nically, and geographically diverse and over 250 times of men aged 64-75 would be at high risk compared with the size of the Framingham cohorts. 24.1% and 86.0% using the Framingham equation. Although levels of completeness were good for some Estimates based on QRISK give 3.2 million patients risk factors, levels were low for serum cholesterol. This aged 35-74 at high risk in the United Kingdom for is a likely source of bias and may have contributed to 2005, with the Framingham equation predicting 4.7 the attenuation of the hazard ratio of the cholesterol million and ASSIGN 5.1 million. ratio term towards one. However, comparison with Overall, 53 668 patients in the validation dataset (9% the mean values for systolic blood pressure, choles- of total) would be reclassified from high to low risk or terol, and body mass index from the health survey for vice versa using QRISK compared with using the Fra- England shows that our cohort is likely to be represen- mingham equation. Of these, 13 494 patients were clas- tative of the population in England. The exception was sified as low risk using the Framingham equation but smoking, where the survey rates were higher for high risk using QRISK, and their observed 10 year risk women but lower for men. A strength of our study is was 22.32% (95% confidence interval 21.36% to the multiple imputation of missing values (see 23.32%). Conversely, of those 40 174 at high risk bmj.com), designed to increase statistical power and using the Framingham equation but at low risk using produce models that are more reliable and applicable QRISK, the observed 10 year risk was 16.45% (15.98% within clinical practice.910 We used a single baseline to 16.93%). measure for systolic blood pressure, which might lead to underestimation of its association with cardio- DISCUSSION vascular disease risk. Also we included a crude mea- We derived and validated a new cardiovascular disease sure of blood pressure treatment. risk equation (QRISK) in the United Kingdom. The The outcome measures in our study were not for- study was validated on an independent sample of prac- mally adjudicated. Validation studies of similar data- tices and performs as well if not better than the Fram- bases show that false positive diagnoses occur in fewer ingham algorithm in the UK population. However, than 10% of cases.18 Similarly, the information for since the validation was carried out in a similar popula- some patients with cardiovascular disease may not be http://www.bmj.com/ tion to the population from which the algorithm was present on clinical records. Such misclassification of derived, it potentially has a “home advantage.” outcomes results in hazard ratios tending towards Inclusion of antihypertensive treatment at baseline is one, which would reduce the discriminatory power of of relevance to 1 in 10 people and failure to account for a risk score. Our subgroup analysis for 2002-7, how- this would lead to a significant underestimation of risk ever, showed a 94% ascertainment rate for cardio- in the population. vascular disease outcomes based on the general Unlike the Framingham equations,14 QRISK practice clinical record alone compared with outcomes includes deprivation. This is likely to be an improve- including death certification data. on 30 September 2021 by guest. Protected copyright. ment as the Framingham algorithm tends to overesti- Under-ascertainment is more likely to affect patients mate risk in affluent areas and underestimate risk in from the most deprived areas and so it is possible that deprived areas.15 The inclusion of a family history of our algorithm will underestimate risk in these patients. premature coronary heart disease is important as stu- Lastly, although we have validated QRISK on a sam- dies indicate that such disease in a first degree relative ple of one third of practices from QRESEARCH, it increases risk by 50% or more.16 17 Further work is could be argued that this is not an entirely independent needed because the inclusion of area measures of eth- population because the practices were using EMIS. nicity in QRISK did not improve the utility of the This potentially gives a “home advantage” to QRISK score, possibly because of the correlation with the as prediction algorithms tend to perform better in area based measure of deprivation. populations similar to the ones in which they were Although QRISK includes more variables than the derived. A second validation is therefore underway Framingham algorithm, it has been derived from rou- using the THIN clinical database. tinely collected data from general practice. Not only is it likely to have a face validity but software can be Comparison with Framingham and ASSIGN algorithms developed for use in general practice. The Framingham equation was based on rates of This study is based on a large, contemporary UK cardiovascular disease at the height of an epidemic population with data from a validated database. We but is now used as a tool to describe risks in contem- used an open cohort design so that patients with more porary European populations. It has repeatedly been recent data could be included. Although not all shown to be inaccurate. QRISK explains a higher per- patients contributed data for 10 years, over 300 000 centage of the variation than the Framingham or patients had follow-up data for at least 10 years. The ASSIGN algorithms in both sexes. Discrimination median follow-up time was therefore relatively short was better with QRISK than with the Framingham

140 BMJ | 21 JULY 2007 | VOLUME 335 RESEARCH BMJ: first published as 10.1136/bmj.39261.471806.55 on 5 July 2007. Downloaded from WHAT IS ALREADY KNOWN ON THIS TOPIC data with general practice data—Nirupa Dattani and Sir Peter Goldblatt (Office for National Statistics), Steve Daniels (Connecting for Health), John Fox and Current cardiovascular risk prediction algorithms based on Dave Roberts (Information Centre), Andy Whitwam and David Stables (EMIS), Bill Kirkup, Maggie Rae, Michael Soljak, and Roger Boyle (Department of the Framingham model tend to over-predict risk and are not Health). well calibrated for the UK population

The Framingham model does not include measures such as Contributors: See bmj.com. deprivation Funding: The study received no external funding. WHAT THIS STUDY ADDS Competing interests: JR chairs, and PB is a member of, the NICE guideline development group on cardiovascular risk assessment “The modification of A new risk score, QRISK, performed at least as well as the blood lipids for the primary and secondary prevention of cardiovascular ” Framingham model for discrimination and was better disease. JHC is codirector of QRESEARCH, a not for profit organisation that is a joint partnership between the University of Nottingham and EMIS (leading calibrated to the UK population than either the Framingham supplier of information technology for 60% of UK general practices). model or the Scottish risk equation ASSIGN QRESEARCH undertakes analyses for the Department of Health and other QRISK would identify a different high risk group of patients government organisations. All research using the database is peer reviewed than the Framingham equation, with 1 in 10 patients being and published. QRESEARCH is not used for studies for the drug industry. reclassified into high or low risk Ethical approval: This study was approved by the Trent multicentre research ethics committee. QRISK includes additional variables that allow better tailoring of management to the individual patient and will help to minimise health inequalities 1 Anderson KM, Odell PM, Wilson PWF, Kannel WB. Cardiovascular disease risk profiles. Am Heart J 1991;121:293-8. 2 North of England Hypertension Guideline Development Group. Essential hypertension: managing adult patients in primary care equation. QRISK tends to increase cardiovascular dis- (NICE guideline). University of Newcastle upon Tyne: Centre for Health Services Research, 2004. ease risk estimates in people living in deprived areas 3 National Institute for Health and Clinical Excellence. Statins for the relative to those in more affluent areas. This effect is prevention of cardiovascular events in patients at increased risk of developing cardiovascular disease or those with established most noticeable for women from deprived areas. Inclu- cardiovascular disease guidance type: Technology appraisal. In: sion of area based deprivation measures within the Excellence NIfHaC, ed, 2006. algorithm is likely to predict more accurately which 4 White C, Galen FV, Chow YH. Trends in social class differences in mortalitybycause.Health Stats Q 2003;20:25-36. patients are at high or low risk and so target inter- 5 Scottish Intercollegiate Guidelines Network. Risk estimation and the ventions more appropriately. prevention of cardiovascular disease. A national clinical guideline. Edinburgh: SIGN, 2007. Neither the Framingham nor ASSIGN equation is 6 Woodward M, Brindle P, Tunstall-Pedoe H. Adding social deprivation well calibrated for the UK population. The degree of and family history to the cardiovascular risk assessment: the ASSIGN over-prediction for the Framingham algorithm is con- score from the Scottish Heart Health Extended Cohort (SHHEC). Heart 2007;2:172-6. http://www.bmj.com/ sistent with other validation studies in the United 7 Townsend P, Davidson N. The Black report. London: Penguin, 1982. Kingdom.19 We think that over-prediction probably 8 Royston P, Ambler G, Sauerbrei W. The use of fractional polynomials to model continuous risk variables in epidemiology. Int J Epidemiol results from a combination of differences in risk factor 1999;28:964-74. means and prevalences and baseline survival function. 9 Clark T, Altman D. Developing a prognostic model in the presence of ASSIGN, for example, is calibrated to the incidence of missing data: an ovarian cancer case study. JClinEpidemiol 2002;56:28-37. cardiovascular disease in Scotland, which is higher 10 Royston P. Multiple imputation of missing values. Stata J than that in England. Although much less noticeable, 2004;4:227-41. 11 Wilson P, D’Agostino R, Levy D, Belanger A, Silbershatz H, Kannel W. on 30 September 2021 by guest. Protected copyright. some over-prediction occurred in the top tenth using Prediction of coronary heart disease using risk factor categories. QRISK, which might reflect an effect of increasing age Circulation 1998;97:1837-47. when the effect of risk factors diminish and survivors 12 Royston P, Sauerbrei W. A new measure of prognostic separation in survival data. Stats Med 2004;23:723-48. are more resilient. Alternatively it could be due to com- 13 Royston P. Explained variation for survival models. Stata J peting risks of death, such as from cancer, which 2006;6:1-14. become more important at older ages. 14 De Becker G, Ambrosioni E, Borch-Johnsen K, Brotons C, Cifkova R, Dallongeville J, et al. European guidelines on cardiovascular disease prevention in clinical practice. Eur Heart J 2003;24:1601-10. Conclusion 15 Brindle P, McConnachie A, Upton M, Hart C, Davey-Smith G, Watt G. The accuracy of the Framingham risk-score in different We have derived and validated a new cardiovascular socioeconomic groups: a prospective study. Br J Gen Pract disease risk prediction score that is better calibrated for 2005;55:838-45. the UK population and has better discrimination. 16 Lloyd Jones X, Lloyd-Jones D, Nam B, D’Agostino RS, Levy D, Murabito J, et al. Parental cardiovascular disease as a risk factor for Overall in patients aged 35-74, the Framingham algo- cardiovascular disease in middle-aged adults: a prospective study of rithm over-predicted cardiovascular disease risk at parents and offspring. JAMA 2004;291:2204-11. 17 Nilsson P, Nilsson J, Berglund G. Family burden of cardiovascular 10 years by 35%, ASSIGN by 36%, and QRISK by mortality: risk implications for offspring in a national register linkage 0.4%. Overall, QRISK would reclassify about 1 in 10 study based upon the Malmo Preventive Project. JInternMed patients from high to low risk or vice versa compared 2004;255:229-35. 18 Jick H, Derby L, Gurewich V, Vasilakis C. The risk of myocardial with the Framingham algorithm. infarction associated with treatment in persons with uncomplicated essential hypertension. We thank EMIS and the EMIS practices contributing to QRESEARCH; Patrick Pharmacotherapy 1996;16:321-6. Royston for advice on multiple imputation, fractional polynomials, and the D 19 Brindle P, Emberson J, Lampe F, Walker M, Whincup P, Fahey T, et al. Predictive accuracy of the Framingham coronary risk score in British statistic; Rod Jackson, Stuart Pocock, Philip Bath, Hugh Tunstall-Pedoe, and men: prospective cohort study. BMJ 2003;327:1267-73. Mark Woodward for reviewing the protocol; and the following who enabled the project, contributed to discussions, or facilitated the linkage of cause of deaths Accepted: 27 June 2007

BMJ | 21 JULY 2007 | VOLUME 335 141