Table 3 Case Study 2 the QRISK2 prediction model

The QRISK2 prediction model is a widely endorsed and validated model to assess cardiovascular risk [15]. The QRISK2 model was developed using data from 531 general practices in the UK, with information from 2.3 million patients. The model was developed so as to identify those patients for whom interventions (i.e. statins) or more intensive follow-up may be required. The models are commonly used as part of directive decision-making at a cut-point of 20% predicted risk [15]. The models were reported to perform well, had C-statistics in the region of 0.80, and were subsequently validated in large cohorts [4, 5]. We report natural frequencies for this prognostic rule at the 20% cut-point, derived using information reported in the external validation cohort study for males which had a reported C-statistic of 0.77 [[4], Table 4]. Using data from this validation cohort, it is expected that out of 1000 (male) individuals between the ages of 35 and 74 years, approximately 90 will have a cardiovascular event over a 10-year period and 910 will not (i.e. a prevalence of 0.09) [4]. From data reported in Table 4 of Collins [4], we also estimated the sensitivity of the rule to be 40% and specificity to be 88%.

Figure 2 illustrates that, when used at the cut-point of 20%, the prediction rule does not do terribly well in identifying those who will have an event: the rule is correctly able to identify 36 out of the 90 who will have an event, but misclassifies 110 of the 910 individuals who will not have an event as “at-risk”. So, for every person identified as needing treatment, another 3 will be treated unnecessarily and two thirds of those in need of treatment will not be treated. Thus, despite having a C-statistic of close to 0.8, the model does not do terribly well at either ruling-in future events or ruling-out future events [32]. If the extra treatment poses no harm, which might arguably be the case for statin use, then over-treating the low risk patients might not be of concern. Nonetheless, presenting the results using the population diagram allows full implications of potential under and overtreatment to be made transparent. Table 4 presents these natural frequencies across two different assumed prevalence. For example, if the actual prevalence in the population was lower than the assumed 9%, the model would identify slightly fewer at risk; but proportionately more of those at higher risk would be identified as “at risk”