Skip to main content

Table 1 Case Study 1 the Canadian Syncope Risk Score

From: Knowledge translation of prediction rules: methods to help health professionals understand their trade-offs

The Canadian Syncope Risk Score (CSRS) was developed to help identify patients presenting to the emergency department with syncope who are at risk of developing a serious adverse event, which typically occurs with a prevalence of about 4% [30]. The model was proposed as a risk stratification tool with cut-points signalling very low, low, high and very high risk. The reported internally validated C-statistic for the developed model was 0.88 (95%CI 0.85, 0.90), with sensitivity of 93% and specificity of 53% for the cut-point “low risk”. The rule was summarised by the statement “the tool will be able to accurately stratify the risk of serious adverse events among patients presenting with syncope, including those at low risk who can be discharged home quickly”. From the reported sensitivity (93%), specificity (53%) and prevalence (0.036) in the development dataset [30] we estimate the natural frequencies at several cut-points and present a population diagram at one of several reported cut-points for illustration.

Figure 1 illustrates how the use of population diagrams can help quantify the implications of using this model at the cut-point “low risk”. This figure illustrates that for this cut-point, whilst of the 540 patients identified as “low risk” by the model only two have a serious adverse event, for every 1000 patients assessed by the model, 460 will be classified as “at risk” of whom only 36 will have a serious adverse event. Therefore, the model used at this cut-point is reasonably able to rule out a serious adverse event, but at the cost of a large proportion of patients undergoing monitoring (i.e. not good at ruling in). Possible consequences of this misclassification are longer stays in hospital for those classified at risk; and a small proportion of patients classified as “low risk” progressing to have a serious adverse event out of hospital. Whilst these might be appropriate trade-offs, they are not obvious when summarising the performance by a C-statistic and sensitivity alone, but become transparent when showing population diagrams. Table 2 presents these natural frequencies across a range of cut-points. For example, if there was a concern that the rule was misclassifying too many people as “at risk” when they would not have the event, increasing the cut-point to 3 for example, would reduce the number classified as “at risk” from 460 to 119; but would increase the number classified as “not at risk” who would have the event from 2 to 12.