Skip to main content

Table 3 Performance of the CAD models based on the determined sample size for various fixed and adaptive sample size methods. Results are shown as medians (with interquartile ranges) across 500 repetitions

From: Adaptive sample size determination for the development of clinical prediction models

 

Sample size

Bootstrap-corrected performance

Sample size method

N

EPP

AUC

Slope

Basic strategy

 Fixeda: 10 EPP

300 (250–300)

11 (10–11)

0.706 (0.684–0.726)

0.774 (0.751–0.795)

 Fixeda: Riley’s method

700 (700–700)

28 (27–28)

0.713 (0.701–0.726)

0.894 (0.886–0.901)

 Adaptive: stopping rule 1b

850 (750–900)

33 (30–35)

0.717 (0.705–0.727)

0.909 (0.905–0.914)

 Adaptive: stopping rule 2b

1500 (1400–1550)

59 (56–62)

0.719 (0.712–0.725)

0.949 (0.946–0.952)

Restricted cubic splines

 Fixeda: 10 EPP

350 (350–400)

11 (10–11)

0.703 (0.684–0.721)

0.766 (0.745–0.783)

 Fixeda: Riley’s method

900 (900–900)

26 (26–27)

0.712 (0701–0.724)

0.887 (0.877–0.894)

 Adaptive: stopping rule 1b

1100 (1050–1200)

31 (28–33)

0.715 (0.705–0.724)

0.907 (0.904–0.911)

 Adaptive: stopping rule 2b

1850 (1800–1900)

58 (55–60)

0.718 (0.712–0.724)

0.947 (0.945–0.949)

Firth’s correction

 Fixeda: 10 EPP

300 (250–300)

11 (10–11)

0.706 (0.682–0.726)

0.822 (0.797–0.846)

 Fixeda: Riley’s method

700 (700–700)

28 (27–28)

0.712 (0.700–0.726)

0.913 (0.904–0.923)

 Adaptive: stopping rule 1b

750 (700–800)

31 (28–32)

0.713 (0.703–0.727)

0.922 (0.917–0.928)

 Adaptive: stopping rule 2b

1500 (1450–1600)

59 (56–63)

0.718 (0.710–0.726)

0.958 (0.956–0.961)

Including backward selection

 Fixeda: 10 EPP

300 (250–300)

11 (10–11)

0.690 (0.666–0.715)

0.798 (0.772–0.819)

 Fixeda: Riley’s method

700 (700–700)

28 (27–28)

0.708 (0.694–0.722)

0.894 (0.883–0.905)

 Adaptive: stopping rule 1b

800 (750–900)

33 (29–36)

0.712 (0.701–0.723)

0.909 (0.905–0.915)

 Adaptive: stopping rule 2b

1550 (1400–1650)

61 (56–65)

0.716 (0.709–0.724)

0.948 (0.946–0.951)

  1. AUC area under the receiver operating characteristic curve (or c-statistic), slope calibration slope, EPP events per parameter
  2. aThe analysis went in batches of 50 patients, therefore fixed sample sizes were rounded upwards to the next multiple of 50
  3. bStopping rule 1: calibration slope ≥ 0.9 and AUC optimism < = 0.02 at two consecutive assessments. Stopping rule 2: calibration slope ≥ 0.9 and AUC optimism < = 0.01 at two consecutive assessments