Skip to main content

Table 2 Performance of the ovarian cancer models based on the determined sample size for various fixed and adaptive sample size methods. Results are shown as medians (with interquartile ranges) across 500 repetitions

From: Adaptive sample size determination for the development of clinical prediction models

  Sample size Bootstrap-corrected performance
Sample size method N EPP AUC Slope
Basic strategy
 Fixeda: 10 EPP 250 (250–250) 11 (11–12) 0.914 (0.901–0.926) 0.813 (0.776–0.845)
 Fixeda: Riley’s method 350 (350–350) 16 (16–17) 0.916 (0.904–0.927) 0.884 (0.860–0.899)
 Adaptive: stopping rule 1b 450 (450–500) 22 (20–24) 0.916 (0.907–0.926) 0.921 (0.914–0.930)
 Adaptive: stopping rule 2b 500 (450–550) 23 (21–24) 0.918 (0.908–0.927) 0.924 (0.916–0.933)
Restricted cubic splines
 Fixeda: 10 EPP 350 (300–350) 11 (10–11) 0.925 (0.915–0.935) 0.840 (0.813–0.859)
 Fixeda: Riley’s method 450 (450–450) 15 (14–15) 0.926 (0.917–0.935) 0.893 (0.878–0.903)
 Adaptive: stopping rule 1b 550 (500–600) 18 (17–19) 0.928 (0.919–0.935) 0.917 (0.900–0.945)
 Adaptive: stopping rule 2b 600 (550–600) 19 (18–20) 0.928 (0.920–0.935) 0.921 (0.914–0.927)
Firth’s correction
 Fixeda: 10 EPP 250 (200–250) 11 (10–12) 0.914 (0.897–0.929) 0.944 (0.927–0.959)
 Fixeda: Riley’s method 350 (350–350) 16 (16–17) 0.915 (0.903–0.927) 0.958 (0.949–0.968)
 Adaptive: stopping rule 1b 250 (200–250) 11 (10–12) 0.916 (0.901–0.930) 0.947 (0.933–0.964)
 Adaptive: stopping rule 2b 400 (350–450) 18 (17–21) 0.916 (0.906–0.928) 0.964 (0.956–0.973)
Including backward selection
 Fixeda: 10 EPP 250 (250–250) 11 (11–12) 0.909 (0.894–0.925) 0.892 (0.875–0.907)
 Fixeda: Riley’s method 350 (350–350) 16 (16–17) 0.913 (0.903–0.925) 0.907 (0.904–0.928)
 Adaptive: stopping rule 1b 350 (300–400) 16 (13–18) 0.913 (0.901–0.926) 0.918 (0.910–0.926)
 Adaptive: stopping rule 2b 400 (350–450) 18 (15–21) 0.915 (0.905–0.927) 0.926 (0.918–0.935)
  1. AUC area under the receiver operating characteristic curve (or c-statistic), slope calibration slope, EPP events per parameter
  2. aThe analysis went in batches of 50 patients, therefore fixed sample sizes were rounded upwards to the next multiple of 50
  3. bStopping rule 1: calibration slope ≥ 0.9 and AUC optimism < = 0.02 at two consecutive assessments. Stopping rule 2: calibration slope ≥ 0.9 and AUC optimism < = 0.01 at two consecutive assessments