Comparison of dynamic updating strategies for clinical prediction models

Table 2 Brier score (BS), AUC, Hosmer-Lemeshow statistic, calibration intercept and slope (from logistic calibration), and number of updates performed under each update strategy when the sliding window equals 1, 2, 4, or 8 quarters and the update interval equals 1 quarter

Sliding window	Metric	1 quarter new	2 quarters (1 q new + 1 q old)	4 quarters (1 q new + 3 q old)	8 quarters (1 q new + 7 q old)
Never update	BS	0.118	0.118	0.118	0.118
	AUC	0.603	0.603	0.603	0.603
	H-L	290.5	290.5	290.5	290.5
	Calibration intercept	−0.754	−0.754	−0.754	−0.754
	Calibration slope	0.791	0.791	0.791	0.791
	# Updates	0	0	0	0
Closed testing procedure	BS	0.116	0.114	0.114	0.114
	AUC	0.575	0.601	0.593	0.600
	H-L	118.4	36.95	34.75	42.47
	Calibration intercept	−1.658	−0.503	−0.829	−0.755
	Calibration slope	0.118	0.766	0.572	0.635
	# Updates	12	8	8	6
Always recalibrate intercept	BS	0.113	0.113	0.113	0.113
	AUC	0.605	0.605	0.607	0.608
	H-L	29.33	33.04	25.56	31.61
	Calibration intercept	−0.472	−0.454	−0.414	−0.437
	Calibration slope	0.758	0.773	0.802	0.803
	# Updates	24	24	24	24
Always recalibrate intercept and slope	BS	0.113	0.113	0.113	0.113
	AUC	0.601	0.602	0.606	0.607
	H-L	18.53	20.89	18.73	22.15
	Calibration intercept	−0.451	−0.317	−0.218	−0.240
	Calibration slope	0.771	0.852	0.914	0.917
	# Updates	24	24	24	24
Always refit (revision)	BS	0.118	0.115	0.114	0.113
	AUC	0.570	0.588	0.594	0.602
	H-L	357.8	81.77	43.42	25.26
	Calibration intercept	−1.835	−1.191	−0.889	−0.504
	Calibration slope	0.021	0.362	0.532	0.761
	# Updates	24	24	24	24

ISSN: 2397-7523