From: Methods for Evaluation of medical prediction Models, Tests And Biomarkers (MEMTAB) 2018 Symposium
Approach | Research question: “What is the incremental value of a new predictor...” | regression model with and without the new predictor | Modeling approach | Incremental value of D-dimer test (95%CI) | |
---|---|---|---|---|---|
1. Existing model | “...when the original model is used in the new dataset as originally developed?” | α + β1lp (a) vs. α + β1lp + β2NP (b) | Refitting all coefficients of the model is not an option. An alternative is to improve the discrimination and calibration of the original model in your dataset using fractional polynomials, splines (‘), or simple recalibration for the linear predictor (lp) of the original model. Note that α + β1 in model (a) may differ from α + β1 in model (b) due to adjustment for the new predictor. | ΔAUC NRIe NRIc | 0.085 (-0.012 to 0.18) 0.084 (0.013 to 0.15) 0.64 (0.56 to 0.72) |
2. Model revision | “...when the original model is optimally fit to the new dataset?” | α + β1X1 + β2X2 (a) vs. α + β1X1 + β2X2 + β3NP (b) | Refit entire model with the same predictors (a) and add the new predictor to a model with the same predictors as in the original model (b). Note that α + β1,2,3 in model (a) may differ from α + β1,2,3 in model (b) due to adjustment for the new predictor. | ΔAUC NRIe NRIc | 0.082 (-0.012 to 0.18) 0.083 (0.0074 to 0.16) 0.61 (0.52 to 0.70) |
3. New model development* | “...when the new predictor is incorporated in the original model in the new dataset?” | α + β1X1 + β3X3 (a) vs. α + β1X1 + β4NP (b) Here β2X2 was removed from the existing model (a). After adding the new predictor (b) β3X3 was replaced by β4NP | Refit entire model with the same predictors, but now allow predictor selection (a). Repeat this step, but now after adding the new predictor to the list of candidate predictors (b). Note that model (b) may include different predictors than model (a), due to replacement of predictors in model (a) by the new predictor in model (b) | ΔAUC NRIe NRIc | 0.080 (-0.015 to 0.17) 0.077 (0.0023 to 0.15) 0.59 (0.48 to 0.70) |