Skip to main content

Table 1 (abstract O31). Overview of the three approaches we distinguish to assess the incremental value of a new predictor and the incremental value of the D-dimer test for the prediction of deep venous thrombosis

From: Methods for Evaluation of medical prediction Models, Tests And Biomarkers (MEMTAB) 2018 Symposium

Approach

Research question:

“What is the incremental value of a new predictor...”

regression model with and without the new predictor

Modeling approach

Incremental value of D-dimer test (95%CI)

1. Existing model

“...when the original model is used in the new dataset as originally developed?”

α + β1lp (a)

vs.

α + β1lp + β2NP (b)

Refitting all coefficients of the model is not an option. An alternative is to improve the discrimination and calibration of the original model in your dataset using fractional polynomials, splines (‘), or simple recalibration for the linear predictor (lp) of the original model.

Note that α + β1 in model (a) may differ from α + β1 in model (b) due to adjustment for the new predictor.

ΔAUC

NRIe

NRIc

0.085 (-0.012 to 0.18)

0.084 (0.013 to 0.15)

0.64 (0.56 to 0.72)

2. Model revision

“...when the original model is optimally fit to the new dataset?”

α + β1X1 + β2X2 (a)

vs.

α + β1X1 + β2X2 + β3NP (b)

Refit entire model with the same predictors (a) and add the new predictor to a model with the same predictors as in the original model (b).

Note that α + β1,2,3 in model (a) may differ from α + β1,2,3 in model (b) due to adjustment for the new predictor.

ΔAUC

NRIe

NRIc

0.082 (-0.012 to 0.18)

0.083 (0.0074 to 0.16)

0.61 (0.52 to 0.70)

3. New model development*

“...when the new predictor is incorporated in the original model in the new dataset?”

α + β1X1 + β3X3 (a)

vs.

α + β1X1 + β4NP (b)

Here β2X2 was removed from the existing model (a). After adding the new predictor (b) β3X3 was replaced by β4NP

Refit entire model with the same predictors, but now allow predictor selection (a). Repeat this step, but now after adding the new predictor to the list of candidate predictors (b).

Note that model (b) may include different predictors than model (a), due to replacement of predictors in model (a) by the new predictor in model (b)

ΔAUC

NRIe

NRIc

0.080 (-0.015 to 0.17)

0.077 (0.0023 to 0.15)

0.59 (0.48 to 0.70)

  1. α = intercept as estimated in the new dataset; βi = the association of predictor i with the outcome in the new dataset; lp = the linear predictor by applying the existing model to individuals in the new dataset; NP = the new predictor; AUC = Area Under the Receiver Operating Characteristic; NRIe = net reclassification improvement (NRI), event-based: risk categories: 0 – 15.9%, 15.9% – 100%; NRIc = net reclassification improvement (NRI), continuous
  2. an “existing model” can be referred to as one that we don’t want to modify the coefficients of because it is so well established in clinical practice. With “model revision” and “new model development” we generally deal with a model that is open for modification and one that is not established in clinical practice and for which the coefficients or even the included predictors can be modified