Skip to main content

Table 2 Mathematical description of statistical models used for studying model variation

From: Validation of prediction models: examining temporal and geographic stability of baseline risk and estimated covariate effects

Model

Model description

Description

Ignoring temporal and geographic variation

 Model 1

logit(p ij ) = α 0 + βX ij where p ij denotes the probability of the outcome for the ith patient at the ith hospital. From this model, we extracted the linear predictor (LP ij )

Fixed effects model, ignoring temporal and geographic heterogeneity

Models accounting for geographic heterogeneity

 Model 2

logit(p ij ) = α 0j  + βX ij where the hospital-specific random effects α 0j  ~ N(α 0, σ 2)

Random intercept model, allowing for variation in baseline risk, but assuming common prognostic effects

 Model 3

logit(p ij ) = α 0j  + α 1LP ij where the hospital-specific random effects α 0j  ~ N(α 0, σ 2)

Rank 1 model, allowing for common effect of the linear predictor

 Model 4

logit(p ij ) = α 0j  + α 1j LP ij where \( \left(\begin{array}{l}{\alpha}_{0 j}\\ {}{\alpha}_{1 j}\end{array}\right)\sim \mathrm{M}\mathrm{V}\mathrm{N}\left(\left(\begin{array}{l}{\alpha}_0\\ {}{\alpha}_1\end{array}\right),\left(\begin{array}{l}{\sigma}_1^2\kern0.5em {\sigma}_{12}\\ {}{\sigma}_{12}\kern0.5em {\sigma}_2^2\end{array}\right)\right) \)

The distribution of the random effects was estimated to be \( \left(\begin{array}{l}{\alpha}_{0 j}\\ {}{\alpha}_{1 j}\end{array}\right)\sim \mathrm{M}\mathrm{V}\mathrm{N}\left(\left(\begin{array}{l}0.005\\ {}1.008\end{array}\right),\left(\begin{array}{l}0.0444\kern1em 0.0139\\ {}0.0139\kern1em 0.0162\end{array}\right)\right) \)

Rank 1 model, allowing for heterogeneity in the effect of the linear predictor

 Model 5

logit(p ij ) = α 0j  + α 1j LP ij  + α 2j X 1ij where \( \left(\begin{array}{l}{\alpha}_{0 j}\\ {}{\alpha}_{1 j}\\ {}{\alpha}_{2 j}\end{array}\right)\sim \mathrm{M}\mathrm{V}\mathrm{N}\left(\left(\begin{array}{l}{\alpha}_0\\ {}{\alpha}_1\\ {}{\alpha}_2\end{array}\right),\left(\begin{array}{l}{\sigma}_1^2\kern0.5em {\sigma}_{12}\kern0.5em {\sigma}_{13}\\ {}{\sigma}_{12}\kern0.5em {\sigma}_2^2\kern0.5em {\sigma}_{23}\\ {}{\sigma}_{13}\kern0.5em {\sigma}_{23}\kern0.5em {\sigma}_3^2\end{array}\right)\right) \) and X 1ij denote an individual predictor (e.g., age)

Fully stratified model, allowing for differential prognostic effects (one model per covariate)

Models accounting for temporal heterogeneity

 Model 6

logit(p ij ) = α 0j  + α 1 T ij  + α 2LP ij where the hospital-specific random effects α 0j  ~ N(α 0, σ 2) and the fixed effect for LP ij are defined as in Model 3, and T ij denotes the temporal period (T = 0 for phase 1 vs T = 1 for phase 2)

Random intercept model with a fixed main effect for phase 2 vs phase 1

 Model 7

logit(p i ) = α 0j  + α 1 T ij  + α 2LP ij  + α 3 T ij  × LP ij

Random intercept model with a fixed interaction effect for phase 2 vs phase 1. The prognostic effect differs between time periods

 Model 8

logit(p i ) = α 0j  + α 1 X ij  + α 2 T ij  + α 3 T ij  × X ij

Random intercept model that allowed effect of each predictor to vary between time periods

Simultaneous exploration of geographic and temporal heterogeneity of predictor effects

 Model 9

logit(p ij ) = α 0j  + α 1j LP ij  + α 2j T ij  + α 3j T ij  × LP ij where \( \left(\begin{array}{l}{\alpha}_{0 j}\\ {}{\alpha}_{1 j}\\ {}{\alpha}_{2 j}\\ {}{\alpha}_{3 j}\end{array}\right)\sim \mathrm{M}\mathrm{V}\mathrm{N}\left(\left(\begin{array}{l}{\alpha}_0\\ {}{\alpha}_1\\ {}{\alpha}_2\\ {}{\alpha}_3\end{array}\right),\left(\begin{array}{l}{\sigma}_1^2\kern0.5em {\sigma}_{12}\kern0.5em {\sigma}_{13}\kern0.5em {\sigma}_{14}\\ {}{\sigma}_{12}\kern0.5em {\sigma}_2^2\kern0.5em {\sigma}_{23}\kern0.5em {\sigma}_{24}\\ {}{\sigma}_{13}\kern0.5em {\sigma}_{23}\kern0.5em {\sigma}_3^2\kern0.5em {\sigma}_{34}\\ {}{\sigma}_{14}\kern0.5em {\sigma}_{24}\kern0.5em {\sigma}_{34}\kern0.5em {\sigma}_4^2\end{array}\right)\right) \)

The effect of the linear predictor varies between hospitals; the effect of temporal period varies across hospitals; and the effect of temporal period on the predictor effects varies across hospitals