Skip to main content

Development of a prediction model of conversion to Alzheimer’s disease in people with mild cognitive impairment: the statistical analysis plan of the INTERCEPTOR project

Abstract

Background

In recent years, significant efforts have been directed towards the research and development of disease-modifying therapies for dementia. These drugs focus on prodromal (mild cognitive impairment, MCI) and/or early stages of Alzheimer’s disease (AD). Literature evidence indicates that a considerable proportion of individuals with MCI do not progress to dementia. Identifying individuals at higher risk of developing dementia is essential for appropriate management, including the prescription of new disease-modifying therapies expected to become available in clinical practice in the near future.

Methods

The ongoing INTERCEPTOR study is a multicenter, longitudinal, interventional, non-therapeutic cohort study designed to enroll 500 individuals with MCI aged 50–85 years. The primary aim is to identify a biomarker or a set of biomarkers able to accurately predict the conversion from MCI to AD dementia within 3 years of follow-up. The biomarkers investigated in this study are neuropsychological tests (mini-mental state examination (MMSE) and delayed free recall), brain glucose metabolism ([18F]FDG-PET), MRI volumetry of the hippocampus, EEG brain connectivity, cerebrospinal fluid (CSF) markers (p-tau, t-tau, Aβ1-42, Aβ1-42/1–40 ratio, Aβ1-42/p-Tau ratio) and APOE genotype. The baseline visit includes a full cognitive and neuropsychological evaluation, as well as the collection of clinical and socio-demographic information. Prognostic models will be developed using Cox regression, incorporating individual characteristics and biomarkers through stepwise selection. Model performance will be evaluated in terms of discrimination and calibration and subjected to internal validation using the bootstrapping procedure. The final model will be visually represented as a nomogram.

Discussion

This paper contains a detailed description of the statistical analysis plan to ensure the reproducibility and transparency of the analysis. The prognostic model developed in this study aims to identify the population with MCI at higher risk of developing AD dementia, potentially eligible for drug prescriptions. The nomogram could provide a valuable tool for clinicians for risk stratification and early treatment decisions.

Trial registration

ClinicalTrials.gov NCT03834402. Registered on February 8, 2019

Peer Review reports

Introduction

Background and rationale

Over the last two decades, significant efforts have been dedicated to the development of potentially disease-modifying treatments for Alzheimer’s disease (AD) [1, 2]. In recent years, phase 3 clinical trials have been completed on monoclonal antibody therapies proposed to slow down the conversion of AD, and requests for marketing authorisation have been submitted to regulatory authorities. Lecanemab was approved by the Food and Drugs Administration (FDA) in July 2023 and is currently under assessment by the European Medicines Agency (EMA). Aducanumab has received accelerated approval from the FDA for the treatment of the early stage of AD, but it has not been approved in Europe. Donanemab is headed for approval in the USA [3]. These monoclonal antibodies target amyloid plaques in the brain and are intended for those with MCI and mild AD. The MCI population has an estimated 15% incidence of dementia within 2 years, making them at higher risk for developing dementia [4]. However, this population also includes individuals who will never convert to AD, ranging from 14 to 38% depending on the study setting [4, 5]. As monoclonal antibody therapies are expensive and have non-negligible adverse effects [6], selectively targeting individuals with MCI who are at a higher risk of converting to AD dementia is needed. In this context, the Italian Ministry of Health and the Italian Medicines Agency (AIFA) initiated the INTERCEPTOR project in 2018 (https://www.interceptorproject.com) with the primary aim of evaluating the most reliable biomarker or set of biomarkers for predicting the conversion from MCI to AD dementia within a 3-year follow-up period. The study is designed as a longitudinal cohort study, where the baseline clinical and biomarker characteristics of the individuals diagnosed with MCI will be analysed in relation to their risk of conversion to AD dementia. The findings of this study will be used to identify individuals eligible for potential disease-modifying treatment in clinical practice. The secondary aim of the project is to define an optimal organisational model that can be readily implemented in clinical practice aligning with the primary goal and being economically sustainable. However, this aim is not part of the present statistical analysis plan.

Study objectives

The primary aim of the study is to develop and internally validate a multivariable prediction model capable of identifying a biomarker or a set of biomarkers to predict the conversion from MCI to Alzheimer’s disease after 3 years of follow-up.

Methods

The analyses outlined in this document are in full compliance with the TRIPOD statement for Transparent Reporting of the development of a multivariable prediction model for Individual Prognosis Or Diagnosis [7] (see Additional file 1). The statistical analysis plan has been finalised prior to the completion of the data collection. The study was registered at www.clinicaltrials.gov (NCT03834402) in January 2019. Details on the study protocol have been published elsewhere [8]. The analyses will be carried out using Stata v17.0 (StataCorp, College Station, Texas, USA) and R software v4.3.0 (R Foundation for Statistical Computing, Vienna, Austria).

Study design

This is a multicentre, interventional, non-therapeutic cohort study. A sample of 500 individuals consecutively diagnosed with MCI at the Center for Cognitive Disorders and Dementia (Centri per i Disturbi Cognitivi e Demenza, CDCD) was planned to be enrolled. The follow-up process, consisting of neuropsychological and clinical assessments, is scheduled every 6 months over a 36-month observation period. Recruitment started on December 21, 2018, and the follow-up of the last recruited person will end on December 22, 2023.

Study population

The study includes individuals of any gender, aged 50 to 85 years, with a formal diagnosis of MCI according to the National Institute on Aging-Alzheimer’s Association (NIA-AA) criteria [9]. The exclusion and inclusion criteria are provided in Table 1. Participants newly diagnosed with MCI were consecutively recruited from 19 Italian memory clinics (CDCD), which are centres with documented expertise in the diagnosis and treatment of AD and MCI, distributed across the national territory in each country’s macro-areas (Northwest, North East, Central, South, Islands) to reflect a representative reproduction of the Italian Health System organisation in this field. Each recruiting centre performed a comprehensive neuropsychological evaluation, collected blood samples for genetic testing, performed lumbar puncture for cerebrospinal fluid (CSF) analysis and acquired an electroencephalogram (EEG), FDG-PET scan and brain magnetic resonance imaging (MRI) either on site or at facilities associated with the recruiting centres. The evaluation of each biomarker and the risk group classification were carried out by four expert centres of reference selected for their expertise. Additional details regarding the organisational architecture of the project, protocol implementation and exam acquisition are reported elsewhere [8].

Table 1 Inclusion and exclusion criteria

Baseline characteristics

Participants will be characterised based on all variables collected at baseline, separately for those who will convert to dementia and those who will not. These variables include socio-demographic information, medical history, physical examination, risk factors, comorbidities and concomitant medications (Additional file 2). Categorical data will be summarised by absolute frequencies and percentages. Continuous data will be summarised using mean and standard deviation for normally distributed variables and median with interquartile range for skewed distributions. Statistical comparisons will be performed using t-tests, Wilcoxon rank-sum tests and chi-square tests as appropriate. Statistical significance will be set at the 5% level.

Outcomes

Primary outcome

The primary outcome is the conversion to AD dementia within 3 years of follow-up after the diagnosis of MCI. Visits to assess the study outcome are planned every 6 months. Follow-up time starts at the date of neuropsychological evaluation at baseline and ends at (i) date of conversion, (ii) end of follow-up, (iii) death and (iv) loss to follow-up, whichever occurs first. The conversion to dementia will be diagnosed by specialists according to the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) on the basis of clinical, cognitive and functional examination, including the CDR. The diagnosis will not be performed based on any of the biomarkers under study as clinicians are blinded with respect to the biomarkers evaluation which is a necessary condition to correctly evaluate the predictive performance of biomarkers and avoid biased estimation of the association between predictors and outcome. Conversion to Alzheimer’s disease is confirmed when an individual receives two consecutive diagnoses of the same type. The date of the first diagnosis will be considered as the date of conversion.

Secondary outcome

The secondary outcomes are the conversion to Alzheimer’s disease within 3 years of follow-up in amnesic patients and the conversion to other forms of dementia.

Predictors

Biomarkers were selected for their accuracy in predicting the conversion from MCI to AD dementia based on the best evidence available in the literature [8]. The biomarkers under investigation include the Mini-Mental State Examination (MMSE), Delayed Free Recall (DFR) as a subscore of the Free and Cued Selective Reminding test (FCSRT), cerebrospinal fluid (CSF) parameters, (18F)FDG-PET, volumetric MRI, EEG for brain connectivity and APOE genotype. Details on the standardisation procedure for acquiring each biomarker are provided in Rossini et al. [8].

For CSF biomarkers, the following fluid levels are considered: threonine-181 (p-tau), total-tau (t-tau), amyloid-beta Aβ1-42, Aβ1-42/1–40 ratio, Aβ1-42/p-tau ratio. 18Fluorodeoxyglucose positron emission tomography (18F)FDG-PET is analysed with voxel-based single subject using the optimised procedure based on statistical parametrical mapping (SPM), involving a large database of normal controls for comparison and the new FDG atlas for normalisation [10, 11]. Two techniques are used to compute the right and left hippocampal volumes, normalised to the intracranial volume in volumetric MRI. The first technique is ACM-Adaboost, which is a machine learning algorithm used to classify the voxels belonging to the hippocampal region. The second technique is Freesurfer v6.0, which is a surface-based approach [12]. Seven parameters of Small World (SW) are computed in delta, theta, alpha1, alpha2, beta1, beta2 and gamma frequency bands to evaluate the brain connectivity via graph theory in EEG [13]. As part of the study, each participant is required to undergo all necessary tests for biomarker acquisition. These tests are required to be completed within 60 days from the neuropsychological battery assessment and the acquisition of informed consent at the beginning of the study. Biomarker testing is conducted in a blinded manner, meaning that the data collected at baseline, including predictor variables such as biomarkers and any information related to medical history or clinical examination, are not taken into account during biomarker testing. Moreover, to improve the predictive accuracy of biomarkers, clinical and demographic characteristics will be also considered. A critical review of the relevant literature was conducted to identify factors associated with the development of dementia [14, 15]. The following predictors, selected according to the clinical relevance among all the demographic and clinical variables collected, will be used in the development and validation of the multivariable prediction model: sex, age, level of education (measured in years of education), type of MCI (amnesic, non-amnesic) according to single or multiple domains, Amsterdam IADL (short version) [16], Cumulative Illness Rating Scale (CIRS) including severity and comorbidity index [17], hypertension, family history of dementia, smoking status, status of cohabitation as a proxy of social isolation, psychiatric disease (past and present) and cardiovascular disease. All these data, collected during the clinical assessment visit, could significantly improve the accuracy of the prediction model. All predictors are collected at baseline.

Sample size

To determine the sample size, the methodological work of Bujang and Adnan [18] was followed. This work provides guidance for estimating the minimum sample size required in a screening or diagnostic study to achieve the desired sensitivity and specificity with adequate power and a controlled level of type I error. Based on the literature review, the level of sensitivity for the biomarkers under study ranges between 52 and 81%, while the specificity ranges between 52 and 86%. After a follow-up of about 2.5 years, the incidence of AD in a cohort of individuals with MCI was observed to range from 29 to 46% [19, 20]. Based on this data and utilising the tables provided by Bujang and Adnan, a minimum sample size of 388 individuals with MCI is needed assuming an expected incidence of AD of 40% after a 3-year follow-up period, a sensitivity value of 80% for an individual biomarker (working hypothesis), a power ≥ 80% and a p-value lower than 5%. Additionally, applying the formula described by Buderer [21] which incorporates the event frequency into the sample size calculation, it was verified that 400 evaluated individuals would ensure an acceptable level of precision (the half-width of the 95% CI) between 4 and 7.5% for both sensitivity and specificity (Table 2). Assuming a dropout rate of 20%, a total sample of 500 individuals with MCI has been determined.

Table 2 Level of precisiona on the estimates of sensitivity and specificity (for a range of values) corresponding to a sample size of 400, according to an incidence of AD ranging from 27 to 40%

The primary objective of this study is to develop a prediction model including biomarkers in addition to clinical variables. Following the practical guidance provided by Riley et al. [22] to calculate the number of events needed for developing a multivariable prediction model, and assuming an expected C-index of 0.8 [23] which corresponds to an apparent Cox-Snell R2 of 0.31, and considering an overall incidence rate of 16% (based on an average follow-up of 2.5 years), we have determined that the number of events required per predictor parameter is 9.5. Consequently, the planned sample is deemed sufficient to include up to 17 candidate predictors.

Statistical analysis method

All the analysis will be conducted on patients with the baseline assessment of biomarkers available; consequently, patients without biomarker assessment will be excluded from the primary analysis.

To exclude selection bias, a comparison of baseline characteristics of individuals not included in the analysis due to incomplete biomarkers assessment and those included in the analysis will be performed.

A multivariable predictive model of conversion to AD will be developed including biomarkers, socio-demographic and clinical predictors. The evaluation of biomarkers for MMSE, DFR, CSF parameters and MRI measures will be conducted considering the continuous form and the categorical form applying validated cut-offs.

For the MMSE [24], a cut-off of 24 (< 24) for the age- and sex-corrected score will be used, as indicated in the literature [25]. For the correction of the crude score, Measso et al. [26] will be used for individuals aged ≤ 64, and Magni et al. [25] will be used for those aged > 64. Regarding the DFR, a test result will be classified as abnormal if the age-adjusted score is ≤ 6.31 [27].

For MRI, measurements of the right and left hippocampus are considered for each technique, ACM-Adaboost and Freesurfer, and atrophy is defined below the 5th percentile according to the age-specific thresholds defined for individuals aged 56–90 years, as published [12]. For individuals aged 50–55 years, the cut-offs are extrapolated from the 5th percentile function (see Additional file 2). Right and left hippocampal atrophy measured with the two techniques will be analysed separately.

For CSF biomarkers, p-tau, t-tau, Aβ1-42, Aβ1-42/1–40 ratio and Aβ1-42 /p-tau ratio will be evaluated separately. The cut-offs used for each biomarker are reported in Table 3 along with the literature references [28,29,30,31]. For the EEG, the classification of results into normal or abnormal based on the SW values will be performed by the expert centre using a machine learning (ML) method based on fine Gaussian support vector machine. This ML classification model has been trained on external data including both healthy individuals and individuals with AD [32] and will be applied to the study cohort. The FDG-PET result will be categorised as either positive or negative by the evaluators at the expert centre based on the methodology previously described [10, 11]. In the case of FDG-PET positivity, it will also be specified whether it is indicative of an AD-like form or other forms of dementia. The APOE-e4 genotype will be classified as none, one (heterozygosis), or both alleles (homozygosis) (see Table 3 and Additional file 3 for details of the cut-offs and literature references).

Table 3 Reference value for identifying abnormal values by biomarkers

However, these cut-offs were determined to discriminate between normal cognition and AD, but the population under study is at higher risk of developing dementia compared to the normal population. Therefore, for continuous biomarkers the receiver operating characteristic (ROC) curves will be used to identify new optimal cut-offs for the specific population under study. For biomarkers known to be influenced by sex and/or age, the corrected values will be used. In this context, sensitivity and specificity are both considered targets for determining the optimal cut-off; therefore, among all the commonly used methods [33], the distance to the left-upper corner was chosen. This method maximises both sensitivity and specificity and guarantees a better balance between the two measures [34]. Warnings are raised about its use as not always the method leads to an optimal cut-off in the intention of maximising overall correct classification rates compared to the commonly used Youden’s method [35]; therefore, Youden’s method will be also provided for comparison.

Statistical model and variable selection

Initially, an analysis of socio-demographic and clinical variables will be conducted using the Cox model to identify a core of variables significantly associated with the outcome that will be included in the development of the prognostic multivariable model. A check of multicollinearity between predictors will be assessed, then a stepwise selection starting from the full model will be applied using a threshold of 0.10 for removal and 0.05 for re-admission of variables. A final check of variables not selected by stepwise will be conducted using the likelihood ratio test [36, 37]. Age and sex will be forced in the procedure selection and retained in the final model. Subsequently, these selected variables will be included in multivariable models with the addition of individual biomarkers one at a time. The additional predictive value of each biomarker beyond clinical and socio-demographic variables will be calculated. The performance results will be used to determine for each biomarker which form (continuous or categorical) will be chosen for the development of the multivariable prediction model. Categorisation is typically preferred for the application in clinical practice, but as any categorisation implies a loss of information, we will select the biomarker in a categorical form if the loss in performance could be considered negligible with respect to the continuous form. Categorisation will also be chosen in cases where non-linearity is detected for the continuous form. The categorisation according to the threshold provided by the experts (Table 3) will be preferred in case a small loss of performance is observed compared to the continuous form.

The final predictive model will be identified through a multivariable model considering as eligible predictors all the biomarkers and all selected predictors among the clinical and socio-demographic variables using the stepwise selection procedure as described above. Age and sex will be forced into the model selection process. A final check of the result from the automatic selection procedure will be conducted to ensure that no significant biomarkers were excluded and, vice versa, that no negligible biomarkers were included.

Checking assumptions and variable handling

The relationship between study outcome and each continuous predictor will be explored. Linearity will be tested. Transformation or categorisation of variables will be considered for dealing with non-linearity. Collinearity between predictors will be measured with the variance inflation factors (VIF) [38, 39]. Values equal to 1 indicate that variables are not correlated, VIF values greater than 5 are generally considered a cause of concern, and values exceeding 10 are indicative of problematic multicollinearity that needs to be corrected. The proportional hazards assumption will be tested using Schoenfeld residuals and checked by visual inspection of the log–log plot. If the assumption is not satisfied, the variable by time interaction will be included in the model.

Predictive performance

For each biomarker in the categorical form, the accuracy measures (sensitivity, specificity, positive predictive value, negative predictive value, overall accuracy, positive and negative likelihood ratios) will be calculated for comparison with the literature. For continuous biomarkers, the area under the receiver operating characteristic curve (AUC) will be also reported. Moreover, a univariate analysis will be conducted using the proportional hazard Cox model.

To evaluate the performance of the prediction models, several measures will be considered. The concordance c-index with its 95% CI will be provided as a discrimination measure and the extensions of binary AUC to survival analysis proposed by Harrell will be used [40, 41]. The Akaike information criterion will be provided as a measure of the goodness of fit of the model.

The added value on performance of each biomarker to the multivariable model with demographical and clinical variables will be assessed by the category-free Net Reclassification Improvement (NRI), which is the net proportion of events reclassified correctly plus the net proportion of non-events reclassified correctly. The delta AUC (difference in c-statistics) will be also provided [41], and the likelihood ratio test will be used to assess the statistical significance of the incremental contribution of each biomarker [36, 37].

In addition, to assess the predictive performance of the final multivariable model, calibration measures will also be reported [38, 42]. A calibration plot will be done by plotting the observed outcome proportion (on the y-axis) versus the ordered predicted outcome probabilities (on the y-axis) by quantile of the predicted risk. Subsequently, the calibration intercept and slope and their 95% CI will be estimated. There is a perfect calibration when the intercept is zero and the slope is one. Moreover, overall measures of performance, the Nagelkerke’s R2 and the Brier score, will be provided. The Brier score is a composite measure of discrimination and calibration with lower scores indicating improved model accuracy, and it will be calculated at a fixed time point (3 years).

Missing data

To contain the number of missing data and patient dropout, telephone contact with individuals who missed scheduled visits, or their caregivers, was planned offering the option of a remote visit if attendance was not feasible. No missing data will be accepted in the evaluation of biomarkers; therefore, it is required that all participants have a full evaluation of the biomarkers under study. A high level of completeness is expected for the main baseline characteristics, especially for age, sex and level of education. At least 80% of completeness will be required to include predictive factors in the analysis and the multiple imputation techniques by chained equations will be applied for dealing with missing data. Missing values for continuous normally distributed data will be imputed using linear regression, while the predictive mean matching method will be applied for not normally distributed data. Logistic regression will be used for binary data and ordinal logistic for ordered categorical data [43]. All predictors, including socio-demographic and clinical variables and biomarkers, will be considered potential variables for the imputation model. The outcome variable will be included in the imputation model adding the censoring indicator and the cumulative baseline hazard function estimated with the Nelson–Aalen method. For each variable with missing data to be imputed, candidate variables for inclusion in the imputation model will be selected based on their association with the missingness indicator. Associations will be assessed using t-test for continuous variables and chi-square test for categorical variables, with a p-value threshold of 0.05. If the frequency of missing data is less than 5% for each selected predictive variable, the main analysis will be conducted using only one imputed value generated according to the approach described above using the predictive mean imputation with 5 nearest neighbours to preserve variance and avoid the inflation of associations among the variable and predictors. A complete case analysis will be performed on this imputed dataset.

Internal validation

The final predictive model will be internally validated using the bootstrap technique with 500 bootstrap samples [38]. The stepwise selection will be repeated in the validation process (i.e. stepwise procedure will be repeated within each bootstrap sample). Consequently, Harrel’s c-index of the predictive model will be adjusted to take into account the estimated potential overfitting and a corrected c-index will be reported. Apparent and corrected calibration measures will be reported [44]. The correction will be performed using the shrinkage factor calculated with bootstrapping. The shrinkage factor may take values between 0 and 1, and a value above 0.90 indicates a small overfitting. As the shrinkage of regression coefficients is an important way to reduce overfitting, the shrinkage factor will be also applied to regression coefficients and intercept of the final model [38] to improve the prediction ability of the regression model when applied to external data. Both the original and shrunken regression coefficients will be reported.

Sensitivity analysis

As sensitivity, the robustness of the stepwise procedure for the selection of variables in the final model will be performed by repeating the stepwise selection considering different levels of p for removal and for re-admission and using the Akaike information criterion (p ≤ 0.157). Additionally, the least absolute shrinkage and selection operator (LASSO) method will be used, and the tuning parameter lambda will be chosen by tenfold cross-validation. The predictive performance of the models will be compared.

A complete case analysis will be performed for the final model if the number of missing data will be contained.

Additionally, the competing risk analysis will be performed taking into account death as a competing event using Fine and Gray’s semiparametric proportional sub-distribution hazards model.

Model presentation and risk groups

For each predictive model, the hazard ratios with 95%CI and p-value will be reported. For the final predictive model, a nomogram will be generated. A nomogram is a graphical representation of the statistical predictive model. It is a visual tool that translates model numeric results into a visual format, making it easier to estimate an outcome based on multiple variables. Each variable occurrence is assigned a specific score based on its contribution to the overall prediction. The total score from all variables is then used to estimate the probability of the predicted event (in this case, the conversion to AD at 3 years) conditioning to the individual characteristics [45]. For this purpose, the covariates in the Cox model will be centred. Sensitivity and specificity corresponding to different thresholds of the prediction risk score will also be provided for the final model. Finally, three or more risk categories will be proposed defining low-, medium- and high-risk groups based on the predictive risk score considering the highest sensitivity corresponding to an adequate level of specificity. This category definition will help the clinician with the choice to treat individuals with MCI based on the likelihood of converting to dementia in the following 3 years. Because avoiding the clinical consequence of treating a false positive is a target as important as treating a true positive, a rate of false positives less than 20% (i.e. specificity of at least 80%) is considered acceptable for a medium–high-risk group. We will start by inspecting deciles of the predictive score and grouping them into homogeneous classes based on levels of sensitivity and specificity. Kaplan–Meier curves will be presented for each risk group.

Additional analysis

The COVID-19 pandemic occurred while the recruitment was still ongoing. A growing literature has shown the impact of restrictive measures adopted during the pandemic such as isolation and COVID-19 infection on cognitive decline [46,47,48]. Data regarding COVID-19 infections, severity of COVID-19 leading to hospitalisation and requirement of intensive care unit and vaccination status will be reported. Characteristics of individuals enrolled before and during the pandemic period will be compared to verify whether the pandemic had an impact on the recruitment of participants. Furthermore, the association of COVID-19 infection will be evaluated by univariate and multivariable Cox model including the status variable as time-dependent.

Analysis of secondary clinical endpoints

For secondary clinical endpoints, conversion to AD in amnesic patients and conversion to other forms of dementia, univariate analysis will be repeated. The final multivariable predictive model identified in the primary analysis will be applied. The model performance will be assessed by Harrel’s C-index, as well as the other performance measures specified in the primary analysis.

Discussion

To our knowledge, the INTERCEPTOR study represents the first publicly funded study to evaluate, through a harmonisation process among specialised centres distributed throughout the Italian territory, a wide set of biomarkers aimed at predicting the conversion from MCI to AD based over an adequate period of follow-up. The strength of this study lies in the inclusion of a broad range of biomarkers and the standardised and homogeneous collection of these biomarkers. Substantial efforts have been made to standardise biomarker acquisition and analysis, with specialised centres providing expertise in risk diagnosis classification [8]. The study has encountered challenges posed by the COVID-19 pandemic, primarily evident in the recruitment slowdown. The COVID-19 pandemic occurred when the study was in the recruitment phase. For patient recruitment, participants needed to undergo multiple visits for clinical assessment and biomarker acquisition, including at clinical centres distinct from the recruitment centre. The pandemic limited access to certain services necessary for clinical visits and biomarker acquisition. The timeline for scheduled visits has been impacted, leading to deviations from the protocol, which will be documented and appropriately addressed. Despite implementing mitigation actions aimed at ensuring the completion of the INTERCEPTOR study, such as extending the recruitment period and introducing remote visits, we were not able to reach the targeted sample size. Although the initial goal of enrolling 500 individuals with MCI was not met, the final sample size is anticipated to provide estimates of accuracy measures with sufficient precision and to enable the development of a predictive model incorporating demographical and clinical information alongside biomarkers. The model will undergo internal validation, enhancing its reliability. Nevertheless, one major limitation of this study is the lack of external validation, which is crucial to assess the generalizability and robustness of the model. However, once the model is defined, a comparison will be made with existing prognostic models for conversion from MCI to AD available in the literature, and the feasibility of conducting an external validation using data provided by other studies will be evaluated. Furthermore, it is important to note that certain methodological decisions, such as handling non-linearity and categorisation of biomarkers, were not strictly guided by the best evidence but were made to prioritise the clinical interpretability and applicability of the model. This highlights a trade-off between model performance and practical utility in clinical settings. The decision to represent the identified predictive model in the form of a nomogram aims to facilitate its interpretability and applicability with the final goal of assisting clinicians in assessing eligibility for prescribing upcoming new treatments proposed for dementia.

Conclusion

This paper presents the details of the statistical analysis to be conducted in accordance with the relevant guidelines. The results of the pre-specified analyses will be subsequently made available to minimise the outcome reporting bias.

Availability of data and materials

Not applicable.

Abbreviations

AD:

Alzheimer’s disease

AIFA:

Italian Medicine Agency

AUC:

Area under the receiver operating characteristic curve

APOE:

Apolipoprotein E

CDCD:

Center for Cognitive Disorders and Dementia

CDR:

Clinical Dementia Rating

CI:

Confidence interval

CSF:

Cerebrospinal fluid

DFR:

Delayed free recall

DSM-5:

Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition

EEG:

Electroencephalogram

EMA:

European Medicines Agency

FCRST:

Free and Cued Selective Reminding Test

FDA:

Food and Drug Administration

FDG-PET:

Fluorodeoxyglucose-positron emission tomography

MCI:

Mild cognitive impairment

MMSE:

Mini-mental state examination

MRI:

Magnetic resonance imaging

ML:

Machine learning

NIA-AA:

National Institute on Aging-Alzheimer’s Association

NRI:

Net Reclassification Improvement

ROC:

Receiver operating characteristic

SPM:

Statistical Parametrical Mapping

SW:

Small World

TRIPOD:

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis

VIF:

Variance inflation factor

References

  1. Cummings J, Lee G, Nahed P, Kambar MEZN, Zhong K, Fonseca J, Taghva K. Alzheimer’s disease drug development pipeline: 2022. Alzheimers Dement (N Y). 2022;8(1):e12295. https://doi.org/10.1002/trc2.12295. PMID: 35516416; PMCID: PMC9066743.

    Article  PubMed  Google Scholar 

  2. Lacorte E, Ancidoni A, Zaccaria V, Remoli G, Tariciotti L, Bellomo G, Sciancalepore F, Corbo M, Lombardo FL, Bacigalupo I, Canevelli M, Piscopo P, Vanacore N. Safety and efficacy of monoclonal antibodies for Alzheimer’s disease: a systematic review and meta-analysis of published and unpublished clinical trials. J Alzheimers Dis. 2022;87(1):101–29. https://doi.org/10.3233/JAD-220046. PMID: 35275549; PMCID: PMC9198746.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Salemme S, Ancidoni A, Locuratolo N, Piscopo P, Lacorte E, Canevelli M, Vanacore N. Advances in amyloid-targeting monoclonal antibodies for Alzheimer’s disease: clinical and public health issues. Expert Rev Neurother. 2023;23(12):1113–29. https://doi.org/10.1080/14737175.2023.2284305.

    Article  CAS  PubMed  Google Scholar 

  4. Petersen RC, Lopez O, Armstrong MJ, et al. Practice guideline update summary: mild cognitive impairment: report of the guideline development, dissemination, and implementation subcommittee of the American Academy of Neurology. Neurology. 2018;90(3):126–35.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Canevelli M, Grande G, Lacorte E, Quarchioni E, Cesari M, Mariani C, Bruno G, Vanacore N. Spontaneous reversion of mild cognitive impairment to normal cognition: a systematic review of literature and meta-analysis. JAMDA. 2016;17:943–8.

    PubMed  Google Scholar 

  6. Filippi M, Cecchetti G, Spinelli EG, Vezzulli P, Falini A, Agosta F. Amyloid-related imaging abnormalities and β-amyloid-targeting antibodies: a systematic review. JAMA Neurol. 2022;79(3):291–304. https://doi.org/10.1001/jamaneurol.2021.5205. PMID: 35099507.

    Article  PubMed  Google Scholar 

  7. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med. 2015;13:1. https://doi.org/10.1186/s12916-014-0241-z.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Rossini PM, Cappa SF, Lattanzio F, Perani D, Spadin P, Tagliavini F, Vanacore N. The Italian INTERCEPTOR Project: from the early identification of patients eligible for prescription of antidementia drugs to a nationwide organizational model for early Alzheimer’s disease diagnosis. J Alzheimers Dis. 2019;72(2):373–88. https://doi.org/10.3233/JAD-190670. Erratum in: J Alzheimers Dis. 2020;74(1):409. PMID: 31594234.

    Article  PubMed  Google Scholar 

  9. Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, Gamst A, Holtzman DM, Jagust WJ, Petersen RC, Snyder PJ, Carrillo MC, Thies B, Phelps CH. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):270–9. https://doi.org/10.1016/j.jalz.2011.03.008. Epub 2011 Apr 21. PMID: 21514249; PMCID: PMC3312027.

    Article  PubMed  Google Scholar 

  10. Della Rosa PA, Cerami C, Gallivanone F, Prestia A, Caroli A, Castiglioni I, Gilardi MC, Frisoni G, Friston K, Ashburner J, Perani D. EADC- PET Consortium A standardized [18F]-FDG-PET template for spatial normalization in statistical parametric mapping of dementia. Neuroinformatics. 2014;12(4):575–93.

    Article  PubMed  Google Scholar 

  11. Perani D, Della Rosa PA, Cerami C, Gallivanone F, Fallanca F, Vanoli EG, Panzacchi A, Nobili F, Pappatà S, Marcone A, Garibotto V, Castiglioni I, Magnani G, Cappa SF, Gianolli L, EADC-PETConsortium. Validation of an optimized SPM procedure for FDG-PET in dementia diagnosis in a clinical setting. Neuroimage Clin. 2014;24(6):445–54.

    Article  Google Scholar 

  12. De Francesco S, Galluzzi S, Vanacore N, Festari C, Rossini PM, Cappa SF, Frisoni GB, Redolfi A. Norms for automatic estimation of hippocampal atrophy and a step forward for applicability to the Italian population. Front Neurosci. 2021;15:656808. https://doi.org/10.3389/fnins.2021.656808. PMID: 34262425; PMCID: PMC8273578.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Vecchio F, Miraglia F, Judica E, Cotelli M, Alù F, Rossini PM. Human brain networks: a graph theoretical analysis of cortical connectivity normative database from EEG data in healthy elderly subjects. Geroscience. 2020;42(2):575–84. https://doi.org/10.1007/s11357-020-00176-2. Epub 2020 Mar 13. PMID: 32170641; PMCID: PMC7205974.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Crous-Bou M, Minguillón C, Gramunt N, Molinuevo JL. Alzheimer’s disease prevention: from risk factors to early intervention. Alzheimers Res Ther. 2017;9(1):71. https://doi.org/10.1186/s13195-017-0297-z. PMID: 28899416; PMCID: PMC5596480.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Qiu C. Epidemiology of Alzheimer’s disease: occurrence, determinants, and strategies toward intervention. Dialogues Clin Neurosci. 2009;11(2):111–28.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Jutten RJ, Peeters CF, Leijdesdorff SM, Visser PJ, Maier AB, Terwee CB, et al. Detecting functional decline from normal aging to dementia: development and validation of a short version of the Amsterdam IADL Questionnaire. Alzheimers Dement Diagn Assess Dis Monit. 2017;8:26–35.

    Google Scholar 

  17. Parmelee PA, Thuras PD, Katz IR, Lawton MP. Validation of the cumulative illness rating scale in a geriatric residential population. J Am Geriatr Soc. 1995;43:130–7. https://doi.org/10.1111/j.1532-5415.1995.tb06377.x.

    Article  CAS  PubMed  Google Scholar 

  18. Bujang MA, Adnan TH. Requirements for minimum sample size for sensitivity and specificity analysis. J Clin Diagn Res. 2016;10(10):YE01–6.

    PubMed  PubMed Central  Google Scholar 

  19. Poil S, de Haan W, Van der Flier W, Mansvelder H, Scheltens P, Linkenkaer-Hansen K. Integrative EEG biomarkers predict progression to Alzheimer’s disease at the MCI stage. Front Aging Neurosci. 2013;5:58. https://doi.org/10.3389/fnagi.2013.00058.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Grande G, Vanacore N, Vetrano DL, Cova I, Rizzuto D, Mayer F, Maggiore L, Ghiretti R, Cucumo V, Mariani C, Cappa SF, Pomati S. Free and cued selective reminding test predicts progression to Alzheimer’s disease in people with mild cognitive impairment. Neurol Sci. 2018;39(11):1867–75. Eur J Neurol submitted.

    Article  PubMed  Google Scholar 

  21. Buderer NMF. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med. 1996;3(9):895–900.

    Article  CAS  PubMed  Google Scholar 

  22. Riley R, Snell K, Ensor J, Burke D, Harrell F, Moons K, Collins G. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat Med. 2018;38(7):1276–96. https://doi.org/10.1002/sim.7992.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Chen Y, Qian X, Zhang Y, Su W, Huang Y, Wang X, Chen X, Zhao E, Han L, Ma Y. Prediction models for conversion from mild cognitive impairment to Alzheimer’s disease: a systematic review and meta-analysis. Front Aging Neurosci. 2022;14:840386. https://doi.org/10.3389/fnagi.2022.840386. PMID: 35493941; PMCID: PMC9049273.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98. https://doi.org/10.1016/0022-3956(75)90026-6. PMID: 1202204.

    Article  CAS  PubMed  Google Scholar 

  25. Magni E, Binetti G, Bianchetti A, Rozzini A, Trabucchi M. Mini-mental state examination: a normative study in elderly population. Eur J Neurol. 1996;3(198):202.

    Google Scholar 

  26. Measso G, Caverzeran F, Zappalà G, Lebowitz BD, Crook TH, Pirozzolo FJ, Amaducci LA, Massari D, Grigoletto F. The mini-mental state examination: normative study of an Italian random sample. Dev Neuropsychol. 1993;9(2):77–95.

    Article  Google Scholar 

  27. Frasson P, Ghiretti R, Catricalà E, Pomati S, Marcone A, Parisi L, Rossini PM, Cappa SF, Mariani C, Vanacore N, Clerici F. Free and cued selective reminding test: an Italian normative study. Neurol Sci. 2011;32(6):1057–62. https://doi.org/10.1007/s10072-011-0607-3. Epub 2011 May 19. Erratum in: Neurol Sci. 2012 Apr;33(2):481. PMID: 21594655.

    Article  CAS  PubMed  Google Scholar 

  28. Bellomo G, Indaco A, Chiasserini D, Maderna E, Paolini Paoletti F, Gaetani L, Paciotti S, Petricciuolo M, Tagliavini F, Giaccone G, Parnetti L, Di Fede G. Machine learning driven profiling of cerebrospinal fluid core biomarkers in Alzheimer’s disease and other neurological disorders. Front Neurosci. 2021;15:647783. https://doi.org/10.3389/fnins.2021.647783. PMID: 33867925; PMCID: PMC8044304.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Bellomo G, Cataldi S, Paciotti S, Paolini Paoletti F, Chiasserini D, Parnetti L. Measurement of CSF core Alzheimer disease biomarkers for routine clinical diagnosis: do fresh vs frozen samples differ? Alzheimers Res Ther. 2020;12(1):121. https://doi.org/10.1186/s13195-020-00689-0. PMID: 32993776; PMCID: PMC7526419.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Doecke JD, Rembach A, Villemagne VL, Varghese S, Rainey-Smith S, Sarros S, Evered LA, Fowler CJ, Pertile KK, Rumble RL, Trounson B, Taddei K, Laws SM, Macaulay SL, Bush AI, Ellis KA, Martins R, Ames D, Silbert B, Vanderstichele H, Masters CL, Darby DG, Li QX, Collins S, AIBL Research Group. Concordance between cerebrospinal fluid biomarkers with Alzheimer’s disease pathology between three independent assay platforms. J Alzheimers Dis. 2018;61(1):169–83. https://doi.org/10.3233/JAD-170128. PMID: 29171991.

    Article  CAS  PubMed  Google Scholar 

  31. Leitão MJ, Silva-Spínola A, Santana I, Olmedo V, Nadal A, Le Bastard N, Baldeiras I. Clinical validation of the Lumipulse G cerebrospinal fluid assays for routine diagnosis of Alzheimer’s disease. Alzheimers Res Ther. 2019;11(1):91. https://doi.org/10.1186/s13195-019-0550-8. PMID: 31759396; PMCID: PMC6875031.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Vecchio F, Miraglia F, Pappalettera C, Nucci L, Cacciotti A, Rossini PM. Small World derived index to distinguish Alzheimer’s type dementia and healthy subjects. Age and Ageing. 2024;53(6):afae121. https://doi.org/10.1093/ageing/afae121.

  33. Habibzadeh F, Habibzadeh P, Yadollahie M. On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Med  2016;26(3):297–307 http://dx.doi.org/10.11613/BM.2016.034.

    Article  Google Scholar 

  34. Unal I. Defining an optimal cut-point value in ROC analysis: an alternative approach. Hindawi Comput Math Methods Med. 2017:3762651. https://doi.org/10.1155/2017/3762651.

  35. Perkins NJ, Schisterman EF. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am J Epidemiol. 2006;163(7):670–5. https://doi.org/10.1093/aje/kwj063. Epub 2006 Jan 12. PMID: 16410346; PMCID: PMC1444894.

    Article  PubMed  Google Scholar 

  36. Cook NR. Quantifying the added value of new biomarkers: how and how not. Diagn Progn Res. 2018;2:14. https://doi.org/10.1186/s41512-018-0037-2.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Vickers AJ, Cronin AM, Begg CB. One statistical test is sufficient for assessing new predictive markers. BMC Med Res Methodol. 2011;11(13):1–7. [PubMed: 21208427].

    Google Scholar 

  38. Steyberg EW. Clinical prediction models. In: A practical approach to development, validation, and updating. New York: Springer; 2019.

    Google Scholar 

  39. Vittinghoff E. Regression methods in biostatistics: linear, logistic, survival, and repeated measures models. New York: Springer; 2005.

    Google Scholar 

  40. Harrel FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. J Am Med Assoc. 1982;247:2543–6.

    Article  Google Scholar 

  41. Newson RB. Comparing the predictive powers of survival models using Harrell’s C or Somers’. Stata J. 2010;10(3):339–58.

    Article  Google Scholar 

  42. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38. https://doi.org/10.1097/EDE.0b013e3181c30fb2. PMID: 20010215; PMCID: PMC3575184.

    Article  PubMed  PubMed Central  Google Scholar 

  43. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99. https://doi.org/10.1002/sim.4067. Epub 2010 Nov 30. PMID: 21225900.

    Article  PubMed  Google Scholar 

  44. Harrell FE. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. 2nd ed. New York: Springer; 2015.

    Book  Google Scholar 

  45. Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26:1364–70. [PMID: 18323559].

    Article  PubMed  Google Scholar 

  46. Quan M, Wang X, Gong M, Wang Q, Li Y, Jia J. Post-COVID cognitive dysfunction: current status and research recommendations for high risk population. Lancet Reg Health West Pac. 2023;38:100836. https://doi.org/10.1016/j.lanwpc.2023.100836. PMID: 37457901; PMCID: PMC10344681.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Pyne JD, Brickman AM. The impact of the COVID-19 pandemic on dementia risk: potential pathways to cognitive decline. Neurodegener Dis. 2021;21(1–2):1–23. https://doi.org/10.1159/000518581. Epub 2021 Jul 28. PMID: 34348321; PMCID: PMC8678181.

    Article  CAS  PubMed  Google Scholar 

  48. Cadar D. The impact of the COVID-19 pandemic on cognitive decline. Lancet Healthy Longev. 2023;4(11):e585–6. https://doi.org/10.1016/S2666-7568(23)00216-7. PMID: 37924837.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the Italian Ministry of Health and to the Italian Medicines Agency for their full sponsorship and technical support of the INTERCEPTOR Project. Members of the INTERCEPTOR network: Maurizio Belfiglio, AIFA, Rome; Cristina Muscio, IRCCS “Carlo Besta”, Milan; Davide Quaranta, “Policlinico Agostino Gemelli”, Roma; Emanuele Cassetta, AFAR, Roma; Mario Barbagallo, ALMA, Palermo; Carlo Gabelli, AOP Padova; Simona Luzzi, AOU, Ancona; Fulvio Lauretani, AOU, Parma; Innocenzo Rainero, AOU, Torino; Carlo Ferrarese, ASST, Monza; Orazio Zanetti, FBF, Brescia; Michela Marcon, “San Bortolo” Hospital, Vicenza; Flavio Mariano Nobili, “San Martino” Hospital, Genova; Giuseppe Pelliccioni, INRCA, Ancona; Sabina Capellari, ISNB, Bologna; Elena Sinforiani, IRCSS Mondino, Pavia; Gioacchino Tedeschi, Policlinico, Napoli; Carmen Gerace, “San Camillo Forlanini”, Roma; Laura Bonanni, UNICH, Chieti; Sandro Sorbi, AOU-Careggi, Firenze; Lucilla Parnetti, UniPG, Perugia.

Funding

This study is financed by AIFA extraordinary funds for the institutional activity and by Ministry of Health funds.

Author information

Authors and Affiliations

Authors

Consortia

Contributions

FL, PL, FM and MM conceived, planned and finalised the statistical analysis plan, with the contribution and review by PP, IB, AA, FS, NL, GR, SS and NV. PMR, SC, DP, PS, FT and NV contributed to the study conceptualisation and drafted study protocol. PMR is the chief investigator of the INTERCEPTOR project. AR, MC, CM, FV, FM and NC contributed to study methodology. All authors have read, amended and approved the final version of the manuscript.

Corresponding author

Correspondence to Flavia L. Lombardo.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Ethics Committee of Fondazione Policlinico Universitario Agostino Gemelli on September 27, 2018. Additionally, approval was obtained from all local ethics committees of the participating clinical centres.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. TRIPOD checklist.

Additional file 2. Variables collected at baseline.

41512_2024_172_MOESM3_ESM.docx

Additional file 3. Age-specific volume thresholds of the 5th percentile for hippocampal volume computed with FreeSurfer and ACM-Adaboost.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lombardo, F.L., Lorenzini, P., Mayer, F. et al. Development of a prediction model of conversion to Alzheimer’s disease in people with mild cognitive impairment: the statistical analysis plan of the INTERCEPTOR project. Diagn Progn Res 8, 11 (2024). https://doi.org/10.1186/s41512-024-00172-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41512-024-00172-6

Keywords