Development and internal validation of a model predicting patient-reported shoulder function after arthroscopic rotator cuff repair in a Swiss setting

Background Prediction models for outcomes after orthopedic surgery provide patients with evidence-based postoperative outcome expectations. Our objectives were (1) to identify prognostic factors associated with the postoperative shoulder function outcome (the Oxford Shoulder Score (OSS)) and (2) to develop and validate a prediction model for postoperative OSS. Methods Patients undergoing arthroscopic rotator cuff repair (ARCR) were prospectively documented at a Swiss orthopedic tertiary care center. The first primary ARCR in adult patients with a partial or complete rotator cuff tear were included between October 2013 and June 2021. Thirty-two potential prognostic factors were used for prediction model development. Two sets of factors identified using the knowledge from three experienced surgeons (Set 1) and Bayesian projection predictive variable selection (Set 2) were compared in terms of model performance using R squared and root-mean-squared error (RMSE) across 45 multiple imputed data sets using chained equations and complete case data. Results Multiple imputation using data from 1510 patients was performed. Set 2 retained the following factors: American Society of Anesthesiologists (ASA) classification, baseline level of depression and anxiety, baseline OSS, operation duration, tear severity, and biceps status and treatment. Apparent model performance was R-squared = 0.174 and RMSE = 7.514, dropping to R-squared = 0.156, and RMSE = 7.603 after correction for optimism. Conclusion A prediction model for patients undergoing ARCR was developed using solely baseline and operative data in order to provide patients and surgeons with individualized expectations for postoperative shoulder function outcomes. Yet, model performance should be improved before being used in clinical routine. Supplementary Information The online version contains supplementary material available at 10.1186/s41512-023-00156-y.


Background
Rotator cuff tear is one of the most frequent orthopedic disorders [1].For patients undergoing arthroscopic rotator cuff repair (ARCR), improvement of shoulder function is one of the main reasons to undergo surgery alongside reduction of shoulder pain or return to sports activities [2].
Outcomes prediction after medical interventions has become a topic of growing interest with the use of prediction models [3].Indeed, such prediction models can provide patients and surgeons with individualized and evidence-based predictions of surgery success (e.g., by predicting the postoperative shoulder function status or the probability of occurrence of adverse events), supporting healthcare in the decision-making process.Patients may have strong expectations, which should best be tailored to their own health profile and injury characteristics [4].Such prediction models require prospective and representative outcome data of high-quality and parsimonious development [5].
Despite the rising interest, it is still unclear which prognostic factors are associated with perceived shoulder function outcomes after ARCR [6][7][8][9][10][11][12].The lack of both prospectively collected data and proper methodology to develop and report multivariable prediction models have dramatically impaired the strength of the underlying evidence [13].The implementation of a local register at a tertiary care clinic in 2013 was the first step towards better documentation and understanding of health outcome data after ARCR [14].This register has laid the foundation for carrying out a large national ARCR study initiated in 2020 [15].Both initiatives are set to predict key health outcome data after ARCR such as that recently established for the occurrence of highly prevalent adverse events, such as postoperative shoulder stiffness [16].

Objectives
Our objectives are (1) to identify potential prognostic factors associated with the postoperative Oxford Shoulder Score (OSS) and (2) to develop and validate a prediction model for postoperative OSS.

Methods
The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement was used as a reporting guideline [17].

Source of data
From January 2010, patients undergoing ARCR in a Swiss tertiary orthopedic clinic were prospectively documented in a register [14].At the clinic, patient followup at 6-month post-surgery comprised various objective and subjective patient-reported outcomes including the Constant-Murley score, Oxford Shoulder Score (OSS), and subjective shoulder value (SSV).The scales were filled out on paper form or via e-mail the week after the clinical examination.An additional 2-to 4-year postoperative evaluation of patient-reported outcomes and level of satisfaction was made by postal questionnaire.All parameters were collected in clinical report forms after the baseline clinical examination or immediately after surgery.Quality checks and data management were done using a Research Electronic Data Capture (REDCap) database [18].

Participants
Adult patients were included if (1) they had a partial or complete rotator cuff tear that was assessed by magnetic resonance imaging (MRI) and confirmed intraoperatively and (2) underwent ARCR between October 2013 and June 2021.Revision surgeries were excluded as well as contralateral ARCRs in patients with bilateral injuries.

Treatment and rehabilitation
Shoulder arthroscopy was performed according to internationally standardized procedures with patients in a beach-chair position under general anesthesia [19].All patients followed a standard 3-phase postoperative physical therapy protocol involving the following: (1) 6 weeks of passive mobilization with an abduction brace (DonJoy UltraSling ER; ORMED GmbH, Freiburg, Germany), (2) 4 to 6 weeks of active-assisted mobilization and coordination training, and (3) specific progressive resistance exercises for the operated shoulder.

Outcome
For prediction modeling of 6-month outcome, we focused on the OSS, which is a 12-item patient-reported outcome assessing daily functional activities in relation to use of the shoulder [20].The OSS is a condition-specific questionnaire developed for patients with a degenerative or inflammatory shoulder condition including rotator cuff injuries.The twelve items or questions are answered by the patient independently and address the degree of pain and possible handicaps experienced during activities of daily living within the last 4 postoperative weeks.There are five response categories for each question corresponding to a score ranging from 0 to 4. All scores are then combined to produce a final score ranging from 0 (worst outcome) to 48 (best outcome).

Prognostic factors
A list of 37 prognostic factors was generated by the primary and senior authors (T.S., L. A.) based on previous systematic reviews [6][7][8][9][10][11][12].Three experienced orthopedic shoulder surgeons (A.M., M. F., M. S.) were asked to independently assess the importance of each prognostic factor using a scale ranging from 0 (not important) to 5 (very important) for the prediction of the 6-month OSS.All 37 factors were documented and available in the local register.We excluded five potential prognostic factors describing redundant information (baseline pain-related question from the Constant-Murley score, general health status, rotator cuff tear pattern, the extent of the rotator cuff tear, and the baseline pain-related question from a visual analog scale).
Thirty-two potential prognostic factors were finally retained for development of the prediction model including the following: 18 patients and disease-related parameters collected at baseline (age at surgery, sex, body mass index, American Society of Anesthesiologists (ASA) physical status classification, dominance of affected side, smoking status at surgery, three preoperative treatment variables (medication, physiotherapy, and steroid infiltrations), traumatic onset, symptom duration, level of depression and anxiety using the European Quality-of-Life 5 Dimensions 5 Level (EQ-5D-5L) scale [21] and baseline functional scales (OSS, Constant-Murley score, range of parameters (flexion, abduction, external rotation)), and muscle strength) and 14 operative findings and details collected during the surgery (supraspinatus tear, subscapularis tear, infraspinatus tear, tear severity, level of fatty infiltration, tendon degeneration, tendon delamination, operation duration, number of anchors used, number of threads used, acromioclavicular joint resection, acromioplasty, capsulotomy, and biceps tendon status and treatment).

Post hoc sample size calculation
An R package developed by Riley et al. [22] was used to estimate the necessary sample size for the development of a multivariable prediction model for continuous outcomes (pmsampsize package).The estimation required an expected R-squared value for the future prediction model of 0.2, a shrinkage factor of 0.9, a multiplicative margin of error of 1.1, the number of parameters to be assessed during the multivariable prediction model development of 32, and an average outcome value in the population of interest (i.e., intercept) of 40 with its standard deviation (SD) of 8.The necessary sample size estimated at 1199 assumed a 0.05 acceptable difference in apparent and adjusted R-squared.

Missing data
Considering our data set showed variable missing data rates and that these missing values were missing at random, multiple imputation of missing data was performed using chained equations [23] with 45 datasets based on all the available information (including the date of surgery).

Statistical analyses
All analyses were performed using R [24].

Type of model used
Linear regression models were fitted using ordinary least square estimation.Univariable regression coefficients and their 95% confidence intervals (CI) were reported and compared between multiple imputed datasets and complete case data.

Prognostic factor handling
Following the ten principles to strengthen prognosis research [25], continuous prognostic factors were kept continuous as far as possible.Based on univariable regression models, second-order polynomial transformations of continuous predictors were tested to account for the nonlinear association with postoperative OSS.For categorical variables, an attempt was made to avoid sparse categories.

Model building procedures
Two sets of factors were compared: "Set 1" regrouped the eleven factors estimated by surgeons as having the best predictive ability ( including the baseline OSS value), and "Set 2" was composed of the variables identified using Bayesian projection predictive variable selection using the projpred package [26].Five-fold cross validation was performed for variable selection based on the complete case data.The set of variables with the best predictive ability (in terms of root-mean-square error (RMSE)) was then identified.

Model performance
The predictive performance of the two sets of factors were compared in terms of R-squared and RMSE.Internal bootstrap validation was performed to estimate optimism-corrected model performance using 500 repetitions [27].Regression coefficients were presented in the main text for the model with the best optimism-corrected predictive ability.

Patient and public involvement in research statement
No patient or member of the public was involved in the design of this study.Patients directly filled out the Oxford Shoulder Score based on their own experience after the surgery.

Participants
Overall, 1555 patients were assessed for eligibility between October 2013 and June 2021.After application of our eligibility criteria, 45 patients were excluded (1 patient owed to the group IV of ASA classification; 44 were scheduled for a revision surgery) leading to a set of 1510 patients.Complete-case data were available for 712 patients.The percentage of missing values for specific variables ranged from 0 to 45.4% (Table 1).
Comparisons between thirty-two prognostic factor distributions across average multiple imputed data and available complete-case data are summarized in Table 1.Univariable associations between each candidate prognostic factor and the 6-month OSS are also reported.No prognostic factors showed nonlinear univariable associations with the postoperative outcome, even after multiple imputation.

Model development Outcome data
After multiple imputation, the median baseline OSS was 29 (interquartile range (IQR) 23,35) for the whole population and increased to 43 points (IQR 37, 48) 6 months after the surgery (Table 1).The 6-month OSS values were recorded between 4 and 11 months after the surgery.Overall, 181 patients achieved the maximum OSS value at 6 months (12%).For 167 patients (11%), the change between baseline and the 6-month postoperative OSS was below or equal to 0.

Model specification and comparison
Set 1 and Set 2 were composed of 11 and 7 prognostic factors, respectively (see Additional file 1 and Table 3).
After correction for optimism, Set 2 had better model performance than any other model after multiple imputation with R-squared (0.156) and RMSE (7.603) (Table 2).

Model presentation
For Set 2, adjusted regression coefficients were similar across multiple imputed and complete-case data (Table 3).This model included data collected during baseline examinations and operative details.

Discussion
We developed a prediction model for shoulder function 6 months after an ARCR.Lack of important prognostic factors might explain the improvable model performance.We used robust techniques for the variables  selection.Seven patients, disease, and operative factors were retained in the model showing the best model performance after multiple imputation.Our model development highlights the importance to take into account baseline and operative details when the aim is to predict postoperative course of functional status after an orthopedic intervention.Yet, the model performance suggests a model update before implementation in clinical practice, which is planned with the implementation of the ARCR_Pred data [15].With sound model performance, such models might be useful for clinicians and policymakers with a primary focus on patient-centered healthcare.Alongside individual expectations of shoulder function outcomes held by the surgeon, such models provide useful insight into the associations between prognostic factors and patientreported outcomes.This modeled evidence might then be discussed with patients during clinical examinations to forecast their possibilities of improvements after surgery based on informed consideration of a set of preestablished factors.The limitations encountered in this study can be addressed by stronger national support in the development of clinical registers, which form the basis for value-based health care in orthopedic surgery [15,16].
In prognostic research studies, associations between a given factor and an outcome have to be replicated across different studies in order to classify the factor as one with potential prognostic importance [28].In our recent systematic review, we identified that low preoperative functional status was associated with greater improvement in PROMs, which seems logical given that patients with poorer outcome scores are expected to have a greater change in outcome [29].In this case, the type of modeled outcome has substantial importance on the direction in which the associations occur.In the present work, patients with a higher preoperative OSS had better a 6-month score.Optimization of preoperative functional scores might therefore be important in order to expect better postoperative outcomes.This could be achieved by implementing a better selection of the patients undergoing an ARCR (i.e., patients below a certain score could first be optimized with preoperative non-interventional treatment).Frangiamore et al. recently developed models for various postoperative outcomes also impacted by substantial ceiling and floor effects, which they addressed with the use of proportional odds ordinal logistic regression [30].Their set comprised twelve factors based on clinical rationale (age at surgery, sex, worker's compensation status, previous cuff repair, tear size, critical shoulder angle, baseline outcome values, length of follow-up, fatty infiltration, tear shape, tendon stump length, and multiple tendon involvement).Unfortunately, this working group did not report a model performance indicator, which leaves us with a lack of information for comparison.In a recent review, we identified the need for researchers to use adequate reporting guidelines when reporting multivariable prediction models [31].

Strengths
This study followed the most recent reporting guidelines along with the availability of the code, which makes this study reproducible and transparent.We included various patient, disease, and procedurerelated factors and asked experienced surgeons to assess their predictive ability.We also implemented robust variable selection techniques, such as the Bayesian projection predictive variable selection.

Limitations
The lack of other important predictors (including postoperative management or peri-operative factors) in our model development may be partially responsible for our inability to accurately make forecasts for patients with extreme postoperative values.For instance, the occurrence of adverse events such as repair failure, neurological injury, persistent pain, or shoulder stiffness might dramatically affect our predictions.It might therefore be important to accurately identify and monitor patients with postoperative adverse events.This study also suffered from the absence of proper external validation.

Conclusions
Before being implemented in clinical practice, our prediction model should show better model performance.We believe the identification of a new set of variables using the ARCR_Pred data might help in that regard by using a variety of new parameters encompassing those of preoperative and/or peri-operative management [15].
• fast, convenient online submission • thorough peer review by experienced researchers in your field • rapid publication on acceptance • support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year

•
At BMC, research is always in progress.

Learn more biomedcentral.com/submissions
Ready to submit your research Ready to submit your research ?Choose BMC and benefit from: ? Choose BMC and benefit from:

Table 1
Study population characteristics

Table 2
Multivariable model comparison

Table 3
Linear multivariable model for Set 21Pooled regression coefficients across the multiple imputed datasets (N = 45)