Skip to main content

Development and internal validation of a model predicting patient-reported shoulder function after arthroscopic rotator cuff repair in a Swiss setting



Prediction models for outcomes after orthopedic surgery provide patients with evidence-based postoperative outcome expectations. Our objectives were (1) to identify prognostic factors associated with the postoperative shoulder function outcome (the Oxford Shoulder Score (OSS)) and (2) to develop and validate a prediction model for postoperative OSS.


Patients undergoing arthroscopic rotator cuff repair (ARCR) were prospectively documented at a Swiss orthopedic tertiary care center. The first primary ARCR in adult patients with a partial or complete rotator cuff tear were included between October 2013 and June 2021. Thirty-two potential prognostic factors were used for prediction model development. Two sets of factors identified using the knowledge from three experienced surgeons (Set 1) and Bayesian projection predictive variable selection (Set 2) were compared in terms of model performance using R squared and root-mean-squared error (RMSE) across 45 multiple imputed data sets using chained equations and complete case data.


Multiple imputation using data from 1510 patients was performed. Set 2 retained the following factors: American Society of Anesthesiologists (ASA) classification, baseline level of depression and anxiety, baseline OSS, operation duration, tear severity, and biceps status and treatment. Apparent model performance was R-squared = 0.174 and RMSE = 7.514, dropping to R-squared = 0.156, and RMSE = 7.603 after correction for optimism.


A prediction model for patients undergoing ARCR was developed using solely baseline and operative data in order to provide patients and surgeons with individualized expectations for postoperative shoulder function outcomes. Yet, model performance should be improved before being used in clinical routine.

Peer Review reports


Rotator cuff tear is one of the most frequent orthopedic disorders [1]. For patients undergoing arthroscopic rotator cuff repair (ARCR), improvement of shoulder function is one of the main reasons to undergo surgery alongside reduction of shoulder pain or return to sports activities [2].

Outcomes prediction after medical interventions has become a topic of growing interest with the use of prediction models [3]. Indeed, such prediction models can provide patients and surgeons with individualized and evidence-based predictions of surgery success (e.g., by predicting the postoperative shoulder function status or the probability of occurrence of adverse events), supporting healthcare in the decision-making process. Patients may have strong expectations, which should best be tailored to their own health profile and injury characteristics [4]. Such prediction models require prospective and representative outcome data of high-quality and parsimonious development [5].

Despite the rising interest, it is still unclear which prognostic factors are associated with perceived shoulder function outcomes after ARCR [6,7,8,9,10,11,12]. The lack of both prospectively collected data and proper methodology to develop and report multivariable prediction models have dramatically impaired the strength of the underlying evidence [13]. The implementation of a local register at a tertiary care clinic in 2013 was the first step towards better documentation and understanding of health outcome data after ARCR [14]. This register has laid the foundation for carrying out a large national ARCR study initiated in 2020 [15]. Both initiatives are set to predict key health outcome data after ARCR such as that recently established for the occurrence of highly prevalent adverse events, such as postoperative shoulder stiffness [16].


Our objectives are (1) to identify potential prognostic factors associated with the postoperative Oxford Shoulder Score (OSS) and (2) to develop and validate a prediction model for postoperative OSS.


The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement was used as a reporting guideline [17].

Source of data

From January 2010, patients undergoing ARCR in a Swiss tertiary orthopedic clinic were prospectively documented in a register [14]. At the clinic, patient follow-up at 6-month post-surgery comprised various objective and subjective patient-reported outcomes including the Constant-Murley score, Oxford Shoulder Score (OSS), and subjective shoulder value (SSV). The scales were filled out on paper form or via e-mail the week after the clinical examination. An additional 2- to 4-year postoperative evaluation of patient-reported outcomes and level of satisfaction was made by postal questionnaire. All parameters were collected in clinical report forms after the baseline clinical examination or immediately after surgery. Quality checks and data management were done using a Research Electronic Data Capture (REDCap) database [18].


Adult patients were included if (1) they had a partial or complete rotator cuff tear that was assessed by magnetic resonance imaging (MRI) and confirmed intraoperatively and (2) underwent ARCR between October 2013 and June 2021. Revision surgeries were excluded as well as contralateral ARCRs in patients with bilateral injuries.

Treatment and rehabilitation

Shoulder arthroscopy was performed according to internationally standardized procedures with patients in a beach-chair position under general anesthesia [19]. All patients followed a standard 3-phase postoperative physical therapy protocol involving the following: (1) 6 weeks of passive mobilization with an abduction brace (DonJoy UltraSling ER; ORMED GmbH, Freiburg, Germany), (2) 4 to 6 weeks of active-assisted mobilization and coordination training, and (3) specific progressive resistance exercises for the operated shoulder.


For prediction modeling of 6-month outcome, we focused on the OSS, which is a 12-item patient-reported outcome assessing daily functional activities in relation to use of the shoulder [20]. The OSS is a condition-specific questionnaire developed for patients with a degenerative or inflammatory shoulder condition including rotator cuff injuries. The twelve items or questions are answered by the patient independently and address the degree of pain and possible handicaps experienced during activities of daily living within the last 4 postoperative weeks. There are five response categories for each question corresponding to a score ranging from 0 to 4. All scores are then combined to produce a final score ranging from 0 (worst outcome) to 48 (best outcome).

Prognostic factors

A list of 37 prognostic factors was generated by the primary and senior authors (T. S., L. A.) based on previous systematic reviews [6,7,8,9,10,11,12]. Three experienced orthopedic shoulder surgeons (A. M., M. F., M. S.) were asked to independently assess the importance of each prognostic factor using a scale ranging from 0 (not important) to 5 (very important) for the prediction of the 6-month OSS. All 37 factors were documented and available in the local register. We excluded five potential prognostic factors describing redundant information (baseline pain-related question from the Constant-Murley score, general health status, rotator cuff tear pattern, the extent of the rotator cuff tear, and the baseline pain-related question from a visual analog scale).

Thirty-two potential prognostic factors were finally retained for development of the prediction model including the following: 18 patients and disease-related parameters collected at baseline (age at surgery, sex, body mass index, American Society of Anesthesiologists (ASA) physical status classification, dominance of affected side, smoking status at surgery, three preoperative treatment variables (medication, physiotherapy, and steroid infiltrations), traumatic onset, symptom duration, level of depression and anxiety using the European Quality-of-Life 5 Dimensions 5 Level (EQ-5D-5L) scale [21] and baseline functional scales (OSS, Constant-Murley score, range of parameters (flexion, abduction, external rotation)), and muscle strength) and 14 operative findings and details collected during the surgery (supraspinatus tear, subscapularis tear, infraspinatus tear, tear severity, level of fatty infiltration, tendon degeneration, tendon delamination, operation duration, number of anchors used, number of threads used, acromioclavicular joint resection, acromioplasty, capsulotomy, and biceps tendon status and treatment).

Post hoc sample size calculation

An R package developed by Riley et al. [22] was used to estimate the necessary sample size for the development of a multivariable prediction model for continuous outcomes (pmsampsize package). The estimation required an expected R-squared value for the future prediction model of 0.2, a shrinkage factor of 0.9, a multiplicative margin of error of 1.1, the number of parameters to be assessed during the multivariable prediction model development of 32, and an average outcome value in the population of interest (i.e., intercept) of 40 with its standard deviation (SD) of 8. The necessary sample size estimated at 1199 assumed a 0.05 acceptable difference in apparent and adjusted R-squared.

Missing data

Considering our data set showed variable missing data rates and that these missing values were missing at random, multiple imputation of missing data was performed using chained equations [23] with 45 datasets based on all the available information (including the date of surgery).

Statistical analyses

All analyses were performed using R [24].

Type of model used

Linear regression models were fitted using ordinary least square estimation. Univariable regression coefficients and their 95% confidence intervals (CI) were reported and compared between multiple imputed datasets and complete case data.

Prognostic factor handling

Following the ten principles to strengthen prognosis research [25], continuous prognostic factors were kept continuous as far as possible. Based on univariable regression models, second-order polynomial transformations of continuous predictors were tested to account for the nonlinear association with postoperative OSS. For categorical variables, an attempt was made to avoid sparse categories.

Model building procedures

Two sets of factors were compared: “Set 1” regrouped the eleven factors estimated by surgeons as having the best predictive ability ( including the baseline OSS value), and “Set 2” was composed of the variables identified using Bayesian projection predictive variable selection using the projpred package [26]. Five-fold cross validation was performed for variable selection based on the complete case data. The set of variables with the best predictive ability (in terms of root-mean-square error (RMSE)) was then identified.

Model performance

The predictive performance of the two sets of factors were compared in terms of R-squared and RMSE. Internal bootstrap validation was performed to estimate optimism-corrected model performance using 500 repetitions [27]. Regression coefficients were presented in the main text for the model with the best optimism-corrected predictive ability.

Patient and public involvement in research statement

No patient or member of the public was involved in the design of this study. Patients directly filled out the Oxford Shoulder Score based on their own experience after the surgery.



Overall, 1555 patients were assessed for eligibility between October 2013 and June 2021. After application of our eligibility criteria, 45 patients were excluded (1 patient owed to the group IV of ASA classification; 44 were scheduled for a revision surgery) leading to a set of 1510 patients. Complete-case data were available for 712 patients. The percentage of missing values for specific variables ranged from 0 to 45.4% (Table 1).

Table 1 Study population characteristics

Comparisons between thirty-two prognostic factor distributions across average multiple imputed data and available complete-case data are summarized in Table 1. Univariable associations between each candidate prognostic factor and the 6-month OSS are also reported. No prognostic factors showed nonlinear univariable associations with the postoperative outcome, even after multiple imputation.

Model development

Outcome data

After multiple imputation, the median baseline OSS was 29 (interquartile range (IQR) 23, 35) for the whole population and increased to 43 points (IQR 37, 48) 6 months after the surgery (Table 1). The 6-month OSS values were recorded between 4 and 11 months after the surgery. Overall, 181 patients achieved the maximum OSS value at 6 months (12%). For 167 patients (11%), the change between baseline and the 6-month postoperative OSS was below or equal to 0.

Model specification and comparison

Set 1 and Set 2 were composed of 11 and 7 prognostic factors, respectively (see Additional file 1 and Table 3). After correction for optimism, Set 2 had better model performance than any other model after multiple imputation with R-squared (0.156) and RMSE (7.603) (Table 2).

Table 2 Multivariable model comparison

Model presentation

For Set 2, adjusted regression coefficients were similar across multiple imputed and complete-case data (Table 3). This model included data collected during baseline examinations and operative details.

Table 3 Linear multivariable model for Set 2


We developed a prediction model for shoulder function 6 months after an ARCR. Lack of important prognostic factors might explain the improvable model performance. We used robust techniques for the variables selection. Seven patients, disease, and operative factors were retained in the model showing the best model performance after multiple imputation. Our model development highlights the importance to take into account baseline and operative details when the aim is to predict postoperative course of functional status after an orthopedic intervention. Yet, the model performance suggests a model update before implementation in clinical practice, which is planned with the implementation of the ARCR_Pred data [15].

With sound model performance, such models might be useful for clinicians and policymakers with a primary focus on patient-centered healthcare. Alongside individual expectations of shoulder function outcomes held by the surgeon, such models provide useful insight into the associations between prognostic factors and patient-reported outcomes. This modeled evidence might then be discussed with patients during clinical examinations to forecast their possibilities of improvements after surgery based on informed consideration of a set of pre-established factors. The limitations encountered in this study can be addressed by stronger national support in the development of clinical registers, which form the basis for value-based health care in orthopedic surgery [15, 16].

In prognostic research studies, associations between a given factor and an outcome have to be replicated across different studies in order to classify the factor as one with potential prognostic importance [28]. In our recent systematic review, we identified that low preoperative functional status was associated with greater improvement in PROMs, which seems logical given that patients with poorer outcome scores are expected to have a greater change in outcome [29]. In this case, the type of modeled outcome has substantial importance on the direction in which the associations occur. In the present work, patients with a higher preoperative OSS had better a 6-month score. Optimization of preoperative functional scores might therefore be important in order to expect better postoperative outcomes. This could be achieved by implementing a better selection of the patients undergoing an ARCR (i.e., patients below a certain score could first be optimized with preoperative non-interventional treatment).

Frangiamore et al. recently developed models for various postoperative outcomes also impacted by substantial ceiling and floor effects, which they addressed with the use of proportional odds ordinal logistic regression [30]. Their set comprised twelve factors based on clinical rationale (age at surgery, sex, worker’s compensation status, previous cuff repair, tear size, critical shoulder angle, baseline outcome values, length of follow-up, fatty infiltration, tear shape, tendon stump length, and multiple tendon involvement). Unfortunately, this working group did not report a model performance indicator, which leaves us with a lack of information for comparison. In a recent review, we identified the need for researchers to use adequate reporting guidelines when reporting multivariable prediction models [31].


This study followed the most recent reporting guidelines along with the availability of the code, which makes this study reproducible and transparent. We included various patient, disease, and procedure-related factors and asked experienced surgeons to assess their predictive ability. We also implemented robust variable selection techniques, such as the Bayesian projection predictive variable selection.


The lack of other important predictors (including postoperative management or peri-operative factors) in our model development may be partially responsible for our inability to accurately make forecasts for patients with extreme postoperative values. For instance, the occurrence of adverse events such as repair failure, neurological injury, persistent pain, or shoulder stiffness might dramatically affect our predictions. It might therefore be important to accurately identify and monitor patients with postoperative adverse events. This study also suffered from the absence of proper external validation.


Before being implemented in clinical practice, our prediction model should show better model performance. We believe the identification of a new set of variables using the ARCR_Pred data might help in that regard by using a variety of new parameters encompassing those of preoperative and/or peri-operative management [15].

Availability of data and materials

The datasets and code used during the current study are available from the corresponding author on reasonable request.



Akaike’s information criterion


Arthroscopic rotator cuff repair


Oxford Shoulder Score


Root-mean-squared error


Standard deviation


Transparent reporting


  1. Urwin M, Symmons D, Allison T, Brammah T, Busby H, Roxby M, Simmons A, Williams G. Estimating the burden of musculoskeletal disorders in the community: the comparative prevalence of symptoms at different anatomical sites, and the relation to social deprivation. Ann Rheum Dis. 1998;57(11):649–55.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Warth RJ, Briggs KK, Dornan GJ, Horan MP, Millett PJ. Patient expectations before arthroscopic shoulder surgery: correlation with patients’ reasons for seeking treatment. J Shoulder Elbow Surg. 2013;22(12):1676–81.

    Article  PubMed  Google Scholar 

  3. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KG, Steyerberg EW, et al. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013;346:e5595.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Riley RD, and others (eds). Prognosis Research in Healthcare: Concepts, Methods, and Impact (Oxford, 2019; online edn, Oxford Academic, 1 Feb. 2019). Accessed 15 Sept 2023.

  5. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–31.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Fermont AJ, Wolterbeek N, Wessel RN, Baeyens JP, de Bie RA. Prognostic factors for successful recovery after arthroscopic rotator cuff repair: a systematic literature review. J Orthop Sports Phys Ther. 2014;44(3):153–63.

    Article  PubMed  Google Scholar 

  7. Khair MM, Lehman J, Tsouris N, Gulotta LV. A systematic review of preoperative fatty infiltration and rotator cuff outcomes. HSS J. 2016;12(2):170–6.

    Article  PubMed  Google Scholar 

  8. Lambers Heerspink FO, Dorrestijn O, van Raay JJ, Diercks RL. Specific patient-related prognostic factors for rotator cuff repair: a systematic review. J Shoulder Elbow Surg. 2014;23(7):1073–80.

    Article  PubMed  Google Scholar 

  9. McElvany MD, McGoldrick E, Gee AO, Neradilek MB, Matsen FA 3rd. Rotator cuff repair: published evidence on factors associated with repair integrity and clinical outcome. Am J Sports Med. 2015;43(2):491–500.

    Article  PubMed  Google Scholar 

  10. Muller AM, Flury M, Alsayed HN, Audige L. Influence of patient and diagnostic parameters on reported retear rates after arthroscopic rotator cuff repair. Knee Surg Sports Traumatol Arthrosc. 2017;25(7):2089–99.

    Article  PubMed  Google Scholar 

  11. Raman J, Walton D, MacDermid JC, Athwal GS. Predictors of outcomes after rotator cuff repair-a meta-analysis. J Hand Ther. 2017;30(3):276–92.

    Article  PubMed  Google Scholar 

  12. Saccomanno MF, Sircana G, Cazzato G, Donati F, Randelli P, Milano G. Prognostic factors influencing the outcome of rotator cuff repair: a systematic review. Knee Surg Sports Traumatol Arthrosc. 2016;24(12):3809–19.

    Article  PubMed  Google Scholar 

  13. Baum C, Müller A, Audigé L, Stojanov T. Prognostische Faktoren der arthroskopischen Rotatorenmanschettenrekonstruktion. Arthroskopie. 2021;34(3):179–84.

    Article  Google Scholar 

  14. Flury M, Kolling C, Grobet C, Kunz SN, Audigé L. Implementation of a local outcome register for arthroscopic rotator cuff tear repair. Obere Extremität. 2015;10(1):33–40.

    Article  Google Scholar 

  15. Audige L, Bucher HCC, Aghlmandi S, Stojanov T, Schwappach D, Hunziker S, Candrian C, Cunningham G, Durchholz H, Eid K, et al. Swiss-wide multicentre evaluation and prediction of core outcomes in arthroscopic rotator cuff repair: protocol for the ARCR_Pred cohort study. BMJ Open. 2021;11(4):e045702.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Audigé L, Aghlmandi S, Grobet C, Stojanov T, Müller AM, Felsch Q, Gleich J, Flury M, Scheibel M. Prediction of shoulder stiffness after arthroscopic rotator cuff repair. Am J Sports Med. 2021;49(11):3030–9.

    Article  PubMed  Google Scholar 

  17. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement. Br J Surg. 2015;102(3):148–58.

    Article  CAS  PubMed  Google Scholar 

  18. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    Article  PubMed  Google Scholar 

  19. Imhoff AB, Ticker JB, Fu FH. Atlas of Shoulder Arthroscopy (1st ed.). CRC Press; 2003.

  20. Dawson J, Rogers K, Fitzpatrick R, Carr A. The Oxford Shoulder Score revisited. Arch Orthop Trauma Surg. 2009;129(1):119–23.

    Article  PubMed  Google Scholar 

  21. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, Bonsel G, Badia X. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727–36.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Riley RD, Snell KIE, Ensor J, Burke DL, Harrell FE Jr, Moons KGM, Collins GS. Minimum sample size for developing a multivariable prediction model: part I–continuous outcomes. Stat Med. 2019;38(7):1262–75.

    Article  PubMed  Google Scholar 

  23. Van Buuren S, Groothuis-Oudshoorn K. Mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45:1–67.

    Article  Google Scholar 

  24. Team RC. R: a language and environment for statistical computing. 2013.

    Google Scholar 

  25. Riley RD, van der Windt D, Croft P, Moons KG. Ten principles to strengthen prognosis research. In: Riley RD, van der, editors. Prognosis research in healthcare: concepts, methods, and impact. Windt D, Croft P, Moons KG: Oxford University Press; 2019. p. 69–84.

    Chapter  Google Scholar 

  26. Piironen J, Paasiniemi M, Vehtari A. Projective inference in high-dimensional problems: prediction and feature selection. 2020.

    Google Scholar 

  27. Harrell FE, Jr. Regression modeling strategies. Springer International Publishing; 2016.

  28. Huguet A, Hayden JA, Stinson J, McGrath PJ, Chambers CT, Tougas ME, Wozney L. Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework. Syst Rev. 2013;2(1):71.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Stojanov T, Laurent A, Modler L, Aghlmandi S, Appenzeller-Herzog C, Loucas R, Loucas M, Müller AM. Prognostic factors for improvement of shoulder function after arthroscopic rotator cuff repair: a systematic review. JSES Int. 2022;7:50–7.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Frangiamore S, Dornan GJ, Horan MP, Mannava S, Fritz EM, Hussain ZB, Moatshe G, Godin JA, Pogorzelski J, Millett PJ. Predictive modeling to determine functional outcomes after arthroscopic rotator cuff repair. Am J Sports Med. 2020;48(7):1559–67.

    Article  PubMed  Google Scholar 

  31. Stojanov T, Modler L, Muller AM, Aghlmandi S, Appenzeller-Herzog C, Loucas R, Loucas M, Audige L. Prognostic factors for the occurrence of post-operative shoulder stiffness after arthroscopic rotator cuff repair: a systematic review. BMC Musculoskelet Disord. 2022;23(1):99.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


The authors would like to acknowledge the support of Dr. Melissa Wilhelmi, medical writer at Schulthess Klinik, Zurich, Switzerland, for manuscript proofreading.


Open access funding provided by University of Basel This project was supported by a Swiss National Science Foundation grant (grant number: 184959) and by Schulthess Klinik and the University Hospital of Basel with regard to respective personal salaries.

Author information

Authors and Affiliations



LA and AMM were the initiators and project leaders. LA, AMM, TS, and SA were involved in the study design. TS performed the analysis and prepared the manuscript along with SA, LA, and AMM. AMM, MS, and MF were involved in the assessment of prognostic factors. AMM provided clinical expertise for the interpretation of the results. All authors reviewed and approved the last version of the manuscript.

Corresponding author

Correspondence to Thomas Stojanov.

Ethics declarations

Ethics approval and consent to participate

The rotator cuff register implementation and analysis were approved by the cantonal ethics committee of Zürich (KEK-ZH no. 2014–0253). All procedures performed in studies involving human participants were in accordance with the ethical standards dictated by the local cantonal research committee. Informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable.

Competing interests

One or more of the authors have declared the following potential conflicts of interest or sources of funding: M. F. and M. S. have received consulting fees from Arthrex.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Linear multivariable model for Set 1.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stojanov, T., Aghlmandi, S., Müller, A.M. et al. Development and internal validation of a model predicting patient-reported shoulder function after arthroscopic rotator cuff repair in a Swiss setting. Diagn Progn Res 7, 21 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: