Performance of binary prediction models in high-correlation low-dimensional settings: a comparison of methods

Artuur M Leeuwenberg*, Maarten van Smeden, Johannes A Langendijk, Arjen van der Schaaf, Murielle E Mauer, Karel G M Moons, Johannes B Reitsma, Ewoud Schuit

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

27 Downloads (Pure)


BACKGROUND: Clinical prediction models are developed widely across medical disciplines. When predictors in such models are highly collinear, unexpected or spurious predictor-outcome associations may occur, thereby potentially reducing face-validity of the prediction model. Collinearity can be dealt with by exclusion of collinear predictors, but when there is no a priori motivation (besides collinearity) to include or exclude specific predictors, such an approach is arbitrary and possibly inappropriate.

METHODS: We compare different methods to address collinearity, including shrinkage, dimensionality reduction, and constrained optimization. The effectiveness of these methods is illustrated via simulations.

RESULTS: In the conducted simulations, no effect of collinearity was observed on predictive outcomes (AUC, R2, Intercept, Slope) across methods. However, a negative effect of collinearity on the stability of predictor selection was found, affecting all compared methods, but in particular methods that perform strong predictor selection (e.g., Lasso). Methods for which the included set of predictors remained most stable under increased collinearity were Ridge, PCLR, LAELR, and Dropout.

CONCLUSIONS: Based on the results, we would recommend refraining from data-driven predictor selection approaches in the presence of high collinearity, because of the increased instability of predictor selection, even in relatively high events-per-variable settings. The selection of certain predictors over others may disproportionally give the impression that included predictors have a stronger association with the outcome than excluded predictors.

Original languageEnglish
Article number1
Number of pages13
JournalDiagnostic and prognostic research
Issue number1
Publication statusPublished - 11-Jan-2022

Cite this