Mitigating collinearity in linear regression models using ridge, surrogate and raised estimators

Abstract

Collinearity in the design matrix is a frequent problem in linear regression models, for example, with economic or medical data. Previous standard procedures to mitigate the effects of collinearity included ridge regression and surrogate regression. Ridge regression perturbs the moment matrix X'X → X'X + kIp, while surrogate regression perturbs the design matrix X → Xs. More recently, the raise estimators have been introduced, which allow the user to track geometrically the perturbation in the data with X→XX. The raise estimators are used to reduce collinearity in linear regression models by raising a column in the experimental data matrix, which may be nearly linear with the other columns, while keeping the basic OLS regression model. We give a brief overview of these three ridge-type estimators and discuss practical ways of choosing the required perturbation parameters for each procedure.