Regularised Regression (Ridge, Lasso, ElasticNet)

Regularized regression (ridge, lasso, and elastic net) fits a linear model with a penalty on coefficient size that shrinks estimates to reduce overfitting, and the lasso and elastic net can set coefficients exactly to zero.

정규화 회귀(릿지, 라쏘, 엘라스틱넷)는 계수 크기에 벌점을 부과해 추정값을 축소함으로써 과적합을 줄이는 선형모형이며, 라쏘와 엘라스틱넷은 일부 계수를 정확히 0으로 만들 수 있습니다.

How do I build a regression model when I have many or highly correlated predictors?

예측변수가 많거나 서로 강하게 상관되어 있을 때 회귀모형은 어떻게 만드나요?

Regularized regression, such as ridge, lasso, or elastic net, adds a penalty to ordinary least squares so that coefficients shrink toward zero and the model generalizes better to new data. It fits prediction problems with many or highly correlated predictors, with the penalty strength usually chosen by cross-validation and the lasso or elastic net also performing variable selection.

When to use Regularised Regression (Ridge, Lasso, ElasticNet)

• Many predictors relative to the number of observations
• Highly correlated predictors that make ordinary least squares estimates unstable
• Prediction of a continuous outcome where automatic variable selection is useful

When to use something else

• Few predictors and the goal is inference with p-values: use multiple linear regression
• Strong nonlinearities or interactions: use random forest or gradient boosting

Assumptions and requirements

• Predictors standardized before fitting so the penalty treats them equally
• Penalty strength (lambda) and mixing parameter (alpha) tuned by cross-validation
• Approximately linear relationships between predictors and outcome
• Conventional p-values are not valid after penalized selection

What the analysis reports

• Coefficient path plot across penalty values
• Cross-validation error curve with the selected lambda
• Shrunken coefficients and the set of selected predictors
• Cross-validated or test-set RMSE and R²

How to report it

The lasso model, with λ selected by [k]-fold cross-validation, retained [value] predictors and achieved a cross-validated RMSE of [value].

Related methods

• Multiple Linear Regression
• Principal Component Regression
• Partial Least Squares Regression

Run Regularised Regression (Ridge, Lasso, ElasticNet) on Quorant