Partial Least Squares Regression

Partial least squares regression builds latent components that maximize covariance between predictors and the outcome and regresses the outcome on them, handling many collinear predictors.

PLS(부분최소제곱) 회귀는 설명변수와 결과의 공분산이 가장 큰 잠재성분을 뽑아 그 성분으로 회귀하는 방법으로, 공선성이 큰 다수의 설명변수를 다룹니다.

I have many strongly correlated predictors, even more than observations; how can I build a regression and find which ones matter?

서로 강하게 상관된 설명변수가 매우 많을 때 회귀모형을 만들고 중요한 변수를 어떻게 고르나요?

Partial least squares regression extracts a few components that are linear combinations of the predictors chosen to covary most with the outcome, then regresses the outcome on those components. The number of components is chosen by cross-validation, and variable importance in projection (VIP) scores rank predictors.

When to use Partial Least Squares Regression

• Predictors are numerous and highly collinear, as in spectroscopy or survey batteries
• The number of predictors approaches or exceeds the number of observations
• Prediction and a ranking of predictor importance are the goals

When to use something else

• You need unbiased coefficients with standard errors for a few predictors; use multiple linear regression instead
• You want components that summarise predictors only, without reference to the outcome; use principal component regression instead

Assumptions and requirements

• The outcome depends linearly on a few latent components
• Predictors are standardized internally, so results do not depend on their units
• Cross-validation folds represent the population

What the analysis reports

• Number of components (up to 10) selected by cross-validation
• Cross-validated RMSE and Q²
• Training R²
• Coefficients on the original scale
• VIP scores for each predictor

How to report it

A partial least squares regression retained [value] components by 10-fold cross-validation (Q² = [value], RMSE = [value]); [predictor 1] and [predictor 2] had VIP scores above 1 ([value] and [value]).

Related methods

• Principal Component Regression
• Regularised Regression (Ridge, Lasso, ElasticNet)
• Multiple Linear Regression

Run Partial Least Squares Regression on Quorant