Robust Regression Analysis

Robust regression analysis is a family of regression methods, such as M-estimation, that downweight outliers and influential observations so coefficient estimates are less distorted than ordinary least squares.

로버스트 회귀분석은 M-추정 등으로 이상치와 영향점의 가중치를 낮추어 최소제곱법보다 계수 추정이 덜 왜곡되도록 하는 회귀분석 방법군이다.

How do I run a regression when outliers are distorting the results?

이상치가 회귀분석 결과를 왜곡할 때 어떻게 분석하나요?

Robust regression estimates a linear model using loss functions such as Huber or bisquare weights that reduce the influence of large residuals, instead of minimizing squared errors. It fits when the data contain outliers or heavy-tailed errors that would unduly pull ordinary least squares estimates.

When to use Robust Regression Analysis

• Residual diagnostics show outliers or heavy-tailed errors in a linear model
• A few influential cases change OLS coefficients substantially
• Continuous outcome where outliers are genuine and should not simply be deleted

When to use something else

• Only the standard errors are a concern due to heteroscedasticity: use OLS with heteroscedasticity-consistent (robust) standard errors
• Interest lies in conditional medians or other quantiles: use quantile regression

Assumptions and requirements

• The linear model form is correctly specified
• Outliers are a minority of observations
• Independent observations
• M-estimators handle outliers in the outcome; high-leverage outliers in predictors need MM- or other high-breakdown estimators

What the analysis reports

• Robust coefficient estimates with standard errors and confidence intervals
• Case weights showing which observations were downweighted
• Comparison with OLS estimates
• Residual scale estimate

How to report it

Robust regression with Huber M-estimation showed that [predictor] was associated with [outcome], b = [value], SE = [value], 95% CI [[value], [value]].

Related methods

• Multiple Linear Regression
• Quantile Regression
• Heteroscedasticity-Consistent Standard Errors

Run Robust Regression Analysis on Quorant