Stepwise Regression Analysis

Stepwise regression is a variable selection procedure that adds or removes predictors from a regression model one at a time based on a criterion such as a p-value threshold or AIC.

단계적 회귀분석은 p값 기준이나 AIC 같은 기준에 따라 회귀모형에 예측변수를 하나씩 추가하거나 제거하는 변수선택 절차이다.

How do I automatically choose which predictors to keep in a regression model?

회귀모형에 어떤 예측변수를 남길지 자동으로 고르려면 어떻게 하나요?

Stepwise regression builds a model by adding predictors forward, removing them backward, or both, according to significance or information criteria. It fits exploratory screening of many candidate predictors, but its p-values and coefficients are biased after selection, so results should be validated on new data.

When to use Stepwise Regression Analysis

• Exploratory screening of many candidate predictors for a continuous outcome
• Building a parsimonious prediction model when theory gives little guidance
• Comparing forward, backward and bidirectional selection paths

When to use something else

• Predictors are chosen by theory: use hierarchical (blockwise) regression
• Many correlated predictors or more stable selection is needed: use lasso or elastic net regression

Assumptions and requirements

• Assumptions of linear regression (linearity, independence, homoscedasticity, normal residuals)
• Adequate sample size relative to the number of candidate predictors
• No severe multicollinearity

What the analysis reports

• Selected predictors and entry/removal steps
• Regression coefficients and standard errors
• R² and adjusted R² at each step
• AIC or F-change statistics

How to report it

Stepwise selection retained [k] predictors, and the final model explained [value]% of the variance in [outcome], F([df1], [df2]) = [value], p = [value].

Related methods

• Multiple Linear Regression
• Lasso Regression
• Hierarchical Regression

Run Stepwise Regression Analysis on Quorant