Boruta Feature Selection Algorithm

Boruta is a feature selection algorithm that compares the random forest importance of each real feature with that of randomly permuted shadow features to identify all features relevant to the outcome.

Boruta는 각 실제 변수의 랜덤포레스트 중요도를 무작위로 섞은 그림자 변수의 중요도와 비교하여 결과와 관련된 모든 변수를 찾아내는 변수선택 알고리즘이다.

How do I find all the variables that are relevant for predicting an outcome with machine learning?

머신러닝으로 결과를 예측하는 데 관련된 모든 변수를 어떻게 찾나요?

Boruta repeatedly trains random forests on the original features plus shuffled copies and keeps features whose importance consistently exceeds the best shadow feature. It fits all-relevant feature selection, where the goal is to identify every informative variable rather than a minimal predictive subset.

When to use Boruta Feature Selection Algorithm

• Many candidate features and a classification or regression outcome
• Identifying all variables with predictive signal, including weak ones
• Screening features before building an interpretable model

When to use something else

• You want a small, minimal predictor set for a linear model: use lasso regression
• You need a ranking within a final model: use SHAP values or permutation importance

Assumptions and requirements

• Enough observations to train stable random forests
• Features without leakage of the outcome
• Sufficient iterations to resolve tentative features
• Correlated features may all be marked relevant

What the analysis reports

• Decision per feature (confirmed, tentative, rejected)
• Importance distributions versus shadow features
• Importance box plot
• Number of iterations

How to report it

Boruta confirmed [n] of [total] features as relevant, rejected [n], and left [n] tentative after [value] iterations.

Related methods

• Random Forest
• Lasso Regression
• Recursive Feature Elimination

Run Boruta Feature Selection Algorithm on Quorant