Random Forest

Random forest is an ensemble machine learning method that combines many decision trees, each trained on a bootstrap sample with random subsets of features, to make classification or regression predictions.

랜덤포레스트는 부트스트랩 표본과 무작위 변수 부분집합으로 각각 학습한 많은 의사결정나무를 결합하여 분류나 회귀 예측을 하는 앙상블 머신러닝 방법이다.

Which machine learning model should I start with for prediction on tabular data with nonlinear effects?

비선형 효과가 있는 표 형식 자료로 예측하려면 어떤 머신러닝 모형부터 시작해야 하나요?

Random forest averages the predictions of many decorrelated decision trees, capturing nonlinear relationships and interactions without manual specification. It fits classification or regression on tabular data and provides out-of-bag error and variable importance with little tuning.

When to use Random Forest

• Tabular data with nonlinear relationships and interactions
• Classification or regression with mixed numeric and categorical predictors
• A baseline model with little tuning and built-in out-of-bag validation

When to use something else

• Maximum predictive accuracy on large tabular data is the goal: try gradient boosting such as XGBoost
• Interpretable coefficients and inference are required: use linear or logistic regression

Assumptions and requirements

• Enough observations for the number of features
• Reasonable choice of trees, features per split and tree depth
• Class imbalance handled with weighting or resampling
• Evaluation on held-out or out-of-bag data

What the analysis reports

• Out-of-bag error or cross-validated accuracy, AUC or RMSE
• Variable importance (impurity or permutation)
• Confusion matrix for classification
• Partial dependence plots

How to report it

The random forest ([n] trees) achieved an out-of-bag accuracy of [value], and [feature] was the most important predictor by permutation importance.

Related methods

• XGBoost
• Decision Tree
• Gradient Boosting Machine

Run Random Forest on Quorant