XGBoost

XGBoost (extreme gradient boosting) is a gradient-boosted decision tree algorithm that builds trees sequentially, each correcting the errors of the previous ones, with regularization to limit overfitting.

XGBoost(익스트림 그래디언트 부스팅)는 이전 나무의 오차를 보정하는 나무를 순차적으로 추가하고 정규화로 과적합을 제한하는 그래디언트 부스팅 의사결정나무 알고리즘이다.

How do I get high predictive accuracy on a tabular dataset for classification or regression?

표 형식 자료로 분류나 회귀에서 높은 예측 정확도를 얻으려면 어떻게 하나요?

XGBoost fits an ensemble of shallow trees by gradient boosting, adding L1 and L2 regularization, shrinkage and subsampling to control overfitting. It fits structured tabular data where predictive accuracy is the main goal, provided hyperparameters are tuned with cross-validation.

When to use XGBoost

• Tabular classification or regression where accuracy matters most
• Large datasets with nonlinear effects, interactions and missing values
• Structured business, health or survey data with many features

When to use something else

• Small samples or a need for interpretable coefficients: use logistic or linear regression
• Little tuning time and a robust baseline are needed: use random forest

Assumptions and requirements

• Hyperparameters tuned by cross-validation (learning rate, depth, number of trees)
• Held-out test data or early stopping to avoid overfitting
• Class imbalance handled with weights or appropriate metrics
• No leakage of outcome information into features

What the analysis reports

• Cross-validated performance (AUC, accuracy, RMSE)
• Learning curves and selected number of boosting rounds
• Feature importance (gain)
• SHAP values

How to report it

The XGBoost model achieved a cross-validated AUC of [value] (SD [value]), and SHAP analysis identified [feature] as the strongest predictor.

Related methods

• Random Forest
• LightGBM
• CatBoost

Run XGBoost on Quorant