CatBoost

CatBoost is a gradient boosting algorithm on decision trees that uses ordered boosting and ordered target statistics to encode categorical features natively while reducing target leakage and overfitting.

CatBoost는 결정나무 기반 그래디언트 부스팅 알고리즘으로, 순서형 부스팅과 순서형 타깃 통계량을 사용하여 범주형 변수를 직접 인코딩하고 타깃 누수와 과적합을 줄인다.

How do I build an accurate prediction model on tabular data with many categorical variables?

범주형 변수가 많은 표 형식 데이터에서 정확한 예측모형을 어떻게 만드나요?

CatBoost builds an ensemble of symmetric decision trees with gradient boosting and handles categorical variables directly through ordered target statistics, without manual one-hot encoding. It fits tabular classification and regression problems with many or high-cardinality categorical features.

When to use CatBoost

• Tabular data with many categorical or high-cardinality features
• Classification or regression with a labelled outcome
• Wanting strong default performance with limited tuning

When to use something else

• An interpretable model with explicit coefficients is required; use logistic or linear regression
• Unstructured data such as images or text; use neural networks

Assumptions and requirements

• Labelled training data with a defined outcome
• Hold-out or cross-validation to tune iterations and learning rate
• Training and future data from a similar distribution

What the analysis reports

• Cross-validated accuracy, AUC, RMSE or similar metric
• Feature importance
• SHAP values
• Learning curves on validation data
• Confusion matrix for classification

How to report it

The CatBoost model achieved a cross-validated AUC of [value] (SD [value]).

Related methods

• XGBoost
• LightGBM
• Random Forest

Run CatBoost on Quorant