K-Nearest Neighbors (KNN)

K-nearest neighbors (KNN) is a nonparametric supervised learning method that predicts a case's class or value from the majority vote or average of the k most similar training cases.

K-최근접 이웃(KNN)은 학습 데이터에서 가장 유사한 k개 사례의 다수결 또는 평균으로 새 사례의 범주나 값을 예측하는 비모수 지도학습 방법이다.

How do I classify cases based on similarity to labelled examples?

레이블이 있는 사례와의 유사성으로 새 사례를 어떻게 분류하나요?

K-nearest neighbors finds the k training cases closest to a new case in feature space and predicts the most common class (classification) or the average outcome (regression). It fits small to medium datasets with few, scaled numeric features where similarity in feature space is meaningful.

When to use K-Nearest Neighbors (KNN)

• Small to medium labelled datasets with numeric features
• Decision boundaries that are irregular and hard to describe with a formula
• A simple baseline model for classification or regression

When to use something else

• There are many features or irrelevant features: use regularized regression or gradient boosting
• The dataset is very large and prediction speed matters: use tree-based ensembles such as random forest

Assumptions and requirements

• Features are standardized so no variable dominates the distance
• k chosen by cross-validation
• Enough labelled cases per class, ideally balanced
• A meaningful distance metric for the features

What the analysis reports

• Cross-validated accuracy, F1, or RMSE
• Chosen value of k and distance metric
• Confusion matrix
• Performance across candidate k values

How to report it

A k-nearest neighbors classifier (k = [value]) achieved a cross-validated accuracy of [value] (SD [value]).

Related methods

• Logistic Regression
• Support Vector Machine
• Random Forest

Run K-Nearest Neighbors (KNN) on Quorant