Weighted Kappa

Weighted kappa extends Cohen's kappa to ordered categories by giving partial credit to disagreements according to their distance, usually with linear or quadratic weights.

가중 카파는 순서가 있는 범주에서 두 평정자의 불일치를 범주 간 거리에 따라 부분적으로 인정하는 Cohen 카파의 확장으로, 보통 선형 또는 이차 가중치를 씁니다.

Two clinicians graded 80 wounds on a 1–5 severity scale; how consistent are their grades when near misses count partly?

두 명의 임상의가 상처 80개의 중증도를 1~5점으로 평가했는데, 한 칸 차이는 덜 문제 삼는다면 두 사람의 평가가 얼마나 일치하나요?

Weighted kappa compares weighted observed agreement with weighted chance agreement, so a one-step disagreement is penalised less than a four-step one. Quadratic weights are the common default and make the coefficient closely related to the intraclass correlation.

When to use Weighted Kappa

• Two raters score subjects on an ordinal scale
• Near disagreements should count less than distant ones
• You want an agreement index comparable to an intraclass correlation

When to use something else

• Categories have no order, where unweighted Cohen's kappa is appropriate
• Scores are continuous measurements, where Lin's concordance correlation or Bland–Altman analysis fits better

Assumptions and requirements

• Exactly two independent raters
• Categories are ordered and treated as equally spaced
• Subjects are independent

What the analysis reports

• Weighted kappa with asymptotic SE and 95% CI
• z test of weighted kappa = 0
• Weighting scheme used (linear or quadratic)
• Weighted observed and expected agreement
• Rater-by-rater contingency table

How to report it

Agreement between the two raters with quadratic weights was κw = [value], 95% CI [[value], [value]], z = [value], p = [value] (N = [value]).

Related methods

• Cohen's Kappa
• Intraclass Correlation (ICC)
• Lin's Concordance Correlation Coefficient

Run Weighted Kappa on Quorant