Cohen's Kappa

Cohen's kappa measures agreement between two raters assigning the same subjects to nominal categories, correcting the observed proportion of agreement for the agreement expected by chance.

Cohen의 카파는 두 평정자가 같은 대상을 명목 범주로 분류했을 때 관찰된 일치 비율에서 우연히 기대되는 일치를 보정한 일치도 지수입니다.

Two coders classified the same 120 interview excerpts into five themes; how well do they agree beyond chance?

두 명의 코더가 같은 면담 발췌문 120개를 다섯 가지 주제로 분류했는데, 우연을 넘어선 일치도가 어느 정도인가요?

Cohen's kappa compares the observed proportion of matching codes with the proportion expected if the two coders assigned categories independently at their own base rates. A value of 0 means chance-level agreement and 1 means perfect agreement, and the asymptotic standard error gives a confidence interval and a test against zero.

When to use Cohen's Kappa

• Exactly two raters classify every subject
• Categories are nominal with no ordering
• You need a chance-corrected agreement index for a coding or diagnostic study

When to use something else

• Categories are ordered, where weighted kappa credits near misses
• There are three or more raters, where Fleiss' kappa or Krippendorff's alpha applies

Assumptions and requirements

• Subjects are rated independently of one another
• The two raters work independently
• Both raters use the same fixed set of categories

What the analysis reports

• Kappa with asymptotic SE and 95% CI
• z test of kappa = 0
• Observed and chance-expected agreement
• Prevalence- and bias-adjusted kappa (PABAK)
• Rater-by-rater contingency table

How to report it

Agreement between the two raters was κ = [value], 95% CI [[value], [value]], z = [value], p = [value], with [value]% observed agreement (N = [value]).

Related methods

• Weighted Kappa
• Fleiss' Kappa
• Krippendorff's Alpha

Run Cohen's Kappa on Quorant