Intraclass Correlation (ICC)
Intraclass correlation (ICC) is a reliability coefficient expressing how well ratings of the same subjects agree across raters or occasions, as the share of total variance due to differences between subjects.
급내상관계수(ICC)는 같은 대상에 대한 평정이 평정자나 측정 시점에 걸쳐 얼마나 일치하는지를, 전체 분산 가운데 대상 간 차이가 차지하는 비율로 나타내는 신뢰도 계수이다.
How do I measure agreement between several raters scoring the same participants on a continuous scale?
여러 평정자가 같은 참가자를 연속 척도로 평가했을 때 일치도는 어떻게 측정하나요?
The intraclass correlation expresses inter-rater or test-retest reliability as the proportion of rating variance that reflects true differences between subjects rather than rater differences or error. The two-way random-effects, absolute-agreement form, ICC(2,1) for a single rater and ICC(2,k) for the mean of k raters, is a common choice when raters are a sample from a larger pool.
When to use Intraclass Correlation (ICC)
- Several raters score the same subjects on a continuous or interval scale
- Test-retest reliability with repeated measurements of the same subjects
- Deciding whether a single rater's score or the average of several raters is reliable enough
When to use something else
- Categorical ratings: use Cohen's or Fleiss' kappa
- Consistency of items within a scale: use Cronbach's alpha or McDonald's omega
Assumptions and requirements
- Each column is one rater or occasion and each row one subject
- Raters treated as a random sample (two-way random effects, absolute agreement)
- Complete ratings for each subject (incomplete rows are removed)
- Approximately normal ratings with constant error variance
What the analysis reports
- ICC(2,1) and ICC(2,k): two-way random effects, absolute agreement
- ICC(1,1), ICC(1,k), ICC(3,1) and ICC(3,k) for comparison
- F test and confidence interval for each form
- Number of subjects and raters
How to report it
Inter-rater reliability among [k] raters was [good], ICC(2,1) = [value], based on a two-way random-effects, absolute-agreement, single-rater model.
Related methods
Run Intraclass Correlation (ICC) on Quorant
- Data you need
- a table (CSV or Excel) with item columns (e.g. questionnaire items)
- Price
- 250 credits for one analysis up to 10,000 rows (complexity level 2); larger or wider data costs more. Optional methodologist review: 1200 credits.
- Checking
- Checked by property tests in every deployment
- Before you pay
- A free pre-analysis diagnosis checks your data and gives an exact quote; nothing is deducted until you approve it.