Robust Effect Sizes for Group Differences

Robust effect sizes for group differences summarise how far two groups differ using trimmed and winsorized statistics and rank-based dominance measures that resist outliers and heavy tails.

강건 효과크기는 절사평균·윈저화 분산과 순위 기반 우월 확률을 이용해 이상값과 두꺼운 꼬리에 흔들리지 않게 두 집단의 차이 크기를 나타내는 지표입니다.

My two groups have skewed scores with outliers; which effect size should I report instead of Cohen's d?

두 집단 점수가 치우쳐 있고 이상값이 있는데, Cohen's d 대신 어떤 효과크기를 보고해야 하나요?

Robust effect sizes replace means and standard deviations with trimmed means, winsorized variances or pairwise dominance so that a few extreme values cannot drive the result. Reporting the Algina–Keselman–Penfield d together with Cliff's delta and the probability of superiority, each with a bootstrap interval, gives a magnitude that stays meaningful when normality fails.

When to use Robust Effect Sizes for Group Differences

• Two independent groups are compared on a continuous or ordinal outcome with outliers or heavy tails.
• A Yuen trimmed-means test or a Mann–Whitney test is reported and needs a matching effect size.
• Reviewers ask for an effect size with a confidence interval that does not assume normality.

When to use something else

• The same people are measured twice; use a paired effect size such as the paired-samples d or a matched-pairs rank-biserial correlation.
• More than two groups are compared at once; use an omnibus effect size such as robust explanatory power from a trimmed-means ANOVA.

Assumptions and requirements

• Observations are independent within and between the two groups.
• The outcome is at least ordinal; AKP d and xi additionally need an interval scale.
• Percentile bootstrap intervals need moderate group sizes (about 10 or more per group).

What the analysis reports

• AKP robust d (trimmed means over pooled winsorized SD, rescaled to the Cohen d metric) with bootstrap CI
• Wilcox–Tian explanatory measure xi with bootstrap CI
• Cliff's delta with bootstrap CI
• Probability of superiority A (Vargha–Delaney) with bootstrap CI
• Trimmed means of both groups

How to report it

The robust standardized difference was d = [value], 95% bootstrap CI [[value], [value]] (20% trimming); Cliff's delta = [value], 95% CI [[value], [value]], probability of superiority A = [value].

Related methods

• Yuen's Trimmed Means t Test
• Mann-Whitney U Test
• Cohen's d

Run Robust Effect Sizes for Group Differences on Quorant