Equivalence Test (TOST)

The two one-sided tests (TOST) procedure tests whether a difference between two group means lies within prespecified equivalence bounds by rejecting both one-sided nulls at the bounds.

TOST 동등성 검정은 두 집단 평균 차이가 미리 정한 동등성 구간 안에 있는지를 하한과 상한에 대한 두 개의 단측 검정으로 판정하는 방법이다.

My t test was not significant; can I conclude that the two groups are the same?

t 검정에서 차이가 유의하지 않으면 두 집단이 같다고 말해도 되나요?

No, a non-significant difference is not evidence of equivalence. TOST tests the difference against a lower and an upper equivalence bound with two one-sided t tests, and equivalence is concluded only when both are rejected, which is the same as the 90% confidence interval lying inside the bounds.

When to use Equivalence Test (TOST)

• You want to show that two treatments or groups are practically equivalent.
• A smallest effect size of interest can be justified in advance.
• You want to complement a non-significant difference test.

When to use something else

• You want to show one treatment is not worse by more than a margin; use a non-inferiority test instead.
• The bounds cannot be justified; report the confidence interval of the difference instead.

Assumptions and requirements

• Observations are independent.
• Each group is approximately normal or large enough for the t approximation.
• Equivalence bounds are set before seeing the data; if none are given, ±0.5 pooled SD is used.

What the analysis reports

• Mean difference with 90% confidence interval
• Lower and upper one-sided t statistics and p-values
• Overall TOST p-value
• Welch degrees of freedom

How to report it

A Welch TOST procedure with equivalence bounds of [low] and [high] showed that the difference ([value], 90% CI [[value], [value]]) was [within/not within] the bounds, t([df]) = [value] and [value], p = [value].

Related methods

• Welch's t-test
• Independent Samples t test
• Bayesian t Test (Bayes Factor)

Run Equivalence Test (TOST) on Quorant