Permutation Test for Correlation and Regression

The permutation test for correlation and regression obtains p-values for a correlation or each partial regression coefficient by recomputing the t statistic over random rearrangements of the outcome or of reduced-model residuals.

상관이나 회귀계수의 t 통계량을 결과 변수 또는 축소모형 잔차를 무작위로 재배열해 다시 계산하고, 그 분포에서 p 값을 구하는 비모수 검정입니다.

My regression residuals are clearly non-normal and the sample is small. How can I test whether each coefficient differs from zero without relying on the t distribution?

잔차가 정규분포를 따르지 않는데 표본도 작아요. t 분포에 기대지 않고 회귀계수가 0인지 검정하려면 어떻게 하나요?

Use a permutation test: with one predictor the outcome is shuffled against the predictor, and with several predictors the Freedman–Lane scheme shuffles the residuals of the model without the tested variable and adds them back to its fitted values. The p-value is the share of permutations whose |t| is at least as large as the observed |t|, so it does not need normally distributed errors.

When to use Permutation Test for Correlation and Regression

• Errors are skewed or heavy-tailed and the sample is too small to trust large-sample approximations.
• You want an exact or Monte Carlo p-value for a Pearson correlation.
• You need tests of individual partial coefficients in a linear model with a few covariates.

When to use something else

• Observations are clustered or serially dependent, because rows are not exchangeable; use a wild cluster bootstrap or time-series methods instead.
• You need a confidence interval rather than a test; use a bootstrap confidence interval instead.

Assumptions and requirements

• Observations are independent.
• Under the null hypothesis the (reduced-model) residuals are exchangeable.
• The linear model is an adequate description of the mean.

What the analysis reports

• Coefficient, standard error and t for each predictor
• Parametric p-value next to the permutation p-value
• Number of permutations and whether all permutations were enumerated
• Pearson r when there is one predictor

How to report it

Using Freedman–Lane permutation of the outcome ([value] permutations), the partial coefficient for [predictor] was b = [value], t = [value], permutation p = [value].

Related methods

• Permutation Test for Two Groups
• Bootstrap Confidence Intervals (BCa)
• Robust Correlation Coefficients

Run Permutation Test for Correlation and Regression on Quorant