Bayesian Linear Regression

Bayesian linear regression places a prior on the regression coefficients and error variance and reports their posterior distribution, here with a conjugate normal–inverse-gamma prior that gives closed-form posteriors.

베이즈 선형회귀는 회귀계수와 오차분산에 사전분포를 두고 사후분포를 보고하는 방법으로, 정규–역감마 켤레 사전분포를 쓰면 사후분포를 닫힌 식으로 얻는다.

How do I get credible intervals and the probability that a regression coefficient is positive?

회귀계수의 신용구간과 계수가 양수일 확률을 구하려면 어떻게 하나요?

With a normal–inverse-gamma prior, the posterior of each coefficient is a Student t distribution centred near the least-squares estimate when the prior is weak. Credible intervals give the range containing the coefficient with 95% posterior probability, and the posterior directly gives the probability that a coefficient is positive.

When to use Bayesian Linear Regression

• You want probability statements about coefficients
• You want to include prior information or mild shrinkage
• You need credible intervals rather than confidence intervals

When to use something else

• The outcome is binary or a count; use Bayesian logistic or Poisson regression instead
• Predictors greatly outnumber cases; use penalized regression with cross-validation instead

Assumptions and requirements

• Linear relationship between predictors and outcome
• Independent, normally distributed errors with constant variance
• The stated prior is acceptable to readers

What the analysis reports

• Posterior means of coefficients
• 95% credible intervals
• Posterior probability of a positive effect
• Posterior mean of the error variance
• OLS and empirical-Bayes ridge estimates for comparison

How to report it

A Bayesian linear regression with a weakly informative prior estimated the effect of [predictor] as b = [value], 95% CrI [[value], [value]], P(b > 0) = [value].

Related methods

• Multiple Linear Regression
• Bayesian Correlation (Bayes Factor)

Run Bayesian Linear Regression on Quorant