Firth Penalized Logistic Regression

Firth penalized logistic regression adds a Jeffreys-prior penalty to the logistic likelihood, reducing small-sample bias and giving finite estimates under complete or quasi-complete separation.

Firth 벌점 로지스틱 회귀는 로지스틱 우도에 Jeffreys 사전 벌점을 더해 작은 표본의 편향을 줄이고 완전·준완전 분리에서도 유한한 추정값을 내는 방법입니다.

My logistic regression gives huge coefficients and warnings about perfect separation; what should I do?

로지스틱 회귀에서 계수가 엄청 크게 나오고 완전 분리 경고가 떠요. 어떻게 해야 하나요?

Firth's method maximises a penalized likelihood that removes the first-order bias of maximum likelihood, so coefficients stay finite even when a predictor perfectly separates the outcome. Tests use penalized likelihood ratios and confidence intervals use the profile penalized likelihood, which behave better than Wald intervals in small samples.

When to use Firth Penalized Logistic Regression

• Ordinary logistic regression fails because of complete or quasi-complete separation.
• Events are rare or the sample is small relative to the number of predictors.
• Profile-likelihood intervals for odds ratios are wanted.

When to use something else

• The sample is large with many events per predictor; ordinary binary logistic regression gives nearly the same answer.
• Data are matched sets; use conditional logistic regression.

Assumptions and requirements

• Observations are independent.
• The logit of the event probability is linear in the predictors.
• The outcome is binary.

What the analysis reports

• Penalized coefficients and odds ratios
• Profile penalized-likelihood (or Wald) confidence intervals
• Penalized likelihood-ratio test for each coefficient and the whole model
• Separation check against ordinary maximum likelihood

How to report it

Firth's penalized logistic regression gave an odds ratio for [X] of [value], 95% profile CI [[value], [value]], penalized LR chi2(1) = [value], p = [value].

Related methods

• Binary Logistic Regression
• Exact Logistic Regression
• Conditional Logistic Regression

Run Firth Penalized Logistic Regression on Quorant