Why so big difference in the p values obtained from logistic regression and Fisher's exact test?

data analysis formal
,
1

Here is a data frame.

> tmp_df
   BRCA1_MYC2 Best_response3
1       BRCA1              0
2       BRCA1              0
3       BRCA1              0
4       BRCA1              0
5         MYC              1
6       BRCA1              1
8         MYC              1
9       BRCA1              0
10        MYC              1

I have performed fisher exact test and it returned a p value of 0.048.

> table(tmp_df$Best_response3, tmp_df$BRCA1_MYC2) %>% fisher.test()

    Fisher's Exact Test for Count Data

data:  .
p-value = 0.04762
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 0.6944485       Inf
sample estimates:
odds ratio 
       Inf

But when I performed logistics regression ananlysis, it returned a p value of 0.997.

> tmp_fit <- glm(Best_response3 ~ BRCA1_MYC2, data = tmp_df, family = 'binomial')
> summary(tmp_fit)

Call:
glm(formula = Best_response3 ~ BRCA1_MYC2, family = "binomial", 
    data = tmp_df)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-0.60386  -0.60386  -0.60386   0.00008   1.89302  

Coefficients:
              Estimate Std. Error z value Pr(>|z|)
(Intercept)     -1.609      1.095  -1.469    0.142
BRCA1_MYC2MYC   21.176   6208.832   0.003    0.997

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 12.3653  on 8  degrees of freedom
Residual deviance:  5.4067  on 7  degrees of freedom
AIC: 9.4067

Number of Fisher Scoring iterations: 18

I’m wondering why so big difference exists and which p value should be considered. Thanks.

2

This paper by Mansournia et al in the American J of Epidemiology deals with this problem in detail and should give you the answers you need.

3 Likes
3

Be sure to use the lkelihood ratio \chi^2 test. The P-value for that is

1 - pchisq(12.3653 - 5.4067, 2)

which is 0.008. The rms package in R gives you this directly when you use lrm(y ~ x, ...) and there is only one predictor to be tested (like your case).

As @lachlan pointed out, you have the Hauck-Donner effect where the regression coefficient is estimating \infty and the standard error is too large. This ruins the Wald test you are looking at but does not hurt the LR test.

Fisher’s so-called “exact” test is not very accurate. For details see this.

3 Likes