Comparison of Poisson regression models with same counting outcomes but 2 different scores as predictor

data analysis modeling strategy
1

Dear all,
I’m looking for the best way to compare the performance of two scores for a counting outcome.
So, I would like to compare two Poisson regression models, and I’ve found different approaches: one commonly used is to check for AIC in both models but I’ve found some controversial opinions about. Is there a better way to perform that ? In case I would be grateful if you could point me out some literature.

Thanks for your time

2

Since your model is a standard Poisson model that is based on full likelihood approach and the two models belong to the same family of likelihood functions, I would use Likelihood Ratio Test (LRT)

3

I thought LRT could only be used for nested models (Why can't likelihood ratio tests be used for non-nested models? - Cross Validated), in my case we have non-nested models with potentially a different number of predictors too.

2 Likes
4

You may consider Vuong test, which can be used for non-nested model comparisons. A useful R package for non-nested models: https://cran.r-project.org/web/packages/nonnest2/nonnest2.pdf

SAS implementation: 42514 - Tests for comparing nested and nonnested models

Some references:
Vuong, Q. (1989), “Likelihood ratio tests for model selection and non-nested hypotheses,” Econometrica, 57: 307-334.
Clarke, K.A. (2001), “Testing Nonnested Models of International Relations: Reevaluating Realism,” American Journal of Political Science , 45:3, 724-744.

A good read before applying the Vuong test:
Wilson, P. (2015), “The misuse of the Vuong test for non-nested models to test for zero-inflation,” Economics Letters, 127: 51-53.