I’m looking for the best way to compare the performance of two scores for a counting outcome.
So, I would like to compare two Poisson regression models, and I’ve found different approaches: one commonly used is to check for AIC in both models but I’ve found some controversial opinions about. Is there a better way to perform that ? In case I would be grateful if you could point me out some literature.
Thanks for your time
Since your model is a standard Poisson model that is based on full likelihood approach and the two models belong to the same family of likelihood functions, I would use Likelihood Ratio Test (LRT)
I thought LRT could only be used for nested models (Why can't likelihood ratio tests be used for non-nested models? - Cross Validated), in my case we have non-nested models with potentially a different number of predictors too.
You may consider Vuong test, which can be used for non-nested model comparisons. A useful R package for non-nested models: https://cran.r-project.org/web/packages/nonnest2/nonnest2.pdf
SAS implementation: 42514 - Tests for comparing nested and nonnested models
Vuong, Q. (1989), “Likelihood ratio tests for model selection and non-nested hypotheses,” Econometrica, 57: 307-334.
Clarke, K.A. (2001), “Testing Nonnested Models of International Relations: Reevaluating Realism,” American Journal of Political Science , 45:3, 724-744.
A good read before applying the Vuong test:
Wilson, P. (2015), “The misuse of the Vuong test for non-nested models to test for zero-inflation,” Economics Letters, 127: 51-53.