Regression Modeling Strategies: Overview of Maximum Likelihood Estimation

data analysis modeling strategy
1

This topic is for questions and discussions about the maximum likelihood chapter in Regression Modeling Strategies and its course notes. Some resources for maximum likelihood estimation are also included here as well as discussions about additions to current capabilities of the R rms package related to maximum

General Resources

  • Computational aspects of MLE including efficient calculations when there is an unlimited number of intercepts in an ordinal semiparametric model

Resources for Likelihood Ratio Tests and Profile Likelihood Confidence Intervals

1 Like
2
  1. A very brief overview of Wald and likelihood based CIs and the procedure for obtaining profile likelihood-based CI for a single parameter of interest while regarding the other parameters as nuisance, https://journals.sagepub.com/doi/pdf/10.1177/1536867X0700700305 (refer to the introduction)
  2. The original paper for a method for computing profile likelihood-based CI via more than definition, https://www.jstor.org/stable/2347496
1 Like
3

I also found both articles that Charlie mentioned to be very helpful.I also found an article about “A robust and efficient algorithm to find profile likelihood confidence intervals” A robust and efficient algorithm to find profile likelihood confidence intervals | Statistics and Computing That went over existing approaches and suggested a new approach. Which I found interesting.

3 Likes
4

From the stack overflow post the commenter gives us an approach for the estimating a nonlinear CI for the parameter theta = p(0.75)-p(0.25) where p is the probability that Y=1 given X=x. Since theta is a function of our regression coeffs, the commenter suggests first defining a grid of the parameter values over the range suggested by the fit, then constructing the profile likelihood by evaluating the negative LLH at each fixed theta in the grid of possible coeffs pairs. It then finds the min neg LLH at each fixed theta, which generates our curve that approximates the profile likelihood. This curve is then used to construct our CIs by identifying the range of theta values where the likelihood stays within a certain distance from the min, based on the Chi-squared dist. Important to note that this method will not be suitable for many coeffs.

5

Obtain profile likelihood-based CIs via optimization https://arxiv.org/abs/2404.02774

6

A robust and efficient algorithm to find profile likelihood confidence intervals A robust and efficient algorithm to find profile likelihood confidence intervals | Statistics and Computing
PK software implementation: Likelihood interval for nonlinear regression - PMC

1 Like
7

A follow-up: constrained optimization method for obtaining PL-based CIs for single parameter and available package, doi 10.1371/journal.pcbi.1008495

2 Likes
8

Using Bayesian tools to be a better frequentist : “In such settings, even a staunch frequentist may be better served by Bayesian computation”, where “such settings” refers to small sample size and frequentist methods, like Wald-based or even profile likelihood-based CI, rely on large samples for approximation.