Hi everyone - I love the new site!
I’ve been experimenting with LASSO lately to develop a model in a relatively small dataset. I came across something in Friedman, Tibshirani, and Hastie’s 2009 Elements of Statistical Learning book that I thought contradicts my interpretation of Frank’s treatment of shrinkage in his RMS book and wondered about your interpretation:
" Section 3.8.5 - Further properties of the Lasso
Regarding the coefficients themselves, the lasso shrinkage causes the estimates of the non-zero coefficients to be biased towards zero, and in general they are not consistent. One approach for reducing this bias is to run the lasso to identify the set of non-zero coefficients, and then fit an un-restricted linear model to the selected set of features. "
I recognize there is evidence of over-shrinkage with lasso, but I worry about losing the benefits of coefficient shrinkage with the above suggestion. The benefits seem to outweigh the risk of overshrinkage in most simulations I’ve seen.
Thanks for considering,