There’s some really exciting research being done on predictive multiplicity–the observation that many models perform just as well as the “optimal” fit, yet disagree on feature importance and predictions. This research asks interesting and useful questions like:
-
Which models are functionally as performant as the loss-minimizing model?
-
How do I derive and understand that shape and character of models in that set?
-
How can I query for models in that set that meet domain-specific needs?
Yet many of these results haven’t made it out of the papers and into the open-source ecosystem.
Glassy fills this tooling gap and fits right onto standard scikit-learn flows. With it, you can do things like:
- Find simpler models that perform as well as the best loss model
- Check which options agree with your domain expert’s judgment
- Make segment tradeoffs intentionally
For v0.0.1, the scope is capped at OLS linear regression, but hoping to generalize to GLMs in the future. Here’s a five-minute demo notebook.
Curious to see if this is useful to you all, or any requests for future features. Happy modeling!