# RMS Describing, Resampling, Validating, and Simplifying the Model

**URL:** <https://discourse.datamethods.org/t/rms-describing-resampling-validating-and-simplifying-the-model/4790>\
**Category:** modeling strategy\
**Tags:** graphics, bootstrap, validation\
**Created:** [September 14, 2021, 7:24pm UTC](https://discourse.datamethods.org/t/rms-describing-resampling-validating-and-simplifying-the-model/4790 "2021-09-14T19:24:11Z")\
**Posts on this page:** 1\
**Showing post:** 110

<div class="post-metadata">

**Author:** ![brawthy](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/brawthy/32/4027_2.png) [@brawthy](https://discourse.datamethods.org/u/brawthy)\
**Post date:** [July 11, 2023, 6:52pm UTC](https://discourse.datamethods.org/t/rms-describing-resampling-validating-and-simplifying-the-model/4790/110 "2023-07-11T18:52:59Z")

</div>

Good Afternoon Professor Harrell and everyone! It’s awesome to be here. I 've currently making my way through RMS and Flexible Imputation to really round out some material I wasn’t exposed to in undergrad and graduate statistics.

Currently, I am trying to build a diagnostic model on case controlled data. Specifically-I am having issues with some missingness structure, as well as some data that I believe is redundant and needlessly increases the dimensionality of my sample space. I’d like to build a few competing models after imputing and performing redundancy analysis. Our primary goal is a predictive model, so I have been following RMS Chapters 4 and 5 in this regard.

My thoughts on how to proceed without falling into some of the pitfalls that produce over-optimistic models is the two step process below. I have been wrestling with the though that including all steps within the same ‘double bootstrap’ loop is better-but I fell like I need to account for uncertainty in MI + RDA step before any modeling validation and selection is done.

1. Redundancy Analysis + MI

2. Model Validation and Selection:

Am I correct in my approach here? Please let me know how I can elaborate more if needed, and please forgive the markdown-I’d be happy to fix if necessary.

For Reference: I have used this post here to guide my thinking.

> [@Using the double bootstrap for validating prediction models](https://discourse.datamethods.org/t/using-the-double-bootstrap-for-validating-prediction-models/5090):
>
> I have a few questions about using the double bootstrap for the purposes of validating prediction model performance. When is the double bootstrap more preferable than the basic bootstrap? What is a useful definition of the double bootstrap algorithm? The double bootstrap seems suitable for two cases: 1) when a bias adjusted estimate of optimism is desired or 2) to both select from competing models and summarize the performance of the selected model (in the manner of nested k-fold cross valida…

Thank you so much!

---

_[View the full topic](https://discourse.datamethods.org/t/rms-describing-resampling-validating-and-simplifying-the-model/4790)._