# \#validation

**URL:** https://discourse.datamethods.org/tag/validation/10.md

[Latest](https://discourse.datamethods.org/latest.md) · [Categories](https://discourse.datamethods.org/categories.md) · [Tags](https://discourse.datamethods.org/tags.md)

---

## [Synic Evidence - How should an AI tool prove a cited passage actually supports the claim? Feedback wanted on one I built](https://discourse.datamethods.org/t/synic-evidence-how-should-an-ai-tool-prove-a-cited-passage-actually-supports-the-claim-feedback-wanted-on-one-i-built/28903)

<div class="topic-metadata">

**Author:** [@mosisasaba](https://discourse.datamethods.org/u/mosisasaba)\
**Replies:** 7\
**Last updated:** [October 29, 2026, 2:29pm UTC](https://discourse.datamethods.org/t/synic-evidence-how-should-an-ai-tool-prove-a-cited-passage-actually-supports-the-claim-feedback-wanted-on-one-i-built/28903 "2026-10-29T14:29:10Z")

</div>

Hi all, I’m Mosisa, a senior at USC studying Health and Human Sciences and Narrative Medicine. I’m posting here because this is the group that would tell me where my thinking is wrong. The problem I kept hitting: AI too…

---

## [RMS Describing, Resampling, Validating, and Simplifying the Model](https://discourse.datamethods.org/t/rms-describing-resampling-validating-and-simplifying-the-model/4790)

<div class="topic-metadata">

**Author:** [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Replies:** 181\
**Last updated:** [September 13, 2025, 12:01pm UTC](https://discourse.datamethods.org/t/rms-describing-resampling-validating-and-simplifying-the-model/4790 "2025-09-13T12:01:55Z")

</div>

Regression Modeling Strategies: Describing, Resampling, Validating, and Simplifying the Model This is the fifth of several connected topics organized around chapters in Regression Modeling Strategies. The purposes of th…

---

## [BBR: Challenges of Analyzing High Dimensional Data](https://discourse.datamethods.org/t/bbr-challenges-of-analyzing-high-dimensional-data/7231)

<div class="topic-metadata">

**Author:** [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Replies:** 12\
**Last updated:** [November 28, 2023, 1:06pm UTC](https://discourse.datamethods.org/t/bbr-challenges-of-analyzing-high-dimensional-data/7231 "2023-11-28T13:06:25Z")

</div>

This topic is for Q&A and discussions around the BBR chapter on high dimensional data analysis.

---

## [Validation/calibration of clinical prediction model using km estimated survival](https://discourse.datamethods.org/t/validation-calibration-of-clinical-prediction-model-using-km-estimated-survival/6356)

<div class="topic-metadata">

**Author:** [@Robert\_Strother](https://discourse.datamethods.org/u/Robert_Strother)\
**Replies:** 9\
**Last updated:** [May 12, 2023, 6:33am UTC](https://discourse.datamethods.org/t/validation-calibration-of-clinical-prediction-model-using-km-estimated-survival/6356 "2023-05-12T06:33:26Z")

</div>

first time poster - primarily a clinician (medical oncologist), but pursuing a part-time PhD to develop teaching materials for clinical prediction models (CPM) for clinicians. I am looking to review a commonly used onco…

---

## [Internal validation: Imputation nested within bootstrap](https://discourse.datamethods.org/t/internal-validation-imputation-nested-within-bootstrap/5608)

<div class="topic-metadata">

**Author:** [@pravleen](https://discourse.datamethods.org/u/pravleen)\
**Replies:** 2\
**Last updated:** [May 9, 2022, 7:19pm UTC](https://discourse.datamethods.org/t/internal-validation-imputation-nested-within-bootstrap/5608 "2022-05-09T19:19:21Z")

</div>

Background: I would like to internally validate my prediction model using bootstrapping. I would also like to use multiple imputation to deal with missing data and will be nesting the MI within each bootstrap sample. The…

---

## [Bootstrap vs. cross-validation for model performance](https://discourse.datamethods.org/t/bootstrap-vs-cross-validation-for-model-performance/2779)

<div class="topic-metadata">

**Author:** [@Lynild](https://discourse.datamethods.org/u/Lynild)\
**Replies:** 10\
**Last updated:** [April 4, 2022, 2:30pm UTC](https://discourse.datamethods.org/t/bootstrap-vs-cross-validation-for-model-performance/2779 "2022-04-04T14:30:56Z")

</div>

So I have developed some logistic models, and obviously the next step is to validate them. I have no option for external validation, so internal is the way to go for me. On that note, I have been looking into this articl…

---

## [Some help needed regarding how to conduct an external validation study](https://discourse.datamethods.org/t/some-help-needed-regarding-how-to-conduct-an-external-validation-study/5213)

<div class="topic-metadata">

**Author:** [@caffeinatorthesecond](https://discourse.datamethods.org/u/caffeinatorthesecond)\
**Replies:** 0\
**Last updated:** [December 23, 2021, 3:23pm UTC](https://discourse.datamethods.org/t/some-help-needed-regarding-how-to-conduct-an-external-validation-study/5213 "2021-12-23T15:23:05Z")

</div>

Hello everyone. Hope you’re well. I’m a second year surgical resident from Pakistan (with an extremely poor background in medical statistics and a below average background in medical research), and I’m hoping to conduct…

---

## [Internal model validation (Bootstrapping) and Multiple Imputation](https://discourse.datamethods.org/t/internal-model-validation-bootstrapping-and-multiple-imputation/4255)

<div class="topic-metadata">

**Author:** [@TJ61](https://discourse.datamethods.org/u/TJ61)\
**Replies:** 3\
**Last updated:** [August 3, 2021, 3:26am UTC](https://discourse.datamethods.org/t/internal-model-validation-bootstrapping-and-multiple-imputation/4255 "2021-08-03T03:26:16Z")

</div>

What is recommended way to incorporate both bootstrapping (i.e. to compute optimism in the R-squared value) and multiple imputation? For example, is it acceptable to: (1) Perform multiple imputation to generate x numbe…

---

## [Sample size and power determination in high dimensional observational studies](https://discourse.datamethods.org/t/sample-size-and-power-determination-in-high-dimensional-observational-studies/4159)

<div class="topic-metadata">

**Author:** [@Elisabetta](https://discourse.datamethods.org/u/Elisabetta)\
**Replies:** 2\
**Last updated:** [February 26, 2021, 1:37pm UTC](https://discourse.datamethods.org/t/sample-size-and-power-determination-in-high-dimensional-observational-studies/4159 "2021-02-26T13:37:59Z")

</div>

Hi all, I am collaborationg (with some biologists and clinicians) to the writing of a project. Last year, the proposal was not funded and one of the reviewers made a comment on the absence of any consideration on “power…

---

## [Crowdsourcing best-practices alternative methods for CARG-BC clinical prediction model](https://discourse.datamethods.org/t/crowdsourcing-best-practices-alternative-methods-for-carg-bc-clinical-prediction-model/4112)

<div class="topic-metadata">

**Author:** [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Replies:** 6\
**Last updated:** [February 20, 2021, 12:59pm UTC](https://discourse.datamethods.org/t/crowdsourcing-best-practices-alternative-methods-for-carg-bc-clinical-prediction-model/4112 "2021-02-20T12:59:21Z")

</div>

The CARG-BC clinical prediction model published recently in JCO \[1\] strikes me as methodologically flawed on multiple levels, even decision-theoretically and ethically. In this post, I would hope to explore specifically …

---

## [Model Performance Following Best Subset Selection](https://discourse.datamethods.org/t/model-performance-following-best-subset-selection/2921)

<div class="topic-metadata">

**Author:** [@Lynild](https://discourse.datamethods.org/u/Lynild)\
**Replies:** 4\
**Last updated:** [January 26, 2020, 4:38pm UTC](https://discourse.datamethods.org/t/model-performance-following-best-subset-selection/2921 "2020-01-26T16:38:50Z")

</div>

I was just curious to whether performance measures like R^2, Brier score, calibration slope/intercept etc. are as applicable on Best Subset Selection (BSS)/LASSO-based models compared to a regular GLM based model ? For …

---

## [Best smoothing methods for probability prediction calibration curves](https://discourse.datamethods.org/t/best-smoothing-methods-for-probability-prediction-calibration-curves/2068)

<div class="topic-metadata">

**Author:** [@hanowell](https://discourse.datamethods.org/u/hanowell)\
**Replies:** 1\
**Last updated:** [August 31, 2019, 11:44am UTC](https://discourse.datamethods.org/t/best-smoothing-methods-for-probability-prediction-calibration-curves/2068 "2019-08-31T11:44:14Z")

</div>

When drawing the calibration curve of a probability prediction model, I often observe wildly different results when I use LOWESS vs. a Gaussian additive model with smoothing splines vs. a binomial generalized additive mo…

---

## [Analysis QA - Best practices?](https://discourse.datamethods.org/t/analysis-qa-best-practices/709)

<div class="topic-metadata">

**Author:** [@timdisher](https://discourse.datamethods.org/u/timdisher)\
**Replies:** 22\
**Last updated:** [May 15, 2019, 12:41pm UTC](https://discourse.datamethods.org/t/analysis-qa-best-practices/709 "2019-05-15T12:41:53Z")

</div>

I am in a nursing PhD and am pretty much self-taught on the analysis front outside of our (very general) graduate level stats course. I am very interested in the “secret curriculum” that comes along with formal training …

---

## [Calibration Question](https://discourse.datamethods.org/t/calibration-question/1286)

<div class="topic-metadata">

**Author:** [@chrisarg](https://discourse.datamethods.org/u/chrisarg)\
**Replies:** 6\
**Last updated:** [January 28, 2019, 9:01pm UTC](https://discourse.datamethods.org/t/calibration-question/1286 "2019-01-28T21:01:40Z")

</div>

Good evening, I was wondering whether someone more experienced in this , could shed some light into a question about model calibration. I have two models (let’s call them model1 and model2) which were developed in two …

---

## [About the model validation category](https://discourse.datamethods.org/t/about-the-model-validation-category/45)

<div class="topic-metadata">

**Author:** [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Replies:** 0

</div>

model validation and interpretation