# Who should define reasonable priors and how?

**URL:** <https://discourse.datamethods.org/t/who-should-define-reasonable-priors-and-how/2935>\
**Category:** bayes\
**Tags:** reanalysis, prior, interpretation\
**Created:** [January 30, 2020, 9:40am UTC](https://discourse.datamethods.org/t/who-should-define-reasonable-priors-and-how/2935 "2020-01-30T09:40:55Z")\
**Posts on this page:** 1\
**Showing post:** 13

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [May 26, 2020, 2:57pm UTC](https://discourse.datamethods.org/t/who-should-define-reasonable-priors-and-how/2935/13 "2020-05-26T14:57:40Z")

</div>

I thought I’d bump this thread after finding a valuable discussion between Andrew Gelman and Sander Greenland on this precise issue of setting priors.

**Specifying a prior distribution for a clinical trial: What would Sander Greenland do?**  
[https://statmodeling.stat.columbia.edu/2008/02/05/specifying\_a\_pr\_1/](https://statmodeling.stat.columbia.edu/2008/02/05/specifying_a_pr_1/)

**Specifying a prior distribution for a clinical trial**  
[https://statmodeling.stat.columbia.edu/2008/01/24/specifying\_a\_pr/](https://statmodeling.stat.columbia.edu/2008/01/24/specifying_a_pr/)

My posts in this thread were some speculations I had about how 2 _honest_ Bayesian scientists might negotiate in order to derive an experiment that would settle a question of fact, that is in dispute.

In the real world information is not free. So it would be important for parties of the dispute to know how much it would cost to run this (hopefully) definitive experiment (ie expected number of observations), and if that new information is worth paying for. In the decision theory and economic perspectives, this is known as Value of Information.

> **[Value of information](https://en.wikipedia.org/wiki/Value_of_information)**
>
> Value of information (VOI or VoI) is the amount a decision maker would be willing to pay for information prior to making a decision.
> VoI is sometimes distinguished into value of perfect information, also called value of clairvoyance (VoC), and value of imperfect information. They are closely related to the widely known expected value of perfect information (EVPI) and expected value of sample information (EVSI). Note that VoI is not necessarily equal to "value of decision situation with perfect...

This is the role I see for retrospective meta-analyses – constraining the family of possible informative priors in order to decide on what future experiments (broadly defined, also including observational studies) to perform, to settle disagreements on fact and to aid in selecting a policy.

In Greenland’s terminology, he would call them “covering priors”, a concept I think has substantial merit.

Relevant Discussions

> [@Is it sensible to consider meta-analysis the strongest level of evidence for or against an intervention?](https://discourse.datamethods.org/t/is-it-sensible-to-consider-meta-analysis-the-strongest-level-of-evidence-for-or-against-an-intervention/248/17):
>
> i was reading a bayes re-analysis yesterday that did this, it might not be the best example tho: [bayes re-analysis, ECMO RCT](https://jamanetwork.com/journals/jama/fullarticle/2709620). They say: “data-derived prior distributions were developed based on relevant studies18-20 from a meta-analysis of ECMO for ARDS.21 The treatment effects in these previous studies were combined with the observed data from this trial in a Bayesian hierarchical random-effects model (that itself used minimally informative priors)” they make too much of the convention of …

> [@2019 Nobel Prize in Economic Sciences is about limitations of RCT](https://discourse.datamethods.org/t/2019-nobel-prize-in-economic-sciences-is-about-limitations-of-rct/2634):
>
> This article summarizes the 2019 Nobel Price in Economics" [What randomisation can and cannot do: The 2019 Nobel Prize](https://voxeu.org/article/what-randomisation-can-and-cannot-do-2019-nobel-prize) Their critique of randomized controlled trials (RCT) is three-fold: They make no specific claim about external validity. In economics it requires a pre-existing system the study is embedded within. In a plumbing metaphor, the studies focus on problems in existing plumbing–not upon how to engineer and install a system. Optimal experimental design is generally not an RCT. From …

A Ph.D dissertation by a recent Vanderbilt graduate who was supervised by both Dr. Harrell and Dr. Blume. It is likely that rational scientists, in fields with small samples (especially my field of rehabilitation and physical disabilities) would implement some sort of adaptive randomization to balance the need for efficiency with the need for robustness (ie credibility).

I don’t remember the dissertation going into actual details, but AFAIK, it shouldn’t be hard to prove this type of design is admissible for both parties.

**Chipman, Jonathan Joseph (2019)** Sequential Rematched Randomization and Adaptive Monitoring with the Second-Generation p-Value to Increase the Efficiency and Efficacy of Randomized Clinical Trials. Vanderbilt University, ProQuest Dissertations Publishing, 2019. 13900836.

[http://etd.library.vanderbilt.edu/available/etd-06142019-180645/unrestricted/\_main.pdf](http://etd.library.vanderbilt.edu/available/etd-06142019-180645/unrestricted/_main.pdf)

---

_[View the full topic](https://discourse.datamethods.org/t/who-should-define-reasonable-priors-and-how/2935)._