Nice post @Sander . Although it is somewhat tangential to the title of this thread, I want to highlight some recent work at
http://metalogdistributions.com/publications.html
that I think will become important to specifying priors, at least in a language like Stan that is not concerned with conjugacy or anything like that.
The essence of this research is to use “quantile parameterized distributions”, which are essentially distributions whose parameters are quantiles. So, if you can specify or elicit a prior median for a parameter and at least two or three other quantiles, then it is possible to construct a probability distribution that has those quantiles. In the case of an unbounded distribution like that for a regression coefficient, there are a couple of quantile parameterized distributions, namely the simple q-normal and the megalog(istic). In the case of a distribution that is bounded from above and / or below, there are some good choices based on the Johnson system.
Anyway, for a regression coefficient, one would often set the prior median to be zero and then would need to set a few more quantiles based on what you think is the minimum value for a large effect. I’ll be talking about this idea and Stan a bit more Saturday at the R/Medicine conference if anyone is interested.