A new paper in the Lancet and related twitter thread
https://www.thelancet.com/journals/landia/article/PIIS2213-8587(20)30159-5/fulltext
Using individual patient data for each trial we calculated mean 24-month BMD percent change together with fracture reductions and did a meta-regression of the association between treatment-related differences in BMD changes (percentage difference, active minus placebo) and fracture risk reduction
Many threads on the forum have covered the problems with precent change from baseline in general. Yet here is another influential result where investigators have chosen to go with percent change nonetheless.
Unlike psychiatric clinical outcomes, though BMD, which is a normed/studentized score, is perhaps a little less problematic in terms of the criteria that @f2harrell has outlined before.
- the post value must be linearly related to the pre value
- the variable must be perfectly transformed so that subtraction “works” and the result is not baseline-dependent
- the variable must not have floor and ceiling effects
- the variable must have a smooth distribution
- the slope of the pre value vs. the follow-up measurement must be close to 1.0 when both variables are properly transformed (using the same transformation on both)
See http://hbiostat.org/bbr/md/change.html#sec:changegen
Anyone care to comment whether BMD is a positive example of when percent change is not egregious?
4 Likes
It would be nice to have access to a dataset to demonstrate this, but I think that it is extremely unlikely that percent change of BMD is a valid quantity. Assumptions 2 and 5 are the most likely assumptions to be violated. One has to ask why researchers are so afraid of analyzing raw data when such analysis is more powerful and interpretable? Flexibly covariate adjust for baseline BMD and respect the parallel group design which exists to compare parallel groups, not to examine change from baseline.
3 Likes
Dear Professor @f2harrell, I have a question on this particular topic (surrogate endpoints):
Assuming we agree on the following definition of a surrogate:
“the change/variation in one variable (x) predicts the change/variation in another variable (y)”.
In order to assess/ test whether x can be a surrogate for y, how one could specify the regression model?
m1 -> change_Y ~ change_X
m2 -> post_Y ~ pre_Y + post_X + pre_X
I suppose m1 is a bad choice, but I’m not sure whether m2 makes sense.
That is not an accepted definition of surrogacy. The defiinitions I’ve seen are based on treatment effects. It is a lot easier to say Y1 predicts Y2 than it is to say that Y2 is a surrogate for Y1 with regard to studying a treatment effect.
Forget the surrogacy part. If I want to test whether change in one variable predicts change in another variable, how can I specify the model?
In that case, Y1 is the primary outcome and Y2 is a secondary outcome in a 2-group RCT. The idea is that patients who improve on Y2 (pre-post change) will also improve on Y1 (pre-post change).
Does this model make sense?
Y1_post ~ Y1_pre + Group + Y2_pre + Y2_post
Here is one of the better article outlining the problems with surrogate outcomes: Surrogate end points in clinical trials: are we being misled? by Fleming and DeMets. Beyond the baseline issue that originated this thread is that many surrogates (including BMD) do not necessarily correlate with the outcome of interest. Surrgoates can be uncorrelated and misleading. Best to not use surrogate outcomes at all.
I want to start pushing the idea of replacing surrogate outcomes with intermediate ordinal outcomes, with up-front specification of the amount of trust you have for how the treatment effect on the intermediate outcome bleeds over to the effect open the high-level outcome. This is related to this.
2 Likes