Responder analysis is a most problematic statistical analysis. General problems with responder analysis are described here. Analysis of acute kidney injury (AKI) as a binary outcome is one example of responder analysis. Any time that a binary response is created from an ordinal or continuous response, information, power, and precision are lost, sample size is needed to be greatly increased, and the analysis is afflicted by arbitrariness. It should be noted that if clinicians are interested in the probability that an underlying continuous response will fall below or exceed a certain cutoff, such probabilities can readily be computed from the continuous or ordinal response model, and these probabilities are more precise than those estimated from a binary logistic model on the forced binary response.
A standard definition of AKI is the presence of any of three conditions:
- increase in serum creatinine (SCr) by \geq 0.3 mg/dl within 48h
- increase in SCr to \geq 1.5 \times baseline
- urine volume < 0.5 ml/kg/h for 6 hours
For this discussion let’s consider just the SCr components. First let’s better understand what the (unnecessarily binary) AKI designation means. The following plot shows the region in the baseline and follow-up SCr space that qualifies a patient to be labeled as AKI, i.e., it shows the lower limit of the second SCr measurement that qualifies a patient as AKI, as a function of baseline SCr. You can see that condition 2. is virtually ignored, as the increase of SCr by 0.3 is a much more relaxed condition than requiring an elevation to 1.5 \times the baseline SCr. The gray scale line is the line of identity.
Besides losing all information about the amount of kidney injury, conditions 1. and 2. above suffer from three serious problems:
- SCr does not change in either an incremental or ratio fashion
- The relationship between SCr and mortality is not monotonic; very small SCr is detrimental to health
- Once SCr is updated with a later measurement, the baseline SCr becomes almost irrelevant. The patient’s ultimate prognosis is determined primarily by the most recently measured SCr (plus urine output and other physiologic measures and age)
A prime requirement for a proper response variable is that it mean the same thing to different patients. AKI fails this test, because to undo the undue influence of baseline SCr, one must interact AKI with baseline SCr in order for it to correctly relate to prognosis.
To illustrate point 2. above, consider the following figure from BBR Section 14.4.2.
To illustrate point 3., consider the same section in BBR where risk of death in hospital is estimated for critically ill hospitalized adults on the basis of a smooth nonlinear function of day one SCr and a smooth nonlinear function of day 3 SCr for patients surviving at least to the start of day 3. Here are the adjusted Wald \chi^2 statistics assessing the prognostic importance of the baseline and updated SCr:
One can see that the prognostic influence of baseline (day 1) SCr is almost negligible. Critically ill patients’ prognoses depend almost solely on current levels of organ damage, not on how the patient got there.
The central focus on therapeutic comparisons should be to answer the following question: To what extent do two patients who are alike at baseline save the treatment assignment end up with different outcomes? In the setting of AKI we should ask what is the ultimate renal function given the baseline level of renal function. So envision a general regression model that predicts day 3 SCr from a restricted cubic spline in day 1 SCr with 5 knots (requiring one linear and 3 nonlinear terms).
When renal function is the outcome variable, a semiparametric ordinal model such as the proportional odds model has special benefits. It would allow a “rapid deterioration requiring dialysis” override as the highest ordinal level, as in BBR Section 4.1.2:
Note that if change or log ratio of SCr were used as the response variable instead of using the final SCr, one would have more difficulty knowing where to place dialysis on the outcome scale. A large change in SCr may still result in a final SCr that is not as bad as needing dialysis.
By predicting follow-up SCr from a flexible function of baseline SCr, one can estimate the probability of exceeding any desired change or ratio, and make it completely transparent that this probability must depend on baseline SCr.