Some of my colleagues are analyzing data from an RCT that included patients with a terminal illness. The intervention arm was given an app designed to guide self-management of pain and request clinical opinion when necessary. The control arm received care as usual. The primary outcome was a reduction in pain after 6 months. There was a 1-point average drop in pain in the intervention arm. To translate this into a $ value, the investigators plan to look at hospital visits (admission, outpatient and emergency. no medication costs) for both arms during the study period and infer that the difference in cost is due to the intervention as the study was randomized. This approach intuitively seems fallacious to me but I am not sure. The argument from the analyst that ârandomization should take care of any confoundersâ is confusing me even more. I would be grateful if someone can help me understand this better and suggest a sound approach of analyzing cost data in such situations. I work in the digital health space and everyone is keen on knowing the cost impact above everything else.
Hi,
Theoretically, randomization should enable inferences to be made relative to inter-arm differences. That is the primary point of randomization, where any inter-arm differences at baseline would be do to chance, and not patient selection bias, etc.
There was a great discussion here last year that covered issues such as baseline covariate imbalance and related topics:
That being said, there are reasons that we have more sophisticated randomization methods, beyond simple âcoin-tossâ approaches. These include stratified randomization, minimization, and others, to recognize that there may be a need to control the randomization process proactively, to reduce the possibility of an imbalance in one or more critical baseline factors, that may affect the analysis of primary endpoints.
in the case of your study, it seems to me that there could be potential bias resulting from differences in cancer types, locations of primary tumors and metastases, mental status (cognition, depression, etc.), and related factors, that could easily influence patient perceived pain levels over time, unless perhaps this study is for a very particular type of terminal cancer and related narrow inclusion/exclusion criteria.
That all of your patients survived at least six months, probably infers something about their characteristics when diagnosed as being terminal.
That six month time period might also infer something about differences in treatments that they may have received during that time window, each of which in turn, may have their own side effects that could impact pain levels. Those may even be post-randomization treatment decisions that could affect inter-arm differences.
In reviewing your brief description of the study, I also see a few other factors that can impact your results.
First, what kind of pain scale was used? Is this a typical 0 - 10 pain scale? If so, is a 1 point average difference after six months a clinically relevant difference, or a relevant difference in quality of life from the patientâs perspective?
Second, if you do not include the costs of medication in your assessment of the difference in cost, how can you reasonably assess that difference?
You have two groups of terminal cancer patients, who will at some point during their course, require substantial pain management, likely eventually ending up on IV narcotics, as they approach end of life. I am not sure how you can infer differences in cost, without taking into account costs associated with the utilization of pain medications, either in terms of types, frequency of use or dosing?
I am presuming that these patients are in various care environments, which have their own respective cost differentials that may bias your comparisons. Some will be inpatients in a hospital, some will be at home on an outpatient basis, while others will be in or end up in hospice care either at home or in a dedicated facility. There may be transitions from one location/state to another, as they approach end of life.
For the admissions, outpatient and emergency interactions, are you tracking the reasons for these? Are they specifically relevant to pain management interactions, or other reasons, that may bias your assessment of cost differences?
In essence, it seems to me that you have various issues to consider, relative to the identification of any differences in cost, or lack thereof, between the two arms, and what any differences identified may actually infer.
There are experts out there in pharmacoeconomics, and they can be helpful in specifically assessing costs of care in various settings.
Sounds like bullshit to me, do you have access to the analysis they did? Code data model, etc? (Although everything is usually in the code) I listen to code before I listen to verbal arguments
Hi
Iâll put in my two pennyworth from my perspective as a health economist.
My understanding is that your collaborators wish to conduct a within-trial cost-effectiveness analysis. In this context, the outcome would be difference in hospital cost per point drop in pain over the follow-up of the trial. Iâll leave aside whether a within-trial CEA is appropriate. I suggest this paper (https://www.ncbi.nlm.nih.gov/pubmed/16491461) for a discussion of the reasons why it is usually not appropriate.
Would randomisation take care of any confounders? MSchwartzâs post was great in outlining all the reasons for why the randomisation might not take care of confounders for the effectiveness outcome. These reasons are also relevant for the costs. However, if you have confidence in the result for the effectiveness outcome (the change in pain), the result for the difference in the use of hospital resources should also be unbiased.
There are additional challenges for costs though: the distribution of costs tends to be very skewed with a few patients with very high costs and a lot of patients with lower costs, and there may be a large variability in the use of hospital resources hence costs. There are many techniques to obtain a âtreatment effectâ on costs from the straight forward bootstrapping to generalised linear models etc. Usually in cost-effectiveness, weâre not looking for statistically significant differences but instead which option is expected to be cost-effective, the magnitude of the uncertainty and the consequences of the uncertainty in health outcomes and costs.
It was unclear the rationale for excluding medication costs. For example, the reduction in pain may be via an increase in medication. If the medication is very cheap, hence the impact on costs is tiny, you could argue that you can exclude it; if the medication is expensive, then you should include it. I would err on the side of caution and include it. From the costs youâve mentioned, I imagine that the perspective is that of the hospital. If it is the perspective of the health care system, youâd want to include costs related to primary care. For more information about costing, I recommend this paper: https://link.springer.com/article/10.1007/s40273-019-00771-y
If youâre looking for guidance about the analysis for a within-trial cost-effectiveness analysis, I can recommend this book https://global.oup.com/academic/product/economic-evaluation-in-clinical-trials-9780199685028?cc=gb&lang=en&
Good luck!