Do CI’s indicate uncertainty? No. First of all uncertainty has not been defined well enough. Second of all the definition is parametric and model-dependent: all a CI of 95% says is that 95% of our models fell within this range.
I believe there was an article in Nature some time ago showing calculated CI’s for neutron weight or something like that. You can see visually that past bars did not contain the future right answer.
The case is much worse for survey data (eg www.2008andbeyond.com ) where asking 1000 online survey respondents in no way guarantees that you have captured a significant chunk of 320,000,000 opinions. Unreported victimization (www.worldcat.org/title/estimation-of-victimization-prevalence-using-data-from-the-national-crime-survey/oclc/852792291) the same ---- no assumption can prove something about reality, you can just try to assume different things to get a number at all.
In the case of financial markets, where I’m more comfortable, it’s clearer that a “100-sigma move” can and does happen. In an options model a material change (say RTO of a much larger entity) could, in the same way, be called a change in volatility. A less convoluted and stats-centric way of viewing the world would be to say that the stat model captured enough of the moments for a while to successfully make markets, but when the world changes you need a new model. This reinforces the quick-and-dirty view in which stats can be useful fast Excel answers, not philosophically justifiable “uncertainty quantification”.
Do confidence intervals have any utility? Absolutely yes. Every statistical number in my opinion should be reported as a 98% CI range, since the middle gives a false sense of precision. For example in Cathy Cohen / Gen Next’s survey cited above, the public is led to draw speculative conclusions on the mean which they would never draw on the range.
“according to preliminary estimates, young African Americans are pro-socialism somewhere between -8 and +28 percent
more than pro-capitalism ---- although we didn’t ask a direct comparison” ------- would be accurate, and posting (-8,28) versus (-29,+5) would not invite graph readers to compare means which, statistically, we cannot tell whether they subtract to negative or positive.
It would also point out that very few latin@ americans or asian americans were asked, whereas the plot forwarded to the press makes it look as if latin@s, Asians, Black, and WNH can be compared directly.
In the last study cited, a lack of a direct comparison question between socialism and capitalism is more damning than the failure to use intervals, showing that statisticians need to spend more time on experimental design and that “pure stat” debates won’t probably capture the most relevant factors to truth and inference ---- but that’s another topic.