Written by Clay Smith
This article helps us be clearer with our language as we write about and explain statistics.
Why does this matter?
This article is meant for potential authors who want to publish in Chest, which is not something we would usually cover. But it was so helpful, I just couldn’t help myself.
Statistical pearls for mere mortals
Here are the stats pearls I take away.
There is a right and wrong way to do a study. A RCT should follow CONSORT guidelines. Observational studies should follow STROBE guidelines. See the Equator Network for more. If the study you’re reading doesn’t mention or seem to follow these, beware, you are probably reading a poorly done paper.
There should be a clear study question and clear explanation of the analysis used to answer the primary question. If you can’t find this, throw the paper in the trash.
P values – We don’t accept the null hypothesis; we either reject or do not reject it. P values just above 0.05 should not be described as “a trend.” A trend indicates something is moving. A p value over 0.05 is not a trend; it just means an endpoint did not meet that measure of statistical significance.
A p value of 0.03 does not mean there is a 3% probability that results are due to chance. It doesn’t quantify the probability of a hypothesis. Rather, it is the probability of rejecting the null hypothesis when it is really true. Similarly, a 95% CI does not mean there is a 95% chance the true value falls in that range of numbers; it means that if the experiment was repeated in different samples, there is a 95% chance the true parameter value would fall in that range. At first glance, this seems like splitting hairs, but smart people swear it’s not. You stats wonks put something in the comments for the rest of us.
Statistical significance does not equal clinical significance. For example, you may find a statistically significant ½ point difference on a pain scale, but is decreasing pain from a 10 to a 9.5 really making your patient feel better?
Multivariate and propensity analyses help mitigate but do not remove the impact of confounders and cannot act as a substitute for a randomized controlled trial when it comes to determining causality. For instance, we shouldn’t say, “multivariate analyses removed confounding.”
It is more helpful to discuss the clinical impact of a result, rather than just the statistical facts. For example, one could report the sensitivity, specificity, and AUC for a test. But more relevant would be to report the stats facts and clinical import. For example, if the sensitivity for appendix ultrasound increased, X% of CT scans could be avoided.
The authors say we should avoid saying “may” or “might.” Oh boy…I do that all the time when I don’t want to convey causality. The authors note that saying a hypothesis “may” be true is the reason we do a study. It is also always a true statement unless a hypothesis is proven to false, which the authors point out is very difficult in science. Instead, we should say, “There is evidence that X was associated with Y, and a RCT is needed.” I may change the way I write in the future 🙂 .
Statistical Analysis and Reporting Guidelines for CHEST. Chest. 2020 Jul;158(1S):S3-S11. doi: 10.1016/j.chest.2019.10.064.