March 5, 2024

Statistical Fragility in Surveys: A Cautionary Tale

A recent study by Carnes et al. aimed to examine the effect of a workshop on overcoming bias and improving work climate in departments of medicine. While the goals were admirable, the results should be interpreted with caution due to statistical fragility. Low and uneven response rates, small effect sizes, and multiple significance tests raise concerns about the robustness of the findings. Future studies should aim for higher response rates and larger effect sizes to demonstrate meaningful improvements in these critical issues.

Heston TF. Statistical Fragility in Surveys. Acad Med 2024 Mar 1;99(3):240. doi: 10.1097/ACM.0000000000005584. Epub 2023 Dec 7.