Statistical guidance to authors at top-ranked scientific journals:A cross-disciplinary assessment

Scientific journals may counter the misuse and misinterpretation of statistical methods by providing statistical guidance to authors. Here, we sought to assess the nature and prevalence of statistical guidance offered to authors by 15 journals (top-ranked by Impact Factor) in each of 22 scientific disciplines (N = 330 journals). For each journal, two investigators manually extracted and classified statistical guidance provided on the journal website. 160 (48%) journals offered some statistical guidance and 93 (28%) had a dedicated statistical guidance section in their author instructions. Statistical guidance was most common in biomedical and health-related disciplines. Among twenty prespecified statistical topics, only two were mentioned in more than a quarter of the journals: confidence intervals (n = 90, 27% of journals) and p-values (n = 88, 27% of journals). Guidance on these topics was inconsistent across journals. For six ‘hotly debated’ topics, we found only three journals that explicitly opposed the use of “statistical significance”; more commonly, journals implicitly endorsed the use of p-values (n = 77), statistical significance (n = 35), and Bayesian statistics (n = 39) and explicitly endorsed reporting of effect sizes (n = 62), confidence intervals (n = 85), and sample size justifications (n = 67). Our investigation shows large gaps and inconsistent coverage in statistical guidance provided by top-ranked journals across scientific disciplines. These data may help to inform debates about the appropriate role of journal policy in addressing statistical issues.