On April 1, 2024, APA submitted comments (PDF, 228KB)opens in new window in response to a National Institute of Neurological Disorders and Stroke (NINDS) request for information about publishing null research resultsopens in new window. Specifically, NINDS sought public input on the barriers and solutions to reducing publication bias (i.e., the preferential dissemination of statistically significant or otherwise exciting studies) in biomedical research. The request for information noted, “dissemination of positive as well as null studies (studies where the primary endpoint or key finding does not reach statistical significance) is vital for scientific progress and accurate assessment of cumulative evidence. However, formal dissemination of null studies is rare. Thus, NINDS is seeking input on potential solutions that will address publication bias across biomedical research fields.”
Responding on behalf of APA, Chief Science Officer Mitch Prinstein, PhD, wrote, “all APA core journals encourage the submission of replication studies, regardless of their results, including null findings (i.e., where study hypotheses were not supported or failed to reach conventional levels of statistical significance, yet can still contribute to advancing the literature).”
Prinstein explained that APA has endorsed the Transparency and Openness Promotion (TOP) Guidelinesopens in new window, (Nosek, B., et al., Science, 2016) developed in part to address some of the gaps in the literature due to the “file drawer” problem of null results languishing in metaphorical file drawers.
APA journal staff have observed that authors are pressured to publish in highly cited journals whose editors are reluctant to dilute the perceived quality of their journals by publishing articles whose findings are not novel or significant. Editors may be reluctant to lower their journals’ impact factors by publishing work that is likely to be cited less often. (Duyx, B, et al., Journal of Clinical Epidemiology, 2017).
Prinstein cited APA’s recent Task Force Report on Promotion, Tenure, and Retention of Faculty of Color in Psychology (PDF, 1.6MB)opens in new window, commenting that because the gauntlet of the tenure and promotion process is especially fraught for faculty of color, journal impact factors that are lowered by the publication of null results may even more adversely influence advancement opportunities for those faculty.
Regarding potential remedies to support the publication of null findings, APA noted that 18 of its journals publish registered reports, an article type in which a protocol is peer reviewed for its study design and methodology and accepted on that basis—as long as the final manuscript adheres to the study protocol, it will be published, regardless of results. A more widespread use of preprints by scientific journals would shrink the “file drawer problem.”
For more information, contact Pat Kobor.
