Justify the absence of NHST — Beta

(beta as in beta, not as in beta, or beta)

(In progress, so errors are possible, ok? Also, the list of alternatives is neither complete nor a definitive endorsement — read about them, think, decide.)

So you have read this letter, are aware of potential personal costs, and saw what you can do. And you decided that, even if it does increase the chances of rejection (possible, depending on the journal), you won't use NHST on your next article.

In this case, here's an option: justify that choice.

Below are text templates that you can copy as they are, mix, and modify as you please. These texts were written precisely to be freely copied and save you some time to improve other aspects of your research.

Why justify? To communicate that it was a conscious choice. Of course, some journals may choose to reject it anyway, so no guarantees.


Templates

------------------------------------------------------------------------------------------------

Main text

“Null hypothesis significance testing (NHST) isn't used in this paper. In spite of its persistent popularity, there's substantial evidence that the practice hinders scientific progress (Stang et al. 2010, Perezgonzalez 2015); is not appropriate for the biomedical and social sciences (McShane et al. 2018); and damages real lives (Stang et al. 2010, Gorard 2016, Gorard 2017, Greenland 2017, Berry 2017). Therefore, we join others in accepting that abandoning its use is justified, ethical, and recommended (Gorard 2016, Amrhein & Greenland 2018, McShane et al. 2018, Trafimow et al. 2018).”

References:

  1. Amrhein V, Greenland S. Remove, rather than redefine, statistical significance. Nat Hum Behav 2018. 10.1038/s41562-017-0224-0

  2. Berry D. A p-value to die for. Am stat 2017. doi: 10.1080/01621459.2017.1316279

  3. Gorard S. Damaging real lives through obstinacy: re-emphasising why significance testing is wrong. Sociol Res Online 2016; 21(2), 2. doi: 10.5153/sro.3857

  4. Gorard S. Significance testing is still wrong, and damages real lives. Sociol Res Online 2017; 22(2), 11. doi: 10.5153/sro.4281

  5. Greenland S. The need for cognitive science in methodology. Am J Epidemiol 2017; 186: 639-45. doi: 10.1093/aje/kwx259

  6. McShane BB, Gal D, Gelman A, Robert C, Tackett JL. Abandon statistical significance. arXiv preprint 2018. arXiv:1709.07588

  7. Perezgonzalez JD. Fisher, Neyman-Pearson or NHST? A tutorial for teaching data testing. Front Psychol 2015. doi: 10.3389/fpsyg.2015.00223

  8. Stang A, Poole C, Kuss O. The ongoing tyranny of statistical significance testing in biomedical research. Eur J Epidemiol. 2010 Apr; 25(4): 225-30. doi: 10.1007/s10654-010-9440-x.

  9. Trafimow D, Amrhein V, Areshenkoff CN, Barrera-Causil C, Beh EJ, Bilgiç Y, Bono R, Bradley MT, Briggs WM, Cepeda-Freyre HA, Chaigneau SE, Ciocca DR, Carlos Correa J, Cousineau D, de Boer MR, Dhar SS, Dolgov I, Gómez-Benito J, Grendar M, Grice J, Guerrero-Gimenez ME, Gutiérrez A, Huedo-Medina TB, Jaffe K, Janyan A, Karimnezhad A, Korner-Nievergelt F, Kosugi K, Lachmair M, Ledesma R, Limongi R, Liuzza MT, Lombardo R, Marks M, Meinlschmidt G, Nalborczyk L, Nguyen HT, Ospina R, Perezgonzalez JD, Pfister R, Rahona JJ, Rodríguez-Medina DA, Romão X, Ruiz-Fernández S, Suarez I, Tegethoff M, Tejo M, van de Schoot R, Vankov I, Velasco-Forero S, Wang T, Yamada Y, Zoppino FC, Marmolejo-Ramos F. Manipulating the alpha level cannot cure significance testing. PeerJ Preprints 2018; 6:e3411v2 doi: 10.7287/peerj.preprints.3411v2

------------------------------------------------------------------------------------------------

[higher priority than any particular statistical tool; read this paper]

“Prior and related evidence, plausibility of mechanism, study design and data quality, real world costs and benefits, and novelty of finding are important but usually neglected factors in research (McShane et al. 2018). They played a central role in guiding our research design and analyses.”

References:

  1. McShane BB, Gal D, Gelman A, Robert C, Tackett JL. Abandon statistical significance. arXiv preprint 2018. arXiv:1709.07588

------------------------------------------------------------------------------------------------

“Emphasis is given to estimation, reporting, and interpretation of effect sizes (Stang et al. 2010, Rothman 2014).”

References:

  1. Rothman K. Six persistent research misconceptions. J Gen Intern Med 2014 Jul; 29(7): 1060–1064. doi: 10.1007/s11606-013-2755-z

  2. Stang A, Poole C, Kuss O. The ongoing tyranny of statistical significance testing in biomedical research. Eur J Epidemiol. 2010 Apr; 25(4): 225-30. doi: 10.1007/s10654-010-9440-x.

------------------------------------------------------------------------------------------------

[since some options are complementary, choosing more than one is possible]
[look them up and use what makes sense to you; and/or use others as appropriate]
[remember: there's no magic objective universal tool for statistical inference]

“In addition, ...

were used, which we found appropriate for the present study.”

References:

  1. Bayarri MJ, Benjamin DJ, Berger JO, Sellke TM. Rejection odds and rejection ratios: a proposal for statistical practice in testing hypotheses. J Math Psych 2015; 72: 90–103. arXiv:1512.08552

  2. Benjamin DJ, Berger JO. A simple alternative to p-values. Comments on “The ASA's statement on p-values: context, process, and purpose”. Am stat 2016 Jun; 70(6): 129-133. doi: 10.1080/00031305.2016.1154108

  3. Gorard S. What to do instead of significance testing? Calculating the ‘number of counterfactual cases needed to disturb a finding’. Int J Soc Res Meth 2015 Jun; 19(6): 481-490 doi: 10.1080/13645579.2015.1091235

  4. Greenland S. The need for cognitive science in methodology. Am J Epidemiol 2017; 186: 639-645. doi: 10.1093/aje/kwx259

  5. Greenland S. The unconditional information in P-values and its refutational interpretation via S-values. 2018. Under submission.

  6. Trafimow D. Using the coefficient of confidence to make the philosophical switch from a posteriori to a priori inferential statistics. Educ Psychol Meas 2017. Online version.

------------------------------------------------------------------------------------------------

Please give feedback

If you have suggestions for improvement or want to give your impressions, please email.

And please, please let us know if you actually used it in any of your papers and what the outcome was. Evidence of how it's working could be valuable.