The argument over p-values and the Null Hypothesis Significance Testing (NHST) paradigm

Matt Homer

• Download the slides for this seminar

Abstract

• What is a p-value?
• Why is the significance level set at 5%?
• What is a 95% confidence interval?
• What does ‘statistical significance’ really mean?

If you struggle with these concepts, then fear not – you are certainly not alone.  There is evidence that even teachers of statistics and research methods sometimes get these questions wrong!

http://myweb.brooklyn.liu.edu/cortiz/PDF%20Files/Misinterpretations%20of%20Significance.pdf

The acknowledged problems with NHST, including its apparent misuse in applied research, has led to the recent and somewhat controversial decision of a leading psychology journal, Basic and Applied Social Psychology, to ‘ban’ p-values.  See the editorial announcing the decision: http://www.tandfonline.com/doi/pdf/10.1080/01973533.2015.1012991

This has stimulated a huge debate (really!) in the statistical blogosphere – see, for example, the Royal Statistical Society’s response to the ‘ban’:

http://www.statslife.org.uk/news/2116-academic-journal-bans-p-value-significance-test

So what is the problem? Why do people find these concepts so difficult, why are so many research findings based on using NHST questionable, to say the least, and should we ‘ban’ what was previously considered a foundation of the quantitative research approach?

This talk will discuss the arguments for and against NHST, how it can easily be abused, and will outline what is considered the ‘best’ practice for applied researcher with regard to the carrying out of significance tests.