Null-hypothesis significance tests in behavioral and management research: We can do better

Abstract

Null-hypothesis significance tests (NHST) are a very troublesome methodology that dominates the quantitative empirical research in strategy and management. Inherent limitations and inappropriate applications of NHST impede the accumulation of knowledge and fill academic journals with meaningless “findings,” and they corrode researchers' motivation and ethics. Inherent limitations of NHST include the use of point null hypotheses, meaningless null hypotheses, and dichotomous truth criteria. Misunderstanding of NHST has often led to applications to inappropriate data and misinterpretation of results.

Researchers should move beyond the ritualistic and often inappropriate use of NHST. The chapter does not advocate a best way to do research, but suggests that researchers need to adapt their methods to reflect specific contexts and to use evaluation criteria that are meaningful for those contexts. Researchers need to explain the rationales that guided the selection of evaluation measures and they should avoid excessively complex models with many variables. The chapter also offers four more focused recommendations: (1) Compare proposed hypotheses with naïve hypotheses or the outcomes of alternative treatments. (2) Acknowledge the uncertainty that attends research findings by stating confidence limits for parameter estimates. (3) Show the substantive relevance of findings by reporting effect sizes – preferably with confidence limits. (4) Use statistical methods that are robust against deviations from assumptions about population distributions and the representativeness of samples.

Citation

Schwab, A. and Starbuck, W.H. (2009), "Null-hypothesis significance tests in behavioral and management research: We can do better", Bergh, D.D. and Ketchen, D.J. (Ed.) Research Methodology in Strategy and Management (Research Methodology in Strategy and Management, Vol. 5), Emerald Group Publishing Limited, Leeds, pp. 29-54. https://doi.org/10.1108/S1479-8387(2009)0000005002