Marketing pitfalls of statistical significance testing

Abstract

Purpose

To explore the appropriateness of statistical significance testing to measure the practical, managerial significance of outcomes in marketing programmes.

Design/methodology/approach

An in‐depth analysis of SST's scientific roots is coupled with delineation of a set of general objectives of marketing‐programme measurement to identify the applicability limits of significance testing.

Findings

In particular, it is shown that the relatively well known sample‐size dependence of SST and its somewhat lesser known replicability, representativeness and impact fallacies can severely affect the robustness of significance tests. Statistical significance is not the same concept as practical significance.

Practical implications

Comprehensive discussion of principles and practice leads to a set of prescriptive usage recommendations, directed at the goal of establishing much‐needed applicability rules and limits for the use of significance‐testing methodologies in an applied marketing context.

Originality/value

This robust challenge to the efficacy of significance testing in marketing practice should be of interest to any marketing planner concerned with the collection and use of marketing intelligence.

Citation

Banasiewicz, A.D. (2005), "Marketing pitfalls of statistical significance testing", Marketing Intelligence & Planning, Vol. 23 No. 5, pp. 515-528. https://doi.org/10.1108/02634500510612672