Research Methodology in Strategy and Management: Volume 5

Welcome to the fifth volume of Research Methodology in Strategy and Management (RMSM). We are delighted to provide you with this latest installment of the RMSM series and hope that you will find it to be as informative and educational as we do. This volume represents the work of a diverse set of scholars, some of whom are former presidents and fellows of the Academy of Management, some are ascending and up-and-comers, while others are highly accomplished methodologists who come from outside the strategy field. All of the authors have drawn upon deep and rich methodological experiences to put together excellent chapters. Their contributions not only address important and timely methodological topics but provide plain and straightforward insights as to how we can improve the application of research methods in strategy and management.

Purpose: The purpose of the paper is twofold: first, to examine whether the progress of strategic management research has been damaged by an excessive focus on statistical significance to the exclusion of substantive significance and second, to provide recommendations for improving research practice toward establishing the substantive significance of empirical findings.

Methodology/Approach: We conduct the same survey described in McCloskey and Ziliak (1996) on a sample of all 41 papers published in Strategic Management Journal during 2007 that use regression methodology. We use the criteria for good science represented by these survey questions as the foundation for our discussion. We present our arguments for the relevance of each of these criteria in strategy research with examples of best practice and provide a detailed analysis of areas of research practice that can be improved with associated recommendations.

Findings: Our survey suggests that there is indeed cause for concern, since 90% of our surveyed papers make no distinction between statistical and economic/substantive significance of their results. At the same time, many of the surveyed papers make some attempt to interpret their results in a substantively meaningful fashion.

Originality/Value of Paper: Our paper addresses a critical set of issues that influence progress in strategic management research. We provide a roadmap for how we can address these issues for progress in our field.

Null-hypothesis significance tests (NHST) are a very troublesome methodology that dominates the quantitative empirical research in strategy and management. Inherent limitations and inappropriate applications of NHST impede the accumulation of knowledge and fill academic journals with meaningless “findings,” and they corrode researchers' motivation and ethics. Inherent limitations of NHST include the use of point null hypotheses, meaningless null hypotheses, and dichotomous truth criteria. Misunderstanding of NHST has often led to applications to inappropriate data and misinterpretation of results.

Researchers should move beyond the ritualistic and often inappropriate use of NHST. The chapter does not advocate a best way to do research, but suggests that researchers need to adapt their methods to reflect specific contexts and to use evaluation criteria that are meaningful for those contexts. Researchers need to explain the rationales that guided the selection of evaluation measures and they should avoid excessively complex models with many variables. The chapter also offers four more focused recommendations: (1) Compare proposed hypotheses with naïve hypotheses or the outcomes of alternative treatments. (2) Acknowledge the uncertainty that attends research findings by stating confidence limits for parameter estimates. (3) Show the substantive relevance of findings by reporting effect sizes – preferably with confidence limits. (4) Use statistical methods that are robust against deviations from assumptions about population distributions and the representativeness of samples.

Firms and individuals budget or account for dollars, not standardized dollars, squared dollars, squared deviations from mean dollars, or percentage of squared deviations from mean dollars – my checking account reports my balance in dollars. In contrast, we have all seen a model dismissed because it “only explained 9% of the variance.” However, the Brogden–Cronbach–Gleser (BCG) model clearly shows that rxy (or

) is linearly related to a model's dollar utility to the firm, not

Opens in a new window.

or

Opens in a new window.

. In other words, when rxy (or

Opens in a new window.

) doubles for a strategic management model designed to predict profit (Y$), then the predicted dollar value added to the firm doubles (e.g., when rxy=0.30 and

Opens in a new window.

, the addition of X2 to the model has increased expected dollar value added to the firm by a factor of 2). Hence, a model that explains only 9% of the variance in Y$ in fact explains 30% of the dollar utility available to be explained in Y$, even though tests of the null hypothesis H0: rxy=0 and

Opens in a new window.

will yield mathematically identical outcomes to tests of

Opens in a new window.

and

Opens in a new window.

. Not surprisingly, I rarely see the BCG model cited in the scholarly management literature, and never see it cited by strategic management scholars. So, I will first demonstrate how the BCG model was originally developed to estimate the value of personnel selection systems, though it also characterizes how the dollar impact of any organizational intervention can be estimated, be it strategic, entrepreneurial, HR-related, etc. I will then make some minor adjustments to show how the model can be applied to more macro, strategic research arenas as well as some of the more interesting implications that are seldom fully appreciated in the current management literature. I will conclude this section with an example of how the BCG model might be applied to a recent strategic management study published in a recent issue of the Academy of Management Journal.

Management and especially strategy research rely heavily on the use of ratios to measure a variety of firm, industry, and societal characteristics. Most often, these ratios are created simply to control for size effects (i.e., scaling) emanating from differences in the size of firms, industries, populations, or national economies on the variables of interest. In addition, ratios may also hold theoretical meaning apart from that of their components. Despite the popularity of ratios and regardless of their purpose, the use of ratios is not without controversy. In particular, several studies have demonstrated that the use of ratio measures in correlations and multiple regressions can exaggerate relations of interest leading to biased and unstable results. In this chapter, I review the debate surrounding the use of ratio measures, discuss the problems for estimation and inference that are likely to arise when ratios are used in statistical estimation, and provide alternatives to the use of ratio variables that still satisfy the purpose for which ratio measures are often created.

This cautionary note provides a critical analysis of a statistical practice that is used pervasively by researchers in strategic management and related fields in conducting covariance structure analyses: The argument that a “large” sample size renders the χ2 goodness-of-fit test uninformative and a statistically significant result should not be an indication that the model does not fit the data well. Our analysis includes a discussion of the origin of this practice, what the attributed sources really say about it, how much merit this practice really has, and whether we should continue using it or abandon it altogether. We conclude that it is not correct to issue a blanket statement that, when samples are large, using the χ2 test to evaluate the fit of a model is uninformative and should be simply ignored. Instead, our analysis leads to the conclusion that the χ2 test is informative and should be reported regardless of sample size. In many cases, researchers ignore a statistically significant χ2 inappropriately to avoid facing the inconvenient fact that (albeit small) differences between the observed and hypothesized (i.e., implied) covariance matrices exist.

A multi-site, multi-source research methodology is suggested for scholars of corporate strategy in order to attain generalizability and statistical significance in reporting findings while not losing the nuances and understanding of each firm's environmental context.

Purpose: Case studies are detailed empirical investigations into a complex entity that emphasize the uniqueness of the case and are valuable for making a theoretical contribution. We aim to reveal the types of theoretical contributions case study research can make to the field of strategy and management and explore how case study design can create the opportunities for making a theoretical contribution.

Methodology/Approach: The dynamic capability approach focuses on the firm-specific processes through which firms integrate, build, or reconfigure resources. A comprehensive review of case studies in this field is conducted in five search engines, resulting in a data set of 13 in-depth case studies.

Findings: We demonstrate that using case studies to extend and refine theory enhances knowledge in the field of dynamic capabilities. In strategy and management research, case studies identify and refine constructs and their relationships, develop and confirm propositions, and embed constructs within a larger set of relationships. We reveal that sampling strategy, research setting, and multiple lenses are aspects of case study design that create opportunities for making a theoretical contribution.

Practical Implications: We suggest that case study researchers strategically and purposefully sample cases, vary the setting conditions, or draw upon numerous research fields to make a theoretical contribution.

Originality/Value of Paper: Going beyond the current discussion, we show that case studies have the potential to extend and refine theory. We shed new light on how dynamic capabilities can benefit from case study research by discovering the antecedents that shape the development of capabilities and determining the boundary conditions of the dynamic capabilities approach.

Strategy researchers have given very little attention to services, even though they now dominate the gross domestic product of almost all countries. We encourage more research focused on service as the basic mode of generating revenue today, especially as the economic landscape is being restructured by recent financial crises. This chapter suggests a basic framework for services research and then outlines issues in three areas that are particularly important to customer-oriented service innovation: individuation, standardization, and export. Illustrative examples from Germany provide more specific contexts for considering the range of activity in this under-researched domain.

Transaction cost economics (TCE) has received extensive attention from a variety of disciplines, but it holds a particularly central place in strategic management. The focal issues examined by TCE, vertical integration and interfirm governance (including contract design), are important determinants of firm performance – the central issue in the field of strategy. While several extensive reviews of empirical work in TCE have been undertaken, one key issue has received relatively little attention – construct validity in TCE empirical research. The purpose of this chapter is to highlight some of the challenges of operationalizing key transaction cost predictions and provide some ideas for better measuring core constructs such as asset specificity, uncertainty, and frequency.

We discuss recent methodological advances in the NK modeling in the Strategy literature and analyze issues related to its current use including different implementation algorithms, relative versus absolute performance, establishing significance of simulation results and long- versus short-term performance measurements. To facilitate cross-pollination of ideas, we point to advances and extensions of the model developed in other fields that could be effectively utilized to answer Strategy-related questions. These include modeling the strength of interaction, varying the importance of decision elements, utilizing alternative functional forms, incorporating endogeneity in N and K parameters and embedding the NK model in a broader dynamic framework.

This paper focuses on the potential advantages of strategic investment models in examining firm investment behavior. Strategic investment models are derived from rigorous modeling techniques grounded on formal analytical models, and they have been widely applied in corporate finance and economics to examine the problem of firm underinvestment. In this paper, we present an overview of strategic investment models, including empirical applications that highlight their methodological strengths. We conclude that the empirical application of such investment models in the context of strategic management research presents research opportunities in many new directions.

Finding answers to questions raised in a given debate presents scholars with the opportunity to not only provide a firmer path toward a resolution to the debate at hand, but also the opportunity to employ somewhat atypical methodological approaches. Specifically, when investigating different facets of a given debate, the opportunity presents itself to (1) employ competing hypotheses, (2) integrate theoretical frameworks, and (3) identify boundary conditions.