Performance measurement and management: theory and practice

Citation

Bourne, M., Melnyk, S. and Bititci, U.S. (2018), "Performance measurement and management: theory and practice", International Journal of Operations & Production Management, Vol. 38 No. 11, pp. 2010-2021. https://doi.org/10.1108/IJOPM-11-2018-784

Performance measurement and management: theory and practice

Introduction

Performance measurement has been a key theme of this journal for many years (Neely et al., 1995; Neely, 2005). This should not come as a surprise since performance measurement and management plays a critical role in the operation of any organisation, be it a factory, business, hospital or school. As noted by Magretta and Stone (2002), performance measures are critical because they enhance communication – they enable the organisation to address the following critical question – “Given our mission, how is our performance going to be defined?” Yet, it should also come as a surprise that in spite of this longevity of focus, this topic is still surprised by a great deal of confusion and conflict. For example, until more recently we have not agreed on what we mean by performance measurement (Franco-Santos et al., 2007; Bourne and Bourne, 2011; Melnyk et al., 2004, 2014) and performance management (Bourne and Bourne, 2011; Melnyk et al., 2004, 2014). This confusion is present in both academic research and in the practitioner press. Confounding this confusion is the tight interrelationship that exists between theory and practice.

It has been long argued that many of the developments in performance measurement have come from practice (Johnson, 1972, 1975, 1978, 1981; Wilcox and Bourne, 2003) and this practice has informed the academic performance measurement literature. A very clear example of this phenomenon is the Balanced Scorecard, originally developed at Analogue Devices (Schneiderman, 2001) and subsequently made widely accessible by Kaplan and Norton (1992). But although this type of research is to be applauded, it is not enough. Ultimately, it suffers from being a theoretical, using Sutton and Staw (1995) categorisation scheme. Such research, while important for improving performance and the ability of the firm to record and monitor activities, does not build causality or help to explain or, as is becoming more important in today’s dynamic environment (Nudurupait et al., 2016), predict and deal with the increasing complexity now being faced by researchers and managers alike.

We need to build theory, whether from practice or other areas of business research, and validate our theory through empirical research to develop a deeper understanding and platform for the future development of the field. Furthermore, we need to build theory that operates at the various levels of the organisation – inter-organisational (relevant for a world where supply chain management is becoming so important); organisational at macro or top management levels (the focus of much of the current body of research in this area) and the lower levels of the organisation (i.e. at the function, group and, ultimately, the individual levels – the micro levels). We need to build theories that are anchored not only in the economic world but that also recognise the increasing importance of behavioural issues. So, for this special issue, we were particularly interested in papers that explore the interface between theory and practice and that add richness, as described in the preceding discussion, to the resulting theoretical developments.

But performance measurement and management are broad subjects and performance measurement and management systems (PMMS) have a wide scope. They include the top down processes of aligning the business with strategic direction as promulgated by process approaches to the development and implementation of PMMS (e.g. Kaplan and Norton, 1993; Neely et al., 1996; Bititci et al., 1997; Neely et al., 2000) and to the bottom up use of performance measurement in lean (Bhasin and Burcher, 2006; Krafcik, 1988). They include the dynamics of people and teams, interactions between department, relationships between parent and subsidiary organisations as well as performance measurement and management in the supply chain.

Performance measurement and management is not only the preserve of the private sector. Articles in this journal have reported on the impact of PMMS in both public and third sector organisations (Greatbanks and Tapp, 2007; Moxham and Boaden, 2007). This special issue too reflects this pervasiveness of PMS, despite the fact that the papers published here happen to have focused on the private sector. We have a number of different perspectives on performance measurement and management. This includes three papers on buyer–supplier relationships, one on collaborative performance measurement systems, one the impact of supplier performance management systems (PMSs) and one on contracting. We have five papers looking at the complexities and interactions between different elements of the performance measurement system inside organisations with two of these focusing on lean. The final paper develops a framework for swift and even flow.

So in the rest of this editorial we will discuss the following. Let us begin with an overview of the early literature and the development of the four phases of PMMS before turning our attention to the need for theory to focus on the use of PMMS in directing and managing organisations. Then we will move on to the range of theory used in empirical research and in the papers presented here before going on to describe the contributions made in this special issue. We will then discuss the contribution of the Ferriera and Otley (2009) framework before suggesting future direction for theory development.

The use of theory in performance measurement and management

Early literature on PMMS focused on the four phases of PMMS, design, implementation, use and refresh (Neely et al., 2000; Bourne et al., 2000). There is a strong argument that the debate has now moved on from the design and implementation of PMMS to its use, Franco-Santos and Bourne, 2005), impact (Franco-Santos et al., 2012) and emergence (Pavlov and Bourne, 2011).

The work on approaches (Dixon et al., 1990) and processes for the design and implementation of PMMS (Neely et al., 1997; Bititci et al., 1997; Olve et al., 1999) initially adopted the three tests on feasibility, usability and utility (Platts, 1993) before considering the success and failure of the design to implementation phase (Bourne et al., 2002; Bourne, 2005) and the quality of implementation. One of the overarching pieces here was the Bititci et al. (2006) paper which identified the need for different cultures and management styles when moving from the design and implementation phases to measurement in use phases. We would argue that it is possible to theorise about the design and implementation phases but in reality, these are as much about the implementation of change (Bourne et al., 2004) as anything else. There have also been interesting insights into the formal approaches to refreshing measurement systems (Kennerley and Neely, 2003) and more recently addressing the issue of how to keep measurement systems up to date (Melnyk et al., 2014), but we strongly suggest that there is now a compelling need to develop theory around the continuing use and emergent development of PMMS.

In their paper, Franco-Santos et al. (2012) identified that the empirical work undertaken to research the impact of performance measurement systems on performance had focused on the use of six groups of theories, although it should be noted that a third of the empirical papers reviewed in their paper used no theoretical basis at all. The six groups of theories were:

1. agency theory (Eisenhardt, 1989; Feltham and Xie, 1994; Jensen and Meckling, 1976; Jensen and Murphy, 1990);

2. contingency theory (Donaldson, 2001; Hayes, 1977; Otley, 1980);

3. resource-based view of the firm (Barney, 1991; Day, 1994);

4. cognitive and information processing theories (Miller, 1956; Simon, 1976; Talyer, 2010; Kunda, 1990);

5. goal setting theory (Locke and Latham, 1990); and

6. equity, distributive and procedural justice theories (Adams, 1965; Greenberg, 1990).

In this special issue, we have papers too that use theories matching three of the six groups of theories identified in Franco-Santos et al. (2012). Papers published here are using resource-based view (2), cognitive theories and decision making (2) and contingency theory. But we also have papers focusing on theories more relevant to PMMS in operations (performance-based contracting (PBC) and swift even flow) as well as theories related to complementarity in PMMSs, and extending the concept of levers of control (Simons 1991). The papers in this special issue are summarised in Table I. We will briefly describe these papers next.

The first paper by Vieri Maestrini, Veronica Martinez, Andy Neely, Davide Luzzini, Federico Caniato and Paolo Maccarrone looks at how buyers and suppliers can collaborate in their use of their performance measurement system. In this paper, they present a tool they have called the “Relationship Regulator” which they develop, test and refine. The development was based on the literature and empirical research, whilst the testing was undertaken through workshops and feedback obtained from semi-structured interviews. This is theory building and testing of a collaborative approach to performance measurement.

The second paper by Vieri Maestrini, Davide Luzzini, Federico Caniato, Paolo Maccarrone and Stefano Ronchi is an impact paper in that it researches the impact of supplier performance measurement systems on supplier performance through hypothesis testing using survey data. However, the interest in this paper is the use of resource orchestration theory (ROT) as the theoretical framework for their analysis. They found ROT to be a suitable theoretical framework to explain the role of a mature supplier performance measurement system in orchestrating the suppliers.

The third paper by Andreas Glas, Florian Henne and Michael Essig is a literature review on the intersection of PBC and performance measurement and management. The review highlights the performance measurement and management gap in PBC identifying for research opportunities: strategic alignment (which the authors consider astonishing as PBC is supposed to be outcome orientated), action and refreshing, performance monitoring and reporting, and other aspects including the selection of appropriate KPIs.

The fourth paper by Mohamed Afy-Shararah and Nicholas Rich creates a model that captures the design elements of high performance operating management systems for swift even flow. The work builds on ten longitudinal case studies selected from the UK’s high value manufacturing sector highlighting the links between policy deployment, operational control and improvement to open collaborative partnerships.

The fifth paper by Henrik Nielsen, Thomas Kristensen and Lawrence Grassol is the first of two papers focusing on lean. The paper uses survey research to investigate the impact of social control mechanisms, behavioural control mechanisms and their complementarity on firm performance using data collected from over 4,000 subjects in nearly 700 facilities. The respondents to the survey behind this work were identified from through the “Shingo Prise”.

The sixth paper by Martijn van der Steen and Sandra Tillema takes a different approach and looks at lean implementation. Using case studies in three subsidiary companies, they suggest that lean can be severely constrained by the parent organisations accounting based control systems. The paper demonstrates how external context creates local conditions that may be detrimental to lean in manufacturing.

The seventh paper by Bijana Pesalji, Andrey Pavlov and Pietro Micheli uses the levers of control framework from Simons (1991) to investigate practices in a Dutch-based SME. The paper advances our understanding of the use of technical and social aspects of performance management and suggests that performance management requires the active and continuous use of all four control mechanisms that comprise the levers of control approach.

The eighth paper by Marcus Hasegan, Sai Nudurupati and Stephen Childe reports on the use of action research for developing dynamic performance measurement systems with real-time controls on the production lines to study the impact. The paper explains how the use of tacit knowledge and modelling were used in developing effective cause and effect analysis.

The last paper by Anthony Alexander, Maneesh Kumar and Helen Walker considers the application of decision theory under volatility, uncertainty, complexity and ambiguity (VUCA). Drawing on the Cynefin framework (Snowden, 2000, 2002; Snowden and Boone, 2007), this paper develops the performance alignment matrix (Melnyk et al., 2014) drawing on interview research in seven case studies. The approach is designed to deal with complexity and emergence enabling managers in the positions of authority to take decisions. This paper highlights the need for organisations to adjust their performance measurement and management system over time to adapt to the external environment as a way of reformulating strategy, promoting intended behaviour and organisational learning.

So, from a theory and practice perspective, we must conclude that the special issue has attracted a wide variety of papers, research methods and applications from different contexts. The papers contribute to our understanding of performance measurement and management in buyer–supplier relationships, performance contracting and operational flow. The papers create insight into the issue of complimentary or conflicting aspects of control systems, especially in a lean setting and we have insights into balancing controls in SMEs as well as how to deal with VUCA environments. In the next section, we will focus of discussion on theory in PMMS.

Discussion

If we review the theoretical approach taken both the papers in this special issue and by those empirical pieces identified in the Franco-Santos et al. (2012) literature review, it must be noted that they are not theories of performance measurement and management, but more general management theories applied to this subject area. What we are currently lacking is an underpinning theory to help us advance the field.

However, the field is not completely without a theoretical underpinning, the Ferriera and Otley (2009) theoretical framework is a very useful step in this direction as it gives us a framework and a set of 12 questions to help us analyse PMMSs. The 12 questions developed by Ferriera and Otley (2009, pp. 266-267) are as follows:

1. What is the vision and mission of the organisation and how is this brought to the attention of managers and employees? What mechanisms, processes and networks are used to convey the organisation’s overarching purposes and objectives to its members?

2. What are the key factors that are believed to be central to the organisation’s overall future success and how are they brought to the attention of managers and employees?

3. What is the organisation structure and what impact does it have on the design and use of PMSs? How does it influence and how is it influenced by the strategic management process?

4. What strategies and plans has the organisation adopted and what are the processes and activities that it has decided will be required for it to ensure its success? How are strategies and plans adapted, generated and communicated to managers and employees?

5. What are the organisation’s key performance measures deriving from its objectives, key success factors and strategies and plans? How are these are specified and communicated and what role do they play in performance evaluation? Are there significant omissions?

6. What the level of performance does the organisation need to achieve for each of its key performance measures (identified in the above question), how does it go about setting appropriate performance targets for them and how challenging are those performance targets?

7. What processes, if any, does the organisation follow for evaluating individual, group and organisational performance? Are performance evaluations primarily objective, subjective or mixed and how important are formal and informal information and controls in these processes?

8. What rewards – financial and/or non-financial – will managers and other employees gain by achieving performance targets or other assessed aspects of performance (or, conversely, what penalties will they suffer by failing to achieve them)?

9. What specific information flows – feedback and feedforward – systems and networks has the organisation in place to support the operation of its PMSs?

10. What type of use is made of information and of the various control mechanisms in place? Can these uses be characterised in terms of various typologies in the literature? How do controls and their uses differ at different hierarchical levels?

11. How have the PMSs altered in the light of the change dynamics of the organisation and its environment? Have the changes in PMSs design or use been made in a proactive or reactive manner?

12. How strong and coherent are the links between the components of PMSs and the ways in which they are used.

We should reflect that the framework, whilst informed by the management control literature, was developed inductively and informed by the experience of case study research (Ferriera and Otley, 2009, p. 276), which is why it has such a strong link to practice and why it is so useful as a holistic tool for examining the structure, operation and use of PMSs in organisations.

In many ways, this framework follows some of the early research works (Neely et al., 1996) and publications (Kaplan and Norton, 1993, 2001) on PMMS, whereby the vision and mission of the organisation is translated into a strategy that is operationalised through the performance measurement and management systems. However, the Ferriera and Otley (2009) framework progresses far beyond this as it includes the development of targets, the processes of evaluation, information flow and link to rewards. Further, question 11 alludes to the dynamic nature of PMMSs and recognises that they evolve over time; so their development is not solely determined through interventions such as redesign initiatives.

Although this is an extremely useful theoretical framework, it is not a theory of performance measurement or performance management. To develop such a theory, we need to understand the mechanisms at play in the organisation when performance measures are being used to manage activities, changes of activity and future direction of the organisation. There have been calls for such a meta theory (Franco-Santos et al., 2012; Bittitci et al., 2018) and we will suggest in the next section one possible direction to take in developing such an approach.

Towards one theory of performance measurement and management

The Ferriera and Otley (2009) framework described above assumes that PMMSs are systems. In reality, they may be “systems of systems” (Bourne et al., 2018) but this framework does align with a recent call to take a more systemic approach to developing theory in performance measurement and management (Bittitci et al., 2018). These systems operate through practices and routines in organisations and it is to this subject we turn next.

If we reflect on the current theories used to inform PMMS research, it can be argued that they ignore the mechanisms by which the PMMS operates. From Franco-Santos et al.’s (2012) six theories, four of them (agency, cognitive, goal setting and equity theories) focus on the influence of system parameters on what is usually considered to be the individual decision maker. The other two focus on the impact of the wider environment, be this the external (contingency theory) or internal (resource-based view) context. What we are not arguing here that these are not useful lenses to take in analysing the PMMSs in organisations, we are simply suggesting that adopting an alternative approach that takes a systems perspective by focusing of the operating mechanisms of PMMS could give use new insights.

One such approach is the adoption of routines (Pavlov and Bourne, 2011). In their paper, the authors argue that performance measurement triggers, guides and intensifies the search for solutions to improve the performance of the organisation (see Figure 1). As this happens in performance review meetings (regular events where groups of managers meet to review, evaluate and act on performance information (Martinez et al., 2010); Pavlov and Bourne, 2011) when routines are developed which evolve into mechanisms used in the management of the organisation. This approach takes us away from the concept of the individual decision maker to the domain of multiple decision makers (albeit in a situation where some have considerably more power and influence than others). Understanding the mechanisms at play here would give us different insights.

In reality, organisations have multiple situations where performance is reviewed. These may be formal board meetings, operational planning meetings, sales management meetings, project management meetings, etc. Decisions may be made by individual decision makers acting on their own (although we would suggest that this happens far more infrequently than one would surmise from the focus of research in the management literature) but implementing action invariably requires involving others. This suggests (in all but the smallest organisations) a series of links between individuals at different levels of the organisation. Individual managers involved in running one routine (performance review meeting) will usually be participants in a higher level performance review meeting, whilst the people attending this manager’s meeting may well be running their own routine (performance review meeting) at a lower level in the organisation. In this situation, routines being influenced and guided by other routines (see Figure 2) together with the individual managers’ membership of performance review meetings being the conduit of the PMMS between different levels in the organisation (see Figure 3).

Because these mechanisms are routines, they develop over time as the mental models of what is happening influences the pattern of behaviours and the pattern of behaviours influence the mental models. This is not only sense making in practice, but probably also how emergent strategy develops and is implemented outside the formal mechanisms for strategy development in most organisation.

Conclusion

The domain of performance measurement and management continues to develop and, if performance measurement and management evolve to enable us to control new and emerging forms of organisations in new and emerging contexts, then it will always continue to do so. In this special issue, we have seen the development of interesting new frameworks and understanding based on using theory to reflect on practice and practice to inform our development of theory.

However, we still believe that future research in this field would benefit from adopting a more systems-based approach to understanding the mechanisms at play in PMMS. We have suggested that understanding these mechanisms in practice would be beneficial to the field and would give us a platform for future research. But, if and when we develop this better understanding, we suspect that we will still be using other more encompassing management theories, such as those used and identified here, to understand the pressures and influences on the mechanism.

Figures

Figure 1

Effects of PM on organisational routines

Figure 2

Effects of the structure of PMMS on organisational routines

Figure 3

The overlap of points of membership in the hierarchy of organisation routines

Table I

Summary of papers in this special issue

Authors	Subject	Theoretical lens	Contribution
Vieri Maestrini, Veronica Martinez, Andy Neely, Davide Luzzini, Federico Caniato and Paolo Maccarrone	The development of a smart tool to allow buyer–supplier dyads to collaborate on relationship performance	Framework building and testing	A tested and improved buyer–supplier tool
Vieri Maestrini, Davide Luzzini, Federico Caniato, Paolo Maccarrone and Stefano Ronchi	The impact of supplier performance measurement systems on supplier performance	Resource orchestration theory (Hitt et al., 2011)	Resource orchestration theory explains the role of a mature supplier performance measurement system in orchestrating suppliers
Andreas Glas, Florian Henne and Michael Essig	Performance measurement and management in performance-based contracting	Performance-based contracting	Creates a cross-understanding between performance measurement and management and performance-based contracting
Mohamed Afy-Shararah and Nicholas Rich	A systems approach to measuring and managing flow performance in high value manufacturing	Systems theory Swift even flow (Schmenner and Swink, 1998)	Design elements of a high performing operational management system for swift even flow
Henrik Nielsen, Thomas Kristensen and Lawrence Grasso	The performance effects of complimentary control mechanisms in the implementation of lean	Complementarity of management control systems (MCS)	Deeper understanding of how the complementary effects of lean manufacturing impact on firm performance
Martijn van der Steen and Sandra Tillema	Implication of context on lean implementation	Contingency	Better understanding of how organisational controls, especially accounting control, impact on lean
Biljana Pešalj, Andrey Pavlov and Pietro Micheli	A levers of control perspective on the use of MCS and performance measures in SMEs	Levers of control (Simons, 1991)	Extending the concept of levers of control by identifying practices and the importance of maintaining balance
Marcus Hasegan, Sai Nudurupati and Stephen Childe	A dynamic performance measurement system (DPMS)	Dynamic capabilities Decision making	An understanding of using cause and effect relationships in manufacturing to better predict performance
Anthony Alexander, Maneesh Kumar and Helen Walker	Decision theory and the emergence with complexity in performance measurement and management	Decision theory	Dealing with complexity and emergence, enabling managers in positions of authority to make decisions