Factors influencing control charts usage of operational measures

Abstract

Purpose

The purpose of this paper is to identify factors influencing implementation of control charts on key performance indicators (KPIs).

Design/methodology/approach

Factors driving organizational change described in literature are analyzed inspired by the affinity-interrelationship method. A holistic multiple-case design is used to conduct six workshops to affect the usage of control charts on KPIs at a global company in the automotive industry. The theoretical factors are compared with the result from the case study.

Findings

The important factors for implementation success differ to some extent between the theoretical and empirical studies. High-level commitment and a clear definition of the goal of change could be most important when creating a motivation for change. Thereafter, having a dedicated change agent, choosing an important KPI and being able to describe the gain in financial terms becomes more important.

Practical implications

By using control charts on KPIs, the organization in the case study has become more proactive, addressing the right issues upstream in the process, in the right way, cross-functionally.

Originality/value

Factors affecting the implementation of already available solutions in the industry are highlighted. This potentially provides a basis for improved decision making, which has a significant value.

Citation

Ericson Öberg, A., Hammersberg, P. and Fundin, A. (2017), "Factors influencing control charts usage of operational measures", Measuring Business Excellence, Vol. 21 No. 3, pp. 225-238. https://doi.org/10.1108/MBE-08-2016-0041

Introduction

It is surprising how many good ideas never make it to the shop floor. No matter how extraordinary the research is, as long as it is not implemented, the value to the company is limited:

We are looking at new technologies, hoping they will solve our problems. At the same time we already have a lot of unimplemented solutions, just waiting for us. The potential of being able to implement just a few of them is very big (Technical manager at an international automotive company).

One of these solutions, just waiting to be generally implemented for operational data, is the control chart. The control chart is not a new invention; it has been used since the 1920s when it was initiated by Dr Walter A. Shewhart (Berger et al., 2002). Control charts can be used to study variation to distinguish between assignable causes of variation and random noise in many types of products and processes. Raj et al. (2000) state that control charts are only useful for the regulation of manufacturing processes. Caulcutt (1996) on the other hand states that statistical process control (SPC), which includes the control chart, is not simply a collection of tools but a way of thinking about variability. He continues that any manager who does not understand the most fundamental concept of SPC will very likely cause a waste of resources (when overreacting on random variation). This is applicable to any manager of any process, not simply manufacturing, according to Caulcutt. Danielsson and Holgård (2010), Deming (1994), Roth (2005), Dull and Tegarden (2004) and Wheeler (2000) analogously show that control charts could be very useful for all sorts of strategic information in a company, for example, key performance indicators (KPIs) on management level.

The literature available in the field of performance measurement systems as well as descriptions of different type of control charts is extensive. A gap however exists regarding to what extent control charts are used for KPIs in the manufacturing industry (Ericson Öberg et al., 2016). The purpose of this paper is to better understand the underlying factors that would facilitate a successful implementation, hence the research question:

RQ.

What are the important factors for a perceived successful implementation of control charts of KPIs?

There are numerous models describing change management and factors influencing implementation in general, many based on the stages described by Lewin (1947). They are however not focusing on control chart implementation specifically but more general Six Sigma implementation, e.g. described by Pinedo-Cuenca et al. (2012). In the study described in this paper, theories of factors influencing the implementation rate were instead condensed into a model overview containing 20 themes. Holistic multiple-case studies in the form of workshops addressing variation and the use of control charts on KPIs were conducted to reveal how factors influence the implementation. The workshops were conducted with management teams in one large manufacturing company. The resulting degree of implementation for each case was assessed together with its fulfillment of each theme. Surprisingly, there are indications that themes not frequently presented in the current literature have a large impact on the dissemination. In this particular case, this new practice has proven to align the organization to address the right issues upstream in the process in the right way to act for continuous improvement.

The paper will first introduce the ideas behind control charts followed by theories of business performance measurement systems and change management. After an overview of the methodology used to collect empirical data, findings are presented in relation to current theories. After that the analysis is presented, in which the theories and empirical findings are combined. The paper concludes with a discussion and ideas for future research.

Theoretical framework

The theoretical framework presents theories on variation and control charts and also gives an overview of business performance measurement systems and change management.

Variation and control charts

In quality improvement, variation can be primarily divided into two types, assignable-cause variation and chance-cause variation (Shewhart, 1931) also termed special-cause variation and common-cause or non-assignable cause variation (Deming, 1994). Assignable-cause variation is an unpredictable deviation resulting from a cause that is not an intrinsic part of the process. In general, assignable-cause variation can be traced back to one or a few causes. Variation caused by chance is instead random variation present in stable processes stemming from many underlying causes.

According to Bergman and Klefsjö (2010), two types of mistakes can be made in this context; to react on random phenomena as if they had assignable causes and not to react on assignable causes but mistake them for random variation. A control chart can assist in deciding which type of variation is present. The control chart was initiated by Shewhart in the 1920s (Shewhart, 1926). A control chart consists of a central line, upper control limit, lower control limit and plotted data points (Figure 1). The control limits are statistically calculated and express the natural variation of the process, its designed performance. Control limits should not be mixed up with tolerance limits that are set by customer requirements. When a data point falls outside any of the control limits or an unnatural pattern is shown, it is a sign of a special cause and process instability. A control chart can detect process changes before they drift out of tolerance. Depending on the type of data, different control charts can be used.

The possibility to provide facts to aid decision-making is one of the main advantages of using control charts on KPIs. By presenting the data using a control chart, the manager will be able to ask the right questions to reveal causes instead of symptoms. The focus is on the process, the variation and any signs of instability. Perhaps more important, a control chart will reduce the risk of asking the wrong questions and keeping the organization busy trying to eliminate random variation in vain. Wilcox and Bourne (2003) argue that the possibility to predict performance by introducing Shewhart’s methods to performance measurement methodologies can only add rigor to the process of developing measures and setting realistic targets. They also conclude that adaptations such as Six Sigma have lost the emphasis on prediction and follow more mathematical approaches.

Business performance measurement systems

Strategic measures are identified and followed-up to get information about the business. Based on the result, decisions are made. A performance measurement system is defined by Neely et al. (1995) as a set of metrics used to quantify both efficiency and effectiveness of actions. The research carried out in the area of business performance measurement is extensive. According to Bititci et al. (2012), the performance measurement field seems to have developed over a number of phases: productivity management, budgetary control, integrated performance measurement and integrated performance management. Kaplan and Norton’s concept of the balanced scorecard influences much of the work on performance measurement and management (Kaplan and Norton, 1992). Bourne (2008) explains that the expectation on the balanced scorecard was that the introduction of multi-dimensional KPIs would solve all problems. However, in some respects, non-financial KPIs are more difficult to design and use than accounting measures. In their review, Franco-Santos et al. (2012) highlight the idea that it is not only the contemporary performance measurement system that matters but also the capability of managers and employees to respond to it. The performance measurement system affects communication processes by requiring and providing relevant information that influences how people think, act and interact and affects organizational routines and management practices by changing the way leaders behave. This was highlighted by Bourne et al. (2000) as early as 2000 when they discussed the lack of research on implementation and use of performance measures. Bourne et al. also state that skills need to be developed in critiquing and learning from the performance measures in a group.

Initiate and implement change

There are numerous models describing change management and factors influencing implementation. In this study, a selection of the current theories of factors influencing the implementation are collected using traditional literature review (Jesson et al., 2011) as can be seen in Table I (Kotter, 1995; Jick, 1991; Garvin, 2000; Mento et al., 2002; Deleryd et al., 1999; Does et al., 1997; Sannö, 2015; Hallencreutz, 2012; Jørgensen et al., 2009; Tanner and Oakland, 2007; Vandermerwe and Vandermarwe, 1991; Mann and Kehoe, 1995; Dale and Shaw, 1991; Kotter and Cohen, 2003; Isaksson, 2006). The choice of theories included was made to cover a wide range of time to make sure also earlier ideas were represented. Both empirically and theoretically based models are included. The purpose is not to get a full coverage but rather reflect existing views.

Kurt Lewin’s model of change (Lewin, 1947), divided into unfreeze, move and freeze, has been the basis for many of the models. The first stage that he describes, unfreeze, involves moving ourselves or a department or an entire business toward motivation for change. The second stage includes making the changes that are needed. The final stage is about establishing stability, in which the changes are accepted and become the new norm.

The models included are based on empirical cases or are frameworks built on other theoretical models. They are often divided by their authors into parts named steps, factors, resources, challenges or fundamentals. The common view seems to be that change management consists of several parts. That will be used as a basis when analyzing the models.

Methodology

This section describes the methodology used to define clusters of the change management theories and the holistic multiple case study in the form of workshops.

Organizing themes of change management theories

The factors identified in the change management models were analyzed inspired by the affinity-interrelationship method (AIM) (Alänge, 2009), sometimes also called the KJ-method from its Japanese originator Jiro Kawakita (Bergman and Klefsjö, 2010). The method is based on the first two of the seven management tools suitable when organizing data (Bergman and Klefsjö, 2010). To begin, 157 factors influencing implementation were identified. They were reduced to 141 when two models by Vrakking (1995) and Does et al. (1997) were rejected due to their incompatible structure. The factors were sorted and grouped according to their similarities. A heading describing the group content was created for each group, 38 in total. These second-level groups were then again sorted and grouped to create new groups on the first level, now 20 in total. A heading describing each new group was created. The relationship between the groups was illustrated using arrows.

Case study design

Because of the promising results of using control charts to understand KPI behavior, workshops with the objective of spreading the knowledge and investigating success factors were conducted at a case company. Given the research question, an ideal context for studying this is a large company organized in several functions and having an interest in using control charts. The company chosen has these characteristics. It is a large international company with more than 1,000 employees. The production includes processes such as welding, machining, painting and assembly.

The case study can be described as a holistic multiple-case design with a single unit of analysis according to Yin’s definitions (Yin, 2009). The cases are divided according to which management team they belong to; plant, fabrication, assembly or logistics. The plant management team is one organizational level higher than the other three. The managers of fabrication, assembly and logistics are also on the plant management team. The workshops were performed over two years, as Figure 2 shows, with an action approach as described by Larsson (2006) and Coughlan and Coghlan (2002) meaning that the researcher intervened by introducing the topic.

The result of the workshops at the case company was collected by using questionnaires, interviews, archival data and observations. The participants filled in the questionnaire about their view of the current KPI follow-up before attending the workshop. The workshop content included both theoretical and practical sections, e.g. variation and control charts.

Interviews with 22 persons working at different organizational levels at the case company were conducted by two interviewers after all workshops were finalized. The interviews were mainly based on semi-structured questions according to Lantz (2007). They were recorded, transcribed by a third person, and coded before being analyzed. Ten of the persons interviewed had attended the workshops; an additional three had acquired knowledge from elsewhere.

The managers of each management team were also asked to assess to what extent each theme was fulfilled after the workshops and to rate the implementation success. The managers were also asked to state which themes they experienced as most important for implementation success.

Findings

The research findings have been divided into two areas: the themes influencing implementation and the changes of decisions.

Themes influencing change and implementation

The resulting 20 levels 1 heading from using the AIM can be considered as themes important for successful implementation; they are shown below.

Themes:

1. Improvement teamwork

2. What? The goal of change defined

3. Management in a changing environment

4. Why? Need for change defined

5. The organization’s culture enables commitment

6. Shop floor culture supporting process thinking and fact-based decision making

7. High-level commitment

8. Structured implementation

9. Competence

10. Communication of change effort

11. Systematic follow-up and improvement

12. IT and systems supporting implementation

13. Making the change last

14. Resources for improvement

15. “Show me the money” – Figures on improvement

16. Dedicated change agents

17. Customer for change defined

18. Organizational set-up

19. Personal rewards, What is in it for me?

20. Kick-off

Changes of decisions

The result from the questionnaires shows a very diverse picture. Figure 3 shows how the plant management team answered the question about where the focus of discussion lies at the KPI follow-up meetings. Each color represents the answers from one manager. Answers are present on the entire scale from past to future, reactive and proactive but with a slight emphasis on passivity.

Figure 4 presents the triggers for actions at the KPI/OPI follow-up meeting. OPI means operational performance indicator. A KPI often describes broad categories, whereas an OPI measures a specific function or operation. Each colored X represents one participant. The perception by the plant management team members is that deviation from target and gut feeling trigger actions, whereas the opinions regarding whether trends in data and divergence from customer requirements trigger actions are very diverse.

In all, 22 persons, of whom 10 had participated in the workshops, were interviewed. Control charts were at the time of the interviews only used for audit results and material availability but several persons identified possibilities to use it for, e.g. safety measures and lead time. Training and knowledge is considered crucial for the usage; 64 per cent of the answers indicated that as a success factor. Choosing an important performance indicator, having access to sufficient data quality and practical support at the implementation phase as well as breaking the Excel habit were also mentioned.

For me the control chart has been essential for us to initiate the right activities that give the correct result. In the control chart, we collect the data but also ensure and validate that our initiated activities take us to our goal. Previously we have based our discussions in the morning meetings on a symptom, an isolated event. It has then kept us busy the entire day (Manager at the logistics department).

The process can be stable, but not capable. That was a small “aha” experience, to not just react on being in the red (Plant manager).

The performance indicator “MA@L – Material Availability At Line Side” in Figure 5 is an example of how a control chart was introduced. The performance indicator shows an impressive improvement of the KPI itself over 10 months.

The manager describes it:

We changed the decision making and focus during this trip based on facts. We understood the activities that had the greatest sustainable impact and thus established ourselves on a better level. What we could observe was that the number of additional transports went down, our line stops due to the inability to deliver materials decreased and we had better balances. If you sum up the cost of line stops, balances, and additional transport and take an average of the cost during the same period, I estimate that at around 10,000 euro per week.

Analysis

In this section, the change management theory and the empirical findings from the workshops are combined and analyzed.

The 20 themes, condensed from the change management literature, were used as a basis for the analysis of the control chart implementation at the case company. The themes are sorted based on how many of the models investigated were represented as well as how many of the 141 factors were included in the theme. The value in the column “represented in (per cent) models” is the percentage, calculated by dividing the number of factors originating from different models by the total number of models (which is 17). The value in the column “factors represented” is the percentage of the total number of factors in the theme divided by 141, the total number of factors. The themes, ordered by the number of models the theme is present in, are shown in Table II.

The synthesis of themes evaluated concerning their fulfillment at each case is visualized in Figure 6. The managers for each management team assessed to what extent each theme was fulfilled after the workshops. A mean value was calculated between the managers’ score and the score assessed by the researcher with the exception of fabrication, where the manager has left the company. The result was rounded off to one of the following stages: no fulfillment (0 per cent), initiative started (25 per cent), partial fulfillment (50 per cent), progressing (75 per cent) and fulfilled (100 per cent). The total score (all values for the themes added) and the mean score for each of the multiple cases have been calculated. The managers were also asked to rate the total implementation success, describing the progress of development of taking variation into account when following up KPIs. The managers were also asked to state which themes they experienced as most important for the implementation success. These are marked by a star.

To define the goal of the change, being able to manage in a changing environment and have high-level commitment was not surprisingly considered important by the managers for a successful implementation of variation-based KPI follow-ups. That is in line with the literature. Both the interview study and the interviews with the managers reveal that training and knowledge are crucial for the usage. That is however only included in half of the change management models covered.

What the managers think is important for a successful implementation varies. Only the theme manage in a changing environment was selected by all three managers. Several themes with a small presence in the change management models were influential for the implementation success, according to the manager interviews and observations. As an example, a clear kick-off and dedicated change agents were considered important by the plant manager but were only included in 6 and 24 per cent of the models, respectively. Practical support at the implementation phase was identified as important in the interview study, which agrees with the plant manager’s view of the need of dedicated change agents.

There is a difference in theme fulfillment between the management teams, which agrees with the experienced implementation success. The logistics team has the highest score and mean of theme fulfillment and is also perceived as the most successful in implementing the new way of working. The findings from the remaining management teams follow the same pattern. The difference in fulfillment of the themes considered most important by the managers is also in accordance with the overall implementation success for the management teams.

Discussion

The implications of the research conducted can be viewed either from a theoretical or managerial perspective even though they have impact on both.

Theoretical implications

The result from the interviews and the synthesis of change management themes show a somewhat surprising result. Several themes with a small presence in the change management models were influential for the implementation success according to the interviews and observations, e.g. a clear kick-off and dedicated change agents. All management teams got the same type of training but ended up with different levels of implementation. Competence in working with control charts is identified as essential but is clearly not sufficient for implementation success.

One reason for the differences could be clearer when considering in which step of the process the organization is. Kurt Lewin’s model (1947) has three steps; unfreeze, move and freeze. The steps included in this research are the first two, unfreeze, creating a motivation for change and move, making the changes that are needed. The final stage, establishing stability when the changes are accepted and become the new norm, has not been fully covered in the research conducted. There are certainly differences regarding which themes are important in which phase. When the literature and the people interviewed do not make this distinction in steps it is natural that the answers differ depending on the choice of focus. To have high-level commitment and have the goal of change defined could be most important in the first step, creating a motivation for change. In the second step, to making the changes needed, other factors can become more important such as having a dedicated change agent, choosing an important KPI and being able to describe the gain in financial terms. That different success factors and hinders are apparent in different stages of change is also in line with the research conducted by Pinedo-Cuenca (2012). They state that e.g. commitment, involvement and communication are necessary to “unfreeze” the equilibrium and that teamwork, methods, organizational structure and culture are required in the second “move” stage.

It is interesting to see how the theme fulfillment is connected with the overall implementation success. This could mean that the individual theme fulfillment could be used to predict implementation success. The difference in importance between the themes however needs to be further studied.

Managerial implications

The choice of KPI could certainly affect the implementation success. Does the control chart offer a possibility to visualize the existing data in a better way or is it necessary to start collecting new data? In the latter case, the implementation would certainly slow down, but an improvement for the organization would have been initiated thanks to the control chart initiative. The main objective needs to be considered in this case. There is no intrinsic value in applying control charts on all KPIs; the objective is to improve business by making better decisions – which can be facilitated by using control charts wisely. By questioning the purpose of the KPI, the control chart could even initiate removal of redundant KPIs that only keep the organization busy without adding value.

The questionnaires showed that unprofitability and gut feeling previously had triggered actions at the case company. That could create a culture of blame game and attempts to avoid being in the red. By changing the follow-up to be fact-based, taking variation into account, a lot of the pressure was released. It became clear when to react and when not to. That is also something that became apparent in the interview study; the respondents said it felt better. According to one management team member, it made it fun to go to follow-up meetings. People involved felt relaxed when they were assured they made the right decision and did not have to run on everything to do a good job.

Many of the parts of the research can be considered “old news” for individuals knowledgeable about Six Sigma tools and principles. However, the difficulties lie in translating that individual knowledge into a common cross-functional understanding within the organization. Only then will practical implications arise. According to the study, a dedicated change agent is considered important for the success, but if there is no critical mass of people discussing these issues in the same way, implementation gets very difficult. Lewin (1947) argues that if one succeeds in changing group standards, this will facilitate a change in the individual and stabilize the individual conduct on the group level. The same reflection can be made in connection with the workshops with control charts. One person’s understanding does not make the organization change behavior. By training a team as a team, people will have a joint language and have the same view of the issue, making change possible. The way of working described can, however, be counteracted by the common way of organizing companies in separate functions without natural contact and interaction.

On the other hand, this way of working can influence the entire business, enabling cross-functional problem solving. When presenting KPIs showing variation, the focus tends to be on analyzing the information instead of finding someone to blame. By keeping in mind what information the internal customer needs, the entire manufacturing process could be affected. Increased knowledge can turn the focus on up-stream causes, creating a proactive approach. Instead of solving problems in the final product, properties affecting potential defects can be monitored. Even better, the characteristics are monitored already before the production takes place. Research enabling knowledge about what causes imperfections and how they can be avoided, rather than only detecting them afterwards, is therefore of great importance. Visualizing variation is one important facilitator in this work. The participants in the workshops expected an improved follow-up system but got a welcome side effect. The control charts made the organization more proactive.

Conclusions

The research conducted advances our understanding about critical factors for a successful implementation. The factors identified in the literature do not comply entirely with the factors identified during the empirical studies. The empirical and theoretical studies concur when it comes to the factors high-level commitment and having the goal of change defined. To have a clear kick-off, a dedicated change agent, choosing an important KPI, and being able to describe the gain in financial terms is less common in the change management models but is identified as important in the empirical studies.

The practical implications described are substantial. A successful implementation has shown to have a huge impact on the development of the KPI itself and has thereby a great financial impact. The research is useful for managers to understand what factors to consider (and what factors that are redundant) to achieve a successful implementation.

The most important findings can be summarized as follows:

• The important factors for implementation success differ to some extent between the theoretical and empirical studies.

• High-level commitment and having the goal of change defined could be most important in the first step when creating a motivation for change.

• In the second step, when the change is performed, having a dedicated change agent, choosing an important KPI and being able to describe the gain in financial terms become more important.

The research conducted focused on large companies in the field of welding. Future research could build on this knowledge and investigate if the result is context-dependent. It would also be of interest to see if another choice of included change management theory would result in different themes. Further research is also necessary to investigate the differences in importance between the themes for implementation success. Because this research identifies the current state and factors influencing the start-up of using control charts, a complementary longitudinal study would be of interest to investigate factors influencing the persistence of control chart usage.

Figures

Figure 1

Illustration of a control chart. The chart reveals a pattern of instability in a process

Figure 2

Workshops conducted

Figure 3

Questionnaire answers concerning the focus of discussion for KPI follow-up

Figure 4

Questionnaire answers concerning triggers for actions at the KPI follow-ups

Figure 5

Example of a control chart implemented

Figure 6

Experienced theme fulfillment for each management team participating in the workshop

Table I

Change management models included in the study

Author	Year	Main theme	Type (empirical cases/framework based on other models)
Kotter	1995	8 steps	Empirical
Jick	1991	10 steps	Empirical
Garvin	2000	7 steps	Empirical
Mento et al.	2002	12 steps	Framework
Deleryd et al.	1999	6 critical factors	Empirical
Does et al.	1997	Reasons for failing	Empirical
Sannö	2015	9 key factors for change	Framework
Isaksson	2006	Change management model	Framework
Isaksson	2006	Resources	Framework
Hallencreutz	2012	Organizational change fundamentals	Framework
JØrgensen et al.	2009	Major change challenges	Empirical
Oakland and Tanner	2007	Framework for change	Framework
Oakland and Tanner	2007	Common enablers	Framework
Vandermerwe and Vandermerwe	1991	4 steps and 4 catalysts	Empirical
Mann and Kehoe	1995	7 prime factors affecting the implementation of TQM	Empirical
Dale and Shaw	1991	The typical profile of an organization whose people are raising the type of queries	Empirical
Kotter and Cohen	2003	Steps to empowerment	Empirical

Table II

Themes influencing change ordered by occurrence in change management models

Theme	Represented in % models	Factors represented (%)
What? The goal of change is defined	65	9
Management in a changing environment	59	9
High-level commitment	53	6
Improvement teamwork	47	10
Why? Need for change defined	47	8
The organization’s culture enables commitment	47	7
Shop floor culture supporting process thinking and fact-based decision making	41	7
Structured implementation	41	6
Competence	41	6
Communication of change effort	41	5
Systematic follow-up and improvement	35	5
IT and systems supporting implementation	29	4
Resources for improvement	29	4
Making the change last	24	4
“Show me the money”–Figures on improvement	24	3
Dedicated change agents	24	3
Customer for change defined	18	2
Organizational set-up	12	2
Personal rewards, What is in it for me?	12	1
Kick-off	6	1