A multicriteria model for assessing item importance and risk using operational data from military supply chains

Abstract

Purpose

This paper is motivated by the need to assess the risk profiles associated with the substantial number of items within military supply chains. The scale of supply chain management processes creates difficulties in both the complexity of the analysis and in performing risk assessments that are based on the manual (human analyst) assessment methods. Thus, analysts require methods that can be automated and that can incorporate on-going operational data on a regular basis.

Design/methodology/approach

The approach taken to address the identification of supply chain risk within an operational setting is based on aspects of multiobjective decision analysis (MODA). The approach constructs a risk and importance index for supply chain elements based on operational data. These indices are commensurate in value, leading to interpretable measures for decision-making.

Findings

Risk and importance indices were developed for the analysis of items within an example supply chain. Using the data on items, individual MODA models were formed and demonstrated using a prototype tool.

Originality/value

To better prepare risk mitigation strategies, analysts require the ability to identify potential sources of risk, especially in times of disruption such as natural disasters.

Citation

Rossetti, M., Bright, J., Freeman, A., Lee, A. and Parrish, A. (2024), "A multicriteria model for assessing item importance and risk using operational data from military supply chains", Journal of Defense Analytics and Logistics, Vol. 8 No. 1, pp. 56-79. https://doi.org/10.1108/JDAL-07-2023-0007

1. Introduction

The Defense Logistics Agency (DLA) uses over 6,000 different vendors as sources of supply for a variety of commodities, ranging from fresh fruit to parts for aircraft engines. During periods of supply chain disruption (e.g. due to a natural disaster), analysts must filter the list of potentially affected vendors to identify specific risks to logistics operations and then assess those risks for military service effects. The DLA manages millions of items, each identified by its national item identification number (NIIN). While vendors may have different risk profiles, ultimately it is the items that they supply that are critical to the military services. Thus, an understanding of how risk is associated with items supplied is a key concern. This paper describes an approach to address the need for measuring the risk and importance associated with items (NIINs) stored within a military supply chain.

Using a multiobjective decision analysis (MODA) methodology, risk and importance indices were developed for the analysis of items. In the proposed approach, the modeler must identify operational elements (e.g. items) and related operational metrics. These metrics are then classified as contributing to either risk or importance within the supply chain. Then, the analyst develops value functions and weights for each attribute to translate the raw operational data into risk and importance indices that have a common interpretable scale. The proposed methodology could also be used within commercial supply chains, subject to the identification of operationally relevant data sources that are useful within those settings.

The paper is organized into sections, providing a brief literature review, an overview of the underlying methodology and risk assessment metrics and an illustrative application of the methodology to item risk. We provide some conclusions about future areas of investigation in the final section.

2. Literature review

This section presents an overview of the literature that informed methodological decisions. The purpose of the review is to identify problem elements, metrics and models that were useful in meeting the research goals and methods. A key requirement for a potential methodology is that the required data elements should be readily available without significant effort or that the required model inputs can be readily created via informed domain experts or analysts.

Christopher and Peck (2004) notes, “as supply chains become more complex as a result of global sourcing, …, supply chain risk increases”. The paper concludes that a key challenge is to “manage and mitigate that risk through creating more resilient supply chains”. Henry and Ramirez-Marquez (2012), defines resilience as a function of time using a quantitative framework based on a disruptive event, followed by system disruption, a disrupted state, system recovery and finally moving to a stable recovery state. Military acquisition planners need to account for system behavior over time under conditions of disruption when making sourcing decision (e.g. supply selection, back up suppliers, etc.) and inventory purchasing over time. What has become clear is that the idea of treating the supply chain as a complex adaptive system, see the seminal paper by Choi et al. (2001), is even more applicable today than it was over 20 years ago.

The study of supply chain risk has been a central area of research and practice for decades. A seminal paper by Sheffi (2001) lays the groundwork for identifying some of the key issues to consider when trying to mitigate risk within the operation of a supply chain. Specifically, the key idea of the paper is that while a first-order event (i.e. a terrorist attack) might have initiated the disruption, it is often the secondary response to the event that needs to be considered in the planning. For example, in both the response to the September 11, 2001 event and the response to the pandemic, the direct drivers of major disruptions to supply chains were not the event itself but rather governmental responses to the event.

Building on these concepts, Rice and Sheffi (2005) introduce the concept of resilience within supply chain management. In this view, supply chain management should consider not only reducing the vulnerability to disruptions but also strategically planning for the reduction of the time that it takes to return to normal operations. Designing for resiliency recognizes that the likelihood of a disruption cannot be totally eliminated and thus, designing the supply chain for recovery should be a key overall strategic goal. To be able to incorporate risk into the supply chain design process, we need to be able to measure risk in a quantifiable manner.

Thus, a focus within the early 2000s within the supply chain literature concerned how to measure risk, what factors to include during the risk assessment process and how to formulate mathematical models as part of the risk assessment process. Gaonkar and Viswanadham (2007) present a framework for assessing the risk within supply chains. It describes the risk management problem and the need to address the problem at the three major levels of the enterprise: strategic, operational and tactical. Gaonkar and Viswanadham (2007) identify sources, consequences, drivers and mitigation strategies as the basic constructs within supply chain risk management. It classifies supply chain risk problems into three main areas: deviation (change from the expected), disruption (radical transformation leading to nonavailability) and disaster (temporary irrecoverable shutdown of the supply chain). Finally, Gaonkar and Viswanadham (2007) propose an integer quadratic programming model for supply chain partner selection. The objective of the model was “to choose suppliers such that the expected shortfall in supply, in the face of supplier disruptions, is minimized.” Thus, it should be clear that understanding risks within a supply chain requires considering how vendors and the items they produce contribute to the risk profile. The approach proposed in this paper focuses on quantifying risk based on operationally extracted data on items and vendors.

While it is beyond the scope of this article to provide a comprehensive review of the supply chain risk management literature, it is beneficial to note that considerable effort has been taken to understand the drivers and responses to risk within the literature. For a more substantial review of the literature, the interested reader is encouraged to review the following survey papers: Ho et al. (2015) present a comprehensive literature review of research from 2003 to 2013 within the area of supply chain risk management. During this time frame, the primary areas of investigation included defining risk, classifying types of risk and identifying supply chain risk factors. Some of these risk factors are essential to include in the proposed risk model of this paper. Heckmann et al. (2015) provide a comprehensive review of the literature and subsequent quantitative methods applied within the area of supply chain risk management. It finds that supply chain risk sources (network, process, etc.), exposure and characteristics are vital perspectives when considering supply chain vulnerability. The complexity of some of these papers limits their application in practice. Fan and Stevenson (2018) provide a review of over 354 articles from the 2000–2016 time frame and find that the bulk of the research concentrates on identifying risk types and formulating mitigation strategies. It identifies ten potential research gaps within the literature, which overall suggest the need for holistic models, better monitoring, benchmarking, cost-benefit analysis and a better understanding of how position within the supply chain affects risk. One significant item to note is that the supplier’s position within the supply chain is called out as needing additional research.

In the following, we focus on papers that contributed key insights supporting our modeling approach. In particular, Loredo et al. (2015) motivated a solution approach involving the use of operational data. Dong (2006) develops the concept of “robustness” to describe the ability of a supply chain network to “carry out its functions despite some damage done to it.” Dong (2006) presents a system-wide approach to quantifying the robustness index of supply chain networks. The approach considers both network structural robustness and network functional robustness when a supply chain is faced with disruption, disaster, contingency and terrorism. Dong (2006) measures the efficiency of the network, where efficiency “attempts to measure the effectiveness of the network configuration to accomplish its function.” The model represents the network as a graph of links and nodes, where nodes represent locations in the network (e.g. distribution, production, assembly and storage) and links represent some interaction between the nodes. The approach uses the connectedness of the graph to construct various measures. In general, the methodology represents a static representation at a particular point in time. This representation permits an analysis of weak links within the network and their potential effects. At the time of the writing of the paper, the methodology had not been tested within a realistic case study. A possible shortcoming of the work is the amount of structural data that is required and how to represent the data within a computer. This shortcoming makes the approach difficult to apply to a network as complex as DLA’s supply chain, with all its connections and the varying nature of the connections over time. However, the idea of taking a system-wide approach to building a robustness index contributed to our methodology.

Kinra et al. (2020) describe a methodology to assess high-impact, low-frequency events within a supply chain. A high-impact, low frequency event is an event that is so rare that the determination of its likelihood of occurrence is effectively guesswork. Kinra et al. (2020) ignore the problem of estimating the chance of the event and instead focus on determining the vulnerability of the supply chain to unknown events. The methodology provides a heuristic for risk based on the worst-case scenario analysis, with the time of disruption as an input parameter. The methodology requires estimates of disruption time and determines the maximum loss to operations due to the disruption. The computation of loss is based on every disrupted node in the supply chain and the total loss determined across the supply chain. The heuristic shows the parts of the supply chain where it may be possible to have large losses. The goal is to identify these weak links to focus efforts to mitigate the losses. The methodology is illustrated on a small, three-level supply chain involving suppliers, a warehouse and a customer. Possible shortcomings of the work include that the disruption’s impact is measured at a particular instant and the lack of modeling for partial recoveries or temporary supplies. The methodology also suffers from similar issues as Dong (2006) in that the data to represent the network may be difficult to obtain. In addition, the methodology assumes a simplistic accounting of disruption costs; however, due to its simplicity, it may more readily lend itself to what-if analysis.

Ziegenbein and Baumgart (2006) present a quantitative approach to measuring the probability of occurrence and the financial impact of disruptions in a supply chain. The proposed methodology allows managers to gather information about the magnitude and risk of disruptions to a supply chain. The goal is to assist businesses in estimating the value/cost associated with the disruption and quantifying its impact. The methodology uses a probabilistic analysis rather than a subjective response. The main inputs to the methodology are the number of suppliers and the number of sourced items demanded per supplier. The main outputs include the business interruption value and its impact on the expected gross margin. The authors analyze a number of scenarios involving problems such as forecast error (minor), product quality (medium) and fire/flooding (severe). The shortcomings of the work include the number of assumptions that must be made to model business operations, such as ordering lot-for-lot and inventory operating policies. The focus on business interruption value makes it more difficult to translate the model to a military supply chain context.

Gaudenzi and Borghesi (2006) describe an application of the analytical hierarchical process (AHP) in identifying and managing risk within a supply chain. The process evaluates daily risks to supply chain integrity, such as on-time delivery, completed orders, correct orders, damaged products, etc. These and other metrics are used to measure and evaluate suppliers. The paper proposes a tool for managers to assess a supplier based on past performance and available data using the AHP methodology. The analysis starts by identifying specific risks (damaged products) and assigning causes and priorities to each risk. Then, the causes are examined to identify the origin of the risk. The approach produces a set of metrics and options for managing the risks. Since the approach is based on AHP, there is a significant interaction required to provide input for the model, and the modeling approach is primarily subjective in nature. Also, the analysis requires significant work if there are a large set of vendors. An advantage of the methodology is that it results in a risk (index) associated with each supplier.

Fritsche (2016) presents an Excel-based tool that facilitates the assessment of risks within a supply chain. The approach identifies a set of areas that may contribute to risk and allows managers and domain experts to subjectively grade each area based on best practices used within those areas to control, mitigate or contribute to risk. The tool includes five risk areas: managerial, sourcing, warehousing/transportation, inventory/production and information systems. Questions in each of the five areas are answered during the assessment process. Each area is weighted as to its contribution to risk. The scores for each area’s questions are tabulated and can be compared to those of other organizations. The approach allows for stakeholders to evaluate their supply chain’s risk landscape with minimal quantitative requirements in a relatively short time span. The approach facilitates self-awareness of the practices and issues relative to how operations affect supply chain risk and may help organizations identify areas for improvement. The shortcoming of the work is that it is entirely subjective; however, the approach is simple, facilitates what-if analysis and can be quickly performed because it does not require significant data collection effort.

Moore and Loredo (2013) describe the results of an analysis for the U.S. Air Force to identify supply chain risks. The main result is a comprehensive list of the risks that could affect the Air Force’s supply chain. The report does not detail a model, but instead focuses on listing risks and their supporting factors. The main risks identified are due to natural disasters, acts of war, terrorism/sabotage and accidents. In addition, some environmental operating risks are also identified, such as business environment (taxes, customs, currency devaluation, lawsuits, economic recessions, labor, strikes, etc.), market environment and technological uncertainty. One highlighted area for supply chains is the flow of raw materials, specifically the fact that China is the main producer of aluminum, which is a key raw material in aircraft production. Furthermore, the report summarizes a review of weapon systems and case studies to compare how risk was managed in the past. The findings of the report are useful for the identification of sources of risk and providing guidance on what to consider during a risk assessment.

Loredo et al. (2015) present a methodology for measuring risk within Army supply chains for items and vendors. The approach uses the existing operational data to form risk indices. It defines a risk index as “a heuristic score for (1) the likelihood that vendor could fail to supply the item and (2) a heuristic score for measuring the consequences on Army weapon systems” (Loredo et al., 2015, p. 9). The model includes various operational factors that may influence risk. The risk indices measure overall risk for a vendor by summing all risk values for items that vendors supply. The study was limited by the availability of data (NIIN and vendor characteristics) within the existing operational systems. The work led to an automated tool that can be used on a periodic basis to query the existing data, execute the risk index computations and summarize the analysis of risk based on factors of interest.

Loredo et al. (2015) develops risk measures based on factors within the supply chain. The main areas include demand fluctuation, funding uncertainty and long-lead times. It defines NIIN risk as

NIIN Risk=Vendor Failure Risk*System Input Risk/(1+DaysToRunOut/365)

NIIN risk measures individual items within the overall analysis and is used to “create a weight for a NIIN on the system.” NIIN risk is a component of computing an overall system (supply chain) risk. The report summarizes the various data sources used to compute the components of NIIN risk. The risk scores are standardized between 0 and 1. Vendor failure risk does not represent a probability but rather another heuristic score that is constructed from additional factors related to vendors.

VendorFailRisk=(StatusRisk+RevenuePercentFall+RevenuePercentFallToAllVendors+RevenuePercentSAM+IsContractExpired+Floodrisk+TornadoRisk+HurricaneRisk+QuakeRisk+ForeignRisk)/9.

As can be noted from the VendorFailRisk equation, the conceptualization of vendor risk is functionally related to status, revenue, contract characteristics and risks associated with the location of the vendor as related to natural disasters. Notice that each of the factors has an equal weight (9 factors, divided by 9). The definition of the individual components is provided in Loredo et al. (2015).

In summary, the reviewed literature involves three major approaches: (1) quantitative, (2) qualitative and (3) blended quantitative/qualitative methods. The quantitative methods involve the application of supply chain analysis methods, sometimes utilizing mathematical programming or graph theoretic constructs. The qualitative methods focus on identifying the management process and issues that are salient in understanding supply chain risk. The blended methods tend to utilize qualitative methods for structuring the elements and quantitative methods for developing indices that attempt to measure supply chain risk. Based on the review, we concluded that an approach that blends quantitative analysis with the ability to include subjective (qualitative) factors could be a useful approach, especially if it could be driven by operational data. Thus, we focused on the approach in Loredo et al. (2015).

A compelling aspect of the approach in Loredo et al. (2015) is the ability to compute risk by item and vendor and “roll up” the analysis of risk across various factors. However, there are possibly very serious issues with the mathematical basis for the approach taken to develop the individual risk indices. The first issue is that the values of the individual components are not standardized on a common scale. Secondly, the metrics assume that the individual factors contribute equally to the overall risk index. Because of these two issues, we utilize the theory of MODA to avoid the issues of equal weighting and a lack of common domains and scales. The next section presents the application of MODA to the development of risk indices.

3. Overview of methodology and metrics

This section provides an overview of the basic components and steps within the MODA methodology applied to the risk index development process. The standard MODA methodology is laid out in Parnell et al. (2013). The key step of this process that is applicable to this research is crafting the decision objectives and value measures (or value function hierarchy). This process provides stakeholders with a common framework for identifying and quantifying the most relevant factors in the decision.

An initial step in the MODA process is to identify attributes and risk/importance measures. Within the MODA methodology, a value function hierarchy (VFH) is the major tool to structure multiple characteristics into independent and nonoverlapping groups of criteria. The VFH represents a value tree that encapsulates the attributes that are important to the decision context. Figure 1 illustrates a notional value hierarchy tree for assessing item risk. This tree has only one level, but in general there can be multiple levels, each contributing upwards to the overall measure of the decision context.

As illustrated in Figure 1, the attributes associated with item characteristics can serve as the traditional measures for the application of the MODA methodology. The values of the attributes (as observed from operational data) serve as the raw scores that will be normalized to a common measurement scale by the specification value functions. The value functions represent a mapping from raw scores to a scale (0–100) that represents the attribute’s contribution to the value represented by the tree. In this research, we develop two value function hierarchies: one for item risk and another for item importance. Each hierarchy will result in an overall index, one for item risk and one for item importance. Once the value function hierarchies for item risk and item importance are developed, the value functions for the attributes of each hierarchy are specified.

Each value function has an x-axis and a y-axis, where the x-axis is the scale of the raw score (e.g. production lead time) and the y-axis is a unit-less value measure on the scale from 0 to 100. Continuous value functions typically follow four basic shapes: linear, concave, convex and S-curve, as illustrated in Figure 2. Depending on the impact of each value measure, value functions could be either monotonically increasing or decreasing, as indicated in Figure 2. As suggested in Kirkwood (1997), the shape of value functions is determined by consulting with subject matter experts. Once the general shape is determined, the experts identify the increase/decrease in value from a specific incremental increase in the measure scale. Repeating this multiple times up to the maximum on the measure scale produces a piecewise linear function. The functions illustrated in Figure 2 were produced in a linear piecewise fashion.

Each attribute has raw score, xi based on its natural scale, which is standardized to a common scale by applying a value function, v(xi,), where

• xi is the raw score of measure i on the x-axis of the value function and

• v(xi) is the value of measure i on the y-axis of the value function.

Attribute values are combined into an overall value using weights and an additive model. In this research, the combined overall value is called an index. A higher overall value indicates more risk (or importance) for the item for the given attributes, weights, value functions and raw scores.

The weights depend on both the criticality of the attribute and the impact of varying the score of attributes. A swing weight matrix is one of the well-known methods to determine the weights. This method assesses measure weights by “swinging” the attribute score from its worst to its best. Parnell and Trainor (2009) discuss this method in detail with examples. There are various ways, besides the swing weight matrix, to elicit weights from stakeholders (Clemen and Reilly, 2014; Kirkwood, 1997). After the value functions have been specified, the attribute criteria can be ranked and the results recorded within a swing weight matrix so that an overall index can be formed.

To summarize, using the MODA methodology as outlined here enables the evaluation of the risk based on multiple characteristics extracted from operational data sources. The process starts with specifying a value function hierarchy (i.e. selecting the component (item) measures). After the VFH has been specified, data are collected and normalized, and then mapped to indices (for risk and importance). Finally, an analysis can be completed based on the results. In this step, the scores for the supply chain component (item) can be analyzed. The purpose of this step is to guide further work by identifying assumptions, criteria, parameters and factors that are significant to the overall analysis of risk. The risk or importance index value is a single numerical score, and the output of an additive value function model is based on the value functions and the swing weights applied to each supply chain component (item). After performing an analysis of the risk and importance indices for various supply chain grouping criteria, a sensitivity analysis of the weights and scores can be performed.

We can gain valuable insights by performing a sensitivity analysis on the elements of the MODA model and observing how the results change. Weights and value functions are the elements that can be subjected to sensitivity analysis to gain further insights on the riskiest entities and their influence on overall supply chain risk. The application of the MODA methodology to the building of risk indices mitigates one of the problems associated with the approach suggested within Loredo et al. (2015) by providing a rational and well-established basis for combining measures that have different scales and values within the context of measuring supply chain risk. In the following section, we provide guidance for analysts to select and interpret value functions.

3.1 Value functions

The purpose of a value function is to translate the raw score of an attribute into terms that can be compared across attributes. A value function does this by converting the raw score of the attribute into its value. To a decision-maker, value is a concept that encapsulates the inherent worth or utility of the attribute to the overall determination of value that encompasses all (multiobjective) attributes used in the problem context. In this research, the raw score of each attribute is translated to a risk or importance score between 0 and 100. An overall risk value represents the “common value” translation across the attributes included in the risk measure. Similarly, an overall importance value represents the “common value” translation across the attributes included in the importance measure. This translation from raw score to value can be complicated because the relationship between the raw score of an attribute and its value (its risk or importance) is not necessarily linear or even increasing.

For example, consider two items, A and B. Assume that both items are similar in unit cost, criticality, lead time, etc. Their only difference lies in annual demand. The demand for item A averages about 100 units per year; the demand for item B averages about 300 units per year – three times higher. Most people agree that item B is more important than item A. The question is, how much more important is item B? Does having three times more demand make B three times as important, 50% more important or nine times more important than A? Does the answer change if the demand for A and B are one and three items per year or one million and three millions?

Each variable in the risk and importance indices must be assigned a value function. Four shapes and five functional forms are implemented in the model, for a total of 20 possibilities. Figure 2 displays the available value functions. The gallery is organized into rows for each direction/shape and columns for each functional form.

The first two directions are:

• 1. Increasing: This shape is used when it relates to a higher value. For example, in the case of risk, with an increasing direction, a higher raw score of the attribute implies a higher risk. This is the most common direction for a value function. How much the value goes up depends on the functional form of the value function.

• 2. Decreasing: This shape is used when less relates to a higher value. For example, in the case of risk, with a decreasing direction, a lower raw score of the attribute implies a higher risk. Risk variables often have decreasing value functions. How much the value goes down depends on the functional form.

The last two directions/shapes are compounds of increasing and decreasing:

• 3. Peak: This shape is used when the maximum value occurs at the median raw score. In a peak shape, for low raw scores, an increasing raw score of the attribute relates to increasing value (risk and importance). After reaching its maximum, a further increase in the raw score of the attribute causes a decreasing value (risk and importance).

• 4. Valley: This shape is used when extreme raw scores lead to high values. In a valley shape, for low raw scores, value decreases to a minimum point, after which value increases as the raw score of the attribute increases. This might be useful when having the exact right quantity of a variable minimizes risk and raw scores further from this quantity increase risk.

The direction of a value function depends on how the risk/importance changes in response to an increase in raw score. Direction does not determine how much the value changes when the raw score increases. Direction just indicates the direction the value moves.

The prototype tool provides various shapes via five functional forms for a value function, which are:

1. Linear: Value increases at a constant rate relative to the raw score of the attribute.

2. Logarithmic: Value increases at a decreasing rate relative to the raw score of the attribute.

3. Exponential: Value increases at an increasing rate relative to the raw score of the attribute.

4. Sigmoid: At first, value increases at an increasing rate relative to the raw score of the attribute. After the midpoint, value increases at a decreasing rate relative to the raw score of the attribute.

5. Inverse sigmoid (Logit): At first, value increases more slowly than the raw score of the attribute. After the midpoint, value increases more quickly than the raw score of the attribute.

In the approach that follows, we develop separate indices to measure (1) importance within the supply chain and (2) risks within the supply chain. Importance provides some measure of whether the element should be prioritized within supply chain operations, but that importance does not necessarily (alone) measure the inherent risk associated with the element, that is, we believe that it is useful to analyze supply chain elements based on two criteria: (1) their operational importance and (2) their operational risk. An element may be important but pose little risk. Our interest should be in finding elements that are both important to supply chain operations and pose high risk to supply chain operations. Thus, within the following sections, for each item, we include indices for both importance and risk, which are then combined into an overall item index.

The next two sections illustrate the application of the MODA methodology to the development of item importance and risk indices. After that presentation, we provide an illustrative example of the application and interpretation of the indices.

3.2 Modeling item importance and risk

This section presents the operational fields used in developing the item indices as well as how the fields are combined into an overall index. Fields within DLA operational information systems were reviewed for relevance and availability for use within the item risk analysis. Sets of possible fields based on the reviewed literature and DLA information systems were proposed and evaluated as to their contribution to indicating three main item characteristics: (1) fields that facilitate grouping and analysis, (2) fields associated with attributes that indicate importance and (3) fields associated with attributes that indicate operational risk. DLA subject matter experts (SMEs) assisted in reviewing and collecting example data on the possible fields.

First, we present general mathematical notation for the risk and importance indices for items. Then, the selected fields will be discussed based on DLA data instances. There are additional fields called “grouping fields.” Grouping fields are not represented mathematically since they are not used to produce index values.

1. kr is the number of risk attributes in the item risk index.

2. nr(i) is the name of the field associated with attribute i. These fields are shown in Table 3.

3. vr(i,j) is the value of risk attribute i for item j

4. wr(i,j) is the weight of risk attribute i for item j

5. r(i,j)=vr(i,j)*wr(i,j) is the risk contribution for attribute i for item j

6. R(j) is the risk index for item j, where:

R(j)=∑i=1krr(i,j)

1. rp(i,j)=r(i,j)/R(j) is the proportion of the total risk contributed by attribute i for item j

The value of R(j) for a given item j is the key output from the MODA risk computations. The analyst has the ability to choose which attributes are included in the risk computation as well as the weight associated with each attribute.

As noted in the previous section, we develop a separate index to represent the importance of the factor to the supply chain. Thus, an item that has high risk and importance would be identified for further investigation and control. The mathematical notation for the importance index is as follows:

1. km is the number of importance attributes in the item importance index.

2. nm(i) is the name of the field associated with attribute i

3. vm(i,j) is the value of importance attribute i for item j

4. wm(i,j) is the weight of importance attribute i for item j

5. m(i,j)=vm(i,j)*wm(i,j) is the importance contribution for attribute i for item j

6. M(j) is the importance index for item j, where:

M(j)=∑i=1kmm(i,j)

1. mp(i,j)=m(i,j)/M(j) is the proportion of the total importance contributed by attribute i for item j

M(j) for item j is the key output from the MODA importance computations. The analyst chooses which attributes are included in the importance computation as well as the weight associated with each attribute.

Tables 1–3 list the item fields used in the analysis, their description and their data type. The following describes the fields and how they are used in the analysis for the prototype model: Table 1 presents the item fields selected for grouping. These fields describe an item and its usage within DLA. Grouping fields allow for aggregation and analysis. Table 2 presents the fields related to importance. The data type column indicates the underlying scale associated with the field. Numeric fields can be easily mapped to value-function domains. On the other hand, fields that are ordinal need to be mapped to numeric values that imply an ordering across the set of possible values. This is discussed further when illustrating the computations and analysis in Section 4. Table 3 provides the fields related to risk within the supply chain.

Tables 4 and 5 provide details of the value functions for each of the attributes. The shape column shows the type of value function applied to the field based on Figure 2. The direction column indicates the direction associated with the selected value function. The detailed selections for weight, shape and direction presented in Tables 4 and 5 are implemented in the prototype analysis tool demonstrated in Section 4.

As shown in Tables 4 and 5, the analyst has the option to weight any available field as contributing zero value to the index. Thus, fields can be easily included or excluded within the analysis. The following section illustrates the application of the tool to items within a sample dataset.

4. Illustrative application

In this section, we apply the MODA model to example item data for the DLA supply chain. The purpose of this section is illustrative in nature that is, the results are based on an assumed item risk model for the purpose of illustrating the construction of the model and its internal computations.

First, we provide some illustrative implementation details related to data integrity. This illustration should inform potential users of the methodology on the significance of effective data capture in the model. In addition, these details illustrate some of the potential effects of inaccurate and incomplete data on the interpretation of the results. For fields used in the prototype tool, we filtered the raw numeric data for missing, negative, zero, positive or other values. This allows the user to assess the quality of the raw score data supporting the item indices before transformation by the value functions. A straightforward statistical summary of the raw numeric fields is useful. This information supports a better understanding of the distribution, mean and variance of the raw scores. For categorical data a frequency analysis is useful. This analysis can highlight invalid or missing data values. Based on this input analysis, the analyst can decide to exclude or subset the data to separately analyze the results based on the factors identified during the data quality analysis.

Next, after assessing the quality of the raw data, computing the risk and importance indices can proceed.

Finally, we complete a MODA post-processing analysis to understand the risk and importance indices within the context of their item populations. We examine the distribution of the risk and importance indices across the sample DLA item population. For vendors, we would examine the risk and importance indices across the DLA vendor population.

4.1 Data integrity analysis

A risk or importance index consists of many components, some of which may be missing in the raw item data. We discuss a few options for this situation. First, in the protype tool, default values are specified in the MODA input template. These can be used as substitute for missing data so that analysis can proceed for the item. Second, the analyst can filter out items that have missing components.

Table 6 presents information for the fields Imp_Miss and Risk_miss within the dataset. Thus Imp_Miss (Risk_Miss) indicates the number of missing components in the importance (risk) index computation for an item. 1,322 of the 3,224 item records had all (0 missing) required components in the importance index and 1,118 of the 3,224 item records had all (0 missing) required components for the risk index. This represents approximately 41 and 34% of the records, respectively. This analysis clarifies the quality of the data used to compute the indices. The SAS tool permits default values for missing component values. In addition, the output indicates whether the field is missing. Thus, the analysis can be limited to items that meet a particular frequency or number of missing components. For example, the MODA analysis illustrated in Section 4.2 is for items that have zero missing components.

4.2 Index computation and analysis

Using the protype tool, M(j), R(j) and M(j)*R(j) are computed for every item. Table 7 illustrates the first ten rows of this information. The precise item information (NIIN and description) has been removed to obscure the actual item identity.

After computing the indices, the analyst should have an interest in the following questions:

1. What are the largest, K, items in terms of risk index, R(j)?

2. What are the smallest, K, items in terms of risk index, R(j)?

3. What are the largest, K, items in terms of importance index, M(j)?

4. What are the smallest, K, items in terms of importance index, M(j)?

5. What are the largest, K, items in terms of overall combined index, M(j)*R(j)?

6. What are the smallest, K, items in terms of overall combined index, M(j)*R(j)?

Given the information shown in Table 7, the analyst can rank the indices according to high or low values. Table 8 shows the items that have the ten highest risk index values. As noted from the table, item (2B) has the highest risk index value of 58.85; however, this item ranked 263rd in importance out of the 3,224 items. Thus, an item that has high risk can be interpreted within the context of its importance within the supply chain.

Table 9 shows the items with the ten highest importance index values. As can be noted from the table, item 4C has the highest computed importance index value of 50.19; however, in terms of risk, this item ranked 147. Thus, an item that has high importance should be interpreted within the context of its risk within the supply chain. This is the motivation for the creation of the combined index.

While there are different methods for combining the indices, we simply multiplied the indices together. Because the range for the indices is 0–100, the multiplication will result in a possible range from 0 to 1,000. Thus, we decided to simply take the square root to ensure a value within the range of 0–100. Table 10 presents the items with the ten highest combined index, M(j)*R(j), value. Note that item 9B has the highest combined index, with a combined index of 54.06 and individual importance, 50.00, and risk of 58.45. From those indices ranks, 4th in risk and 25.5 in importance, we can see that this item is associated with both high risk and importance.

Figure 3 shows a scatterplot heatmap of item importance indices vs item risk indices. In Figure 3, it is useful to focus on items that are in the upper right-hand quadrant: those having high risk and high importance. Similarly, one can examine items with low risk and low importance.

Beyond finding these most and least risky items, the MODA methodology supports summary statistical analysis over the entire population of items. Similarly, there is often a need to focus on other critical subsets of the items, which can utilize the included grouping variables. These are illustrated with metrics and analytical exploration as follows:

1. Compute summary statistics of R(j) and M(j) across all items

   • Min, Q1, median, Q3, max, average, standard deviation, count

2. Frequency tabulation of R(j),M(j),R(j)*M(j) into quarters over [0, 100]

   • Divide the range [0, 100] as follows: 0, (0,25], (25,50], (50,75], (75,0.100)v and 100

3. Box plots and histograms of R(j) and M(j) across all items

In what follows, we illustrate possible analytical exploration based on the population of DLA items and their associated indices. Figure 4 provides summary statistics for the distributions of risk, importance and combined indices. Notice that the average value for the indices is typically in the 30s with the maximum in the 50s. Figures 5 and 6 illustrate the distribution of the combined index with a boxplot (Figure 5) and a histogram (Figure 6). From such figures, the analyst can better understand how risk and importance indices vary across the item population.

Figure 7 presents a frequency tabulation of the indices based on dividing the range of possible values [0,100]. We can see that no indices are in the range from 75 to 100 and that most of the items have indices within the range (25, 50]. This result is illustrative; reflecting the sample of DLA items in the computations and thus, may not be typical of DLA or other items.

Besides the analysis across the entire population of items, the grouping fields can be used to explore the indices within subcategories. For example, we can summarize the statistical properties of the item indices based on the service category.

Figure 8 presents the statistical summary of the risk, importance and combined index for each of the six categories of service. An analyst can review these results and systematically assess how risk varies across the service categories. Formal tests could be used to statistically compare the estimated means across groups. Furthermore, the analyst can investigate why the risk might vary by service and develop possible mitigation strategies. Figure 9 provides a box plot summary of the risk index by service. From the plot, it is evident that service category “D” appears to have more variability and that service category “F” seems to have many outliers at the lower end of the risk scale.

As an additional illustrative example, Figures 10 and 11 show the summary statistics for the indices by WSGC and the box plot of the combined index by WSGC. These results do not seem to indicate that WSGC is an important factor in determining risk. Although, based on Figure 11, there does appear to be more variability across the indices, especially for WSGC code A.

5. Summary and future work

Using the MODA methodology, a supply chain analyst can identify operational elements (e.g. items, vendors, etc.) and their associated operational metrics and classify the metrics as attributes contributing to either risk or importance within the supply chain. Then, the analyst can supply inputs such as value functions and categorical rankings of attributes to translate the raw operational data into risk and importance indices that have a common interpretable scale (i.e. between 0 and 100). From the computed MODA indices, the analyst can identify those elements that contribute the most and the least to operational risk. The MODA index approach generalizes readily to other supply chain elements, such as vendors. The element indices can be made available to decision-makers using operational dashboards. In addition, while the model was illustrated for DLA data, the application of the methodology is general and could be readily applied to other military services’ supply chains.

Because the indices are computed from operational data, a future extension of this research should investigate how often the data used within the analysis should be refreshed. Some of the data elements used to define risk and importance indices may not vary significantly with respect to time, while other data elements may change on a regular basis. In addition, the population of potential elements may change over time. For example, some items are phased out. From a methodological standpoint, there is no distinction made between elements that change regularly based on operational conditions and those that do not. This does not limit the methodology, but it does indicate that the effectiveness of the results should be monitored over time. The tracking of a history of indices over time may be of potential benefit. This would be a natural extension to the dashboard concept.

One additional possibility for future extension of the proposed methodology is examining how to combine the indices into an overall index. First, the methodology does address this issue in a limited way by suggesting a combined risk and importance index; however, the methodology suggested here could be improved in a number of ways. For example, a weighted overall index could be constructed. In addition to combining risk and importance, since the methodology can provide an index for any supply chain component, a natural issue would be how to combine those indices into an overall supply chain index. The MODA approach to developing multilevel value hierarchies could be useful in exploring this extension. This would be consistent with the overall approach and the MODA theory proposed in this paper.

Figures

Figure 1

Notional value hierarchy tree for item risk

Figure 2

Value function direction/shape and mathematical forms

Figure 3

Heat map scatter plot of item importance versus risk

Figure 4

Summary statistics of indices across the population of items without missing components

Figure 5

Boxplot of the combined index for the population of items without missing components

Figure 6

Histogram of the combined index for the population of items without missing components

Figure 7

Frequency tabulation of R(j),M(j),R(j)*M(j) into quarters over [0, 1]

Figure 8

Statistical summary of indices by service

Figure 9

Boxplot summary of risk index by service

Figure 10

Statistical summary of indices by weapons system group code (WSGC)

Figure 11

Boxplot summary of combined index by weapons system group code (WSGC)

Table 1

Fields related to grouping for item analysis

Field name	Description/Meaning	Data type
NIIN	National item identification number (NIIN)	NVARCHAR(9)
Year	The year that best represents the data used	NVARCHAR(4)
SERVICE	Military branch	NVARCHAR(2)
NIIN_DESC	Description of the NIIN	NVARCHAR(40)
FSC	Federal supply code	NVARCHAR(4)
SIC	Standard industrial classification	NVARCHAR(6)
UNITOFMEASURE	Measurement code for weight/units	NVARCHAR(4)
VOLUMECODE	Measurement code for volume	NVARCHAR(3)
WSDC	Weapon system designator code	NVARCHAR(3)
WSGC	Weapon systems group code	NVARCHAR(1)
WSN	Weapon system name	NVARCHAR(1)

Source(s): Table by authors

Table 2

Fields selected as attributes related to importance for item analysis

Field name	Description/Meaning	Data type
ADQ	Annual demand quantity	Numeric
ABC_CLASS	Inventory ABC class designator	Ordinal
DPAS	DPAS priority rating	Ordinal
FSI	Federal supply indicator	Binary
WSEC	Weapons system essentiality code	Ordinal
WSIC	Weapons system indicator code	Ordinal

Source(s): Table by authors

Table 3

Fields selected as attributes related to risk for item analysis

Field name	Description/Meaning	Data type
ADF	Annual demand frequency	Numeric
AAC	Acquisition advice code	Ordinal
AMSC	Acquisition method suffix codes	Ordinal
COTS_CODE	Commercial off-the shelf indicator code	Ordinal
NETWEIGHT	Net weight of item	Numeric
VOLUME	Volume of item	Numeric
HMIC	Hazardous material indicator code	Ordinal
TECHOPSREVIEWDATE	Tech ops review date	Date
TECHOPSREVIEWCODE	Tech ops review code	Ordinal
ALT	Administrative lead time (ALT)	Numeric
PLT	Production lead time (PLT)	Numeric
LT_MANU	Lead time	Numeric

Source(s): Table by authors

Table 4

Value functions for item importance attributes

Field name	Weight	Shape	Direction
ADQ	0.167	Logarithmic	Increasing
ABC_CLASS	0.250	Linear	Decreasing
DPAS	0.167	Exponential	Decreasing
FSI	0.167	Linear	Increasing
WSEC	0.250	Linear	Decreasing
WSIC	0.000	Linear	Decreasing

Source(s): Table by authors

Table 5

Default value functions for item risk attributes

Field name	Weight	Shape	Direction
ADF	0.0682	Sigmoid	Decreasing
AAC	0.159	Linear	Decreasing
AMSC	0.114	Linear	Decreasing
COTS_CODE	0.227	Linear	Decreasing
NETWEIGHT	0.114	Exponential	Increasing
VOLUME	0.068	Exponential	Increasing
HMIC	0.091	Exponential	Decreasing
TECHOPSREVIEWDATE	0.000	Linear	Decreasing
TECHOPSREVIEWCODE	0.000	Linear	Decreasing
ALT	0.045	Exponential	Increasing
PLT	0.045	Exponential	Increasing
LT_MANU	0.068	Exponential	Increasing

Source(s): Table by authors

Table 6

Frequency and proportions for the item missing components

	Frequency	Percent	Cumulative frequency	Cumulative percent
Imp_Miss
1	1,659	51.46	1,659	51.46
0	1,322	41	2,981	92.46
2	243	7.43	3,224	100
Risk_Miss
2	1,520	47.15	1,520	47.15
0	1,118	34.68	2,638	81.82
1	429	13.31	3,067	95.13
3	153	4.75	3,220	99.88
4	4	0.12	3,224	100

Source(s): Table by authors

Table 7

Computed risk and importance indices for ten NIINs

Item	Service	FSC	SIC	UOM	VOL	WSDC	WSGC	WSN	Imp	Risk	RI index
1A	D	1,680	336,413	LB	IN3	11F	A	0	8.33	30.43	15.92
2A	A	5,365	332,510	LB	IN3		A	0	33.33	41.19	37.05
3A	F	1,560	336,413	LB	IN3	11F	A	0	33.40	7.43	15.75
4A	F	1,560	336,411	LB	IN3	11F	A	0	33.37	7.82	16.16
5A	F	1,680	336,413	LB	IN3	55N	A	0	25.08	41.22	32.15
6A	N	4,820	332,911	LB	IN3	EON	B	0	25.00	57.28	37.84
7A	F	1,560	336,413	LB	IN3	42F	B	0	8.34	30.15	15.85
8A	F	1,560	336,413	LB	IN3	42F	B	0	33.33	46.20	39.24
9A	N	5,342	332,994	LB	IN3			0	8.34	45.76	19.54
10A	N	1,560	336,413	LB	IN3	EAN	A	0	33.34	41.71	37.29

Source(s): Table by authors

Table 8

Items with 10 highest risk index values

Item	Service	FSC	SIC	UOM	VOL	WSDC	WSGC	WSN	Imp	Risk	RI index	Risk rank	Imp rank	RI index rank
1B	N	1,560	336,413	LB	IN3	EEN	A	0	25.00	58.83	38.35	2	265	74
2B	N	1,560	336,413	LB	IN3	EEN	A	0	25.00	58.85	38.36	1	263	73
3B	N	5,306	332,722	LB	IN3	EEN	A	0	50.01	57.60	53.67	10	18	3
4B	N	5,306	332,722	LB	IN3	EEN	A	0	50.01	57.68	53.71	7	15.5	2
5B	D	1,680	336,413	LB	IN3	Z9N	A	0	25.00	58.10	38.11	5	272.5	75
6B	N	1,560	336,413	LB	IN3	EAN	B	0	8.34	58.64	22.12	3	306	272
7B	F	1,560	336,413	LB	IN3	4F	A	0	33.37	57.79	43.91	6	100	30
8B	F	1,650	336,413	LB	IN3	5F	A	0	33.68	57.67	44.07	8	53	29
9B*	CAO	1,560	336,413	LB	IN3	6F	A	0	50.00	58.45	54.06	4	25.5	1
10B	F	1,680	336,413	LB	IN3	19F	A	0	33.34	57.62	43.83	9	138	31

Source(s): Table by authors

Table 9

Items with ten highest importance index values

Item	Service	FSC	SIC	UOM	VOL	WSDC	WSGC	WSN	Imp	Risk	RI index	Risk rank	Imp rank	RI index rank
1C	D	4,320	333,914	LB	IN3	EON	A	0	50.03	41.22	45.42	133	7	25
2C	F	2,895	336,340	LB	IN3	4F	A	0	50.09	19.04	30.88	312	2	246
3C	D	5,330	339,991	LB	IN3	29 N	A	0	50.03	18.76	30.64	314	6	247
4C	N	5,940	335,931	LB	CF	Z9N	B	0	50.19	41.14	45.44	147	1	24
5C	D	5,315	332,510	LB	IN3	19F	A	0	50.03	23.09	33.99	308	8	143
6C	F	1,560	336,413	LB	IN3	6F	A	0	50.04	46.07	48.01	46	5	11
7C	F	1,560	336,413	LB	IN3	6F	A	0	50.03	41.38	45.50	118	9.5	22
8C	F	1,560	336,413	LB	IN3	6F	A	0	50.03	35.81	42.33	173	9.5	41
9C	F	2,945	314,999	LB	iN3	19F	A	0	50.08	57.26	53.55	17	3	6
10C	F	1,560	336,413	LB	IN3	6F	A	0	50.04	46.85	48.42	35	4	8

Source(s): Table by authors

Table 10

Items with ten highest combined index values

Item	Service	FSC	SIC	UOM	VOL	WSDC	WSGC	WSN	Imp	Risk	RI index	Risk rank	Imp rank	RI index rank
1D	F	1,650	336,413	LB	IN3	2F	C	0	50.01	46.69	48.32	38	24	10
2D	N	5,306	332,722	LB	IN3	EEN	A	0	50.02	57.60	53.68	10	18	3
3D	N	5,310	332,722	LB	IN3	EEN	A	0	50.01	57.42	53.59	13	22	4
4D	N	5,306	332,722	LB	IN3	EEN	A	0	50.02	57.68	53.71	7	15.5	2
5D	D	1,680	336,413	LB	IN3	25F	A	0	50.00	57.42	53.59	12	23	5
6D	N	1,680	336,411	LB	IN3	EAN	B	0	50.01	57.06	53.42	22	20	7
7D	F	1,560	336,413	LB	IN3	6F	A	0	50.02	46.86	48.41	34	15.5	9
9B*	CAO	1,560	336,413	LB	IN3	6F	A	0	50.00	58.45	54.06	4	25.5	1
9C	F	2,945	314,999	LB	IN3	19F	A	0	50.08	57.26	53.55	17	3	6
10C	F	1,560	336,413	LB	IN3	6F	A	0	50.04	46.85	48.42	35	4	8

Source(s): Table by authors