Transportation infrastructure asset management modeling using Markov decision process under epistemic uncertainties

Abstract

Purpose

Transportation infrastructure asset management has long been an active but challenging problem for agencies, which urges to maintain a good state of their assets but faces budgetary limitations. Managing a network of transportation infrastructure assets, especially when the number is large, is a multifaceted challenge. This paper aims to develop a life-cycle cost analysis (LCCA) based transportation infrastructure asset management analytical framework to study the impacts of a few key parameters/factors on deterioration and life-cycle cost. Using the bridge as an example infrastructure type, the framework incorporates an optimization model for optimizing maintenance, repair, rehabilitation (MR&R) and replacement decisions in a finite planning horizon.

Design/methodology/approach

The analytical framework is further developed through a series of model variations, scenario and sensitivity analysis, simulation processes and numerical experiments to show the impacts of various parameters/factors and draw managerial insights. One notable analysis is to explicitly model the epistemic uncertainties of infrastructure deterioration models, which have been overlooked in previous research. The proposed methodology can be adapted to different types of assets for solving general asset management and capital planning problems.

Findings

The experiments and case studies revealed several findings. First, the authors showed the importance of the deterioration model parameter (i.e. Markov transition probability). Inaccurate information of p will lead to suboptimal solutions and results in excessive total cost. Second, both agency cost and user cost of a single facility will have significant impacts on the system cost and correlation between them also influences the system cost. Third, the optimal budget can be found and the system cost is tolerant to budge variations within a certain range. Four, the model minimizes the total cost by optimizing the allocation of funds to bridges weighing the trade-off between user and agency costs.

Originality/value

On the path forward to develop the next generation of bridge management systems methodologies, the authors make an exploration of incorporating the epistemic uncertainties of the stochastic deterioration models into bridge MR&R capital planning and decision-making. The authors propose an optimization approach that does not only incorporate the inherent stochasticity of bridge deterioration but also considers the epistemic uncertainties and variances of the model parameters of Markovian transition probabilities due to data errors or modeling processes.

Citation

Bai, Y., Babanajad, S. and Bian, Z. (2021), "Transportation infrastructure asset management modeling using Markov decision process under epistemic uncertainties", Smart and Resilient Transportation, Vol. 3 No. 3, pp. 249-265. https://doi.org/10.1108/SRT-11-2020-0026

Introduction

Transportation infrastructure asset management has long been an active but challenging problem for agencies, which urges to maintain the good state of their assets but faces budgetary limitations. Managing a network of transportation infrastructure assets, especially when the number is large, is a multifaceted challenge. It requires not only profound knowledge on deterioration mechanism and efficacy of intervention strategies but also a good understanding of the costs of management actions and strategies to different stakeholders. It is essentially a resource allocation problem among a set of elements of one facility or a network of facilities with multifold trade-offs between upfront and future expenses, short-term (responsive) fix and long-term (preventive) preservation, agency and user costs (e.g. level of service), etc. The problem is further complicated by considering the system uncertainties, such as the stochastic deterioration process of infrastructure, as well as data and modeling limitations. Addressing this challenge requires innovative approaches to account for various trade-offs and uncertainties.

Life-cycle cost analysis (LCCA) is a well-recognized concept and methodology for transportation infrastructure management. Its principle is to make economic decisions considering initial investments and future expenses and all essential cost components (e.g. agency and user cost). Typical steps of traditional infrastructure LCCA include defining a base case and several alternative management strategies, specifying deterioration and cost parameters, calculating net present values of the total life-cycle cost (LCC) and finally reviewing and selecting the preferred strategies (NCHRP 12–43). However, traditional LCCA is only able to assess a limited number of maintenance, repair and rehabilitation (MR&R) strategies or scenarios. The number of possible decision scenarios could be enormous and quickly ramp up with problem scale (i.e. problem complexity depending on numbers of assets, decision cycles, actions, constraints, etc.). It is inefficient and practically difficult to enumerate all strategies using traditional LCCA for large-scale problems.

In addition, traditional LCCA normally schedules treatments based on deterministic and static deterioration mechanisms. While the state-of-the-art deterioration models incorporate stochastic deterioration processes into the infrastructure MR&R optimization (e.g. Markov models in pavement and bridge management), the “epistemic” uncertainties of the stochastic deterioration models are rarely considered. The parameters in these models are usually estimated by statistical analysis outputting single average values. If the estimated values of the deterioration parameters differ from their “true” values, either above or below, it will result in suboptimal management solutions and consequently excessive cost to agencies and users. This “internal” error in deterioration modeling inevitably happens due to lack of knowledge, data error and model limitation and such uncertainty are known as “epistemic uncertainty” in statistical theory. Existing infrastructure management optimization models generally take single estimated values (such as the mean values) of the deterioration model parameters and ignore their variances, which could have a non-negligible impact on the results.

This study develops an LCCA-based modeling and analytical framework for managing a network of transportation infrastructure assets. The proposed methodology framework is applicable to general infrastructure types, such as bridges, pavement, pipelines, railways, waterways. Using the bridge as an example, the framework is developed in the context of strategic management of a network of bridge inventory and can be effectively used at the operational-level management of other infrastructure kinds. We use Markov decision process (MDP) optimization techniques to compute the best out of all possible combinations of MR&R decision scenarios for a number of assets over a dynamic, multi-year time horizon. On top of the inherent stochasticity in infrastructure deterioration, we proposed a simulation model to account for the probability distribution and variances of model parameters.

Literature review

The current infrastructure asset management literature develops and optimizes an integrated, multidisciplinary set of strategies with the fundamental goal to minimize the LCC and extend the service life of the assets (Cagle, 2003). In this section, we summarize a literature review that covers important concepts and methodological taxonomy of previous research studies of infrastructure asset management and LCC modeling.

Life-cycle cost analysis

LCCA is a core concept in developing infrastructure asset management systems. The central principle is to make economic decisions considering initial investments and future expenses, as well as the LCC components. The conventional LCCA methods usually involve identifying a number of promising alternatives for various combinations of MR&R strategies. Once selected, each alternative is then analyzed with the alternative’s activity profiles and cash flows for project-level and network-level decisions. The typical steps entail defining a base case and several alternative management strategies, specifying deterioration and cost parameters, calculating the net present values of the total LCC and finally reviewing and selecting preferred strategies (Hawk, 2003). In particular, it involves decision-making among preventive maintenance, repair and replacement intervals, evaluating new maintenance materials and techniques and responding effectively to funding constraints (Thompson et al., 2012). To obtain precise outcomes requires a good understanding of the costs associated with different intervention strategies incurred by different stakeholders. In fact, it is essentially a resource allocation problem addressing multifold trade-offs among upfront and future expenses (on short-term repair or replacement and long-term preventive preservation), investment priorities to certain facilities over other facilities, cost incurred by different stakeholders, i.e. agency and user costs (and other social costs), etc. While simple LCCA identifies best MR&R strategies by evaluating and comparing a set of alternatives using equivalent uniform annual cost, equivalent uniform annual cost or internal rate of return, the state-of-the-art LCC models use optimization techniques to minimize the expected LCC of keeping the facilities above a minimum service level while satisfying agency budget and policy constraints (Thompson et al., 2012). For instance, Hawk (2003) provided a systematic overview of such methodology applied to bridges.

Bridge maintenance, repair and rehabilitation capital planning and optimization

Strategic capital planning for transportation infrastructure asset management is a complex decision-making process, involving the identification, screening, evaluation, comparison and prioritization of multiple activities under alternative funding scenarios. It is especially challenging for agencies when managing a large number of assets that compete for limited financial, material and manpower resources to produce smooth cash flows across different financial cycles. To that extend, the optimization models are developed to determine MR&R decisions for a single or multiple facilities over a planning horizon with the objective to minimize total expected system costs associated with agency and user costs (Liu and Madanat, 2013). The literature of transportation infrastructure asset management optimization systems can be primarily categorized based on the scope (project or network levels), uncertainty consideration (deterministic or stochastic), time and state spaces (continuous or discrete, finite or infinite horizon), etc.

For a single bridge (facility/project-level) problem, the prevailing bridge management literature applies MDPs and dynamic programming for optimizing MR&R decisions of a single bridge facility and the variations components are mainly the deterioration models. More recently, novel approaches for stochastic deterioration modeling (e.g. hazard-based duration models), adaptive control methods (e.g. certainty equivalent control) and reliability-based modeling has been developed to investigate bridge deterioration and MR&R optimization (Hensher and Mannering, 1994; Mishalani and Madanat, 2002; Liu and Madanat, 2013; Robelin and Madanat, 2007; Thoft-Christensen, 1995; Frangopol, 1999; Frangopol and Das, 1999; Das, 2000; Frangopol et al., 2000a, 2000b). For network-level problems, the overarching approaches of optimization can be grouped into two categories. The top-down approach aggregates facilities having similar structural systems, traffic and environmental characteristics into mutually exclusive and collectively exhaustive homogeneous groups. This method addresses the trade-off between rehabilitation of a small number of facilities and maintenance of a larger number of facilities. However, it falls short in accounting for all the detailed project-specific information, mostly relying on the experience of in-charge engineers.

Alternatively, the bottom-up approach first selects a small set of optimal (or near-optimal) sequences of MR&R activities for each facility, covering the desired planning horizon (Yeo et al., 2012). For a given budget, this approach then selects the combination of sequences (one for each facility) that meets the budget constraint while optimizing a system-wide objective (Robelin and Madanat, 2006). Compared to the top-down approach, the bottom-up approach preserves the identity of individual facilities, with all its information (structure, materials, history of construction, MR&R, traffic loading and environment) available. In the meantime, this significantly adds up to the complexity of system-level decision-making procedures.

Recent studies proposed dependency-based models to model the functional and economic dependencies between the facilities by incorporating nonlinear constraints. For example, contiguous facilities are best rehabilitated in the same year to reduce delay costs during the rehabilitation or the other way around when two facilities provide alternative routes. The central challenge to such a problem is the increased computational complexity due to the additional constraints and nonlinearity. The Heuristics algorithms are among the recently developed solutions to solve such computational problems (Yeo et al., 2012). More recently, Wu et al. (2017) developed a life-cycle optimization model using the semi-Markov process for highway bridge maintenance. Nazari et al. (2017) integrate the simultaneous network optimization and facility-level latent MDP into a unifying framework that optimizes spatial and temporal distributions of inspection and MR&R decisions for a system of infrastructure facilities considering measurement uncertainty. Boomen et al. (2020) developed MDP-based optimization models that incorporate the flexibility to choose between multiple successive intervention strategies, regular asset degradation, structural failure and multiple price uncertainties.

Measurement of uncertainties in bridge deterioration

The deterioration of an infrastructure facility, such as a bridge, follows a stochastic process that highly depends on the continuous variation of several factors, many of which are generally not captured by available data (Mishalani and Madanat). To date, only a few researchers have addressed uncertainty in estimating the progression of deterioration in bridge components (Mishalani and Madanat, Agrawal, Babanajad). Among all, Madanat and Ben-Akiva (1994) were among the first to apply a methodology for MR&R activity selection that accounts for the presence of both forecasting and measurement uncertainty. This methodology is the latent MDP, an extension of the traditional MDP that relaxes the assumptions of error-free annual facility inspections. Later, Smilowitz and Madanat (2000) extend this methodology to include network-level constraints. Most existing bridge management systems (BMS) and life-cycle models share similar theoretical bases while the differences rely mostly on the deterioration models. The state-of-art optimization theory, techniques and computational advances are not fully explored or implemented, especially for large-scale network-level problems in which the computational complexity is a major challenge. The development of the next generation of BMS is expected to be an integration of the most advanced data-driven, reliability-based deterioration models and LCC optimization methodologies.

On the path forward to develop the next generation of BMS methodologies, we make an exploration of incorporating the epistemic uncertainties of the stochastic deterioration models into bridge MR&R capital planning and decision-making. We propose an optimization approach that does not only incorporate the inherent stochasticity of bridge deterioration but also considers the epistemic uncertainties and variances of the model parameters of Markovian transition probabilities due to data errors or modeling processes. We simulate how these uncertainties affect the cost estimation and optimal decision. The approach presented here is also useful for risk reduction of budget overruns by explicitly considering its variances quantified through our proposed model.

Markov decision process optimization model formulation

In this section, we formulate a general version of a network-level bridge MR&R problem based on MDP theories. The top-down formulation concept in Liu and Madanat (2013) is adopted to optimize the MR&R decisions on a strategic level. The core model will be presented as mixed-integer programming, which minimizes the total user and total agency costs during a projected time horizon.

Two types of MDP policies are explored including a randomized and a non-randomized policy. Non-randomized MDP provides the deterministic (discrete) MR&R solutions that can be implemented directly, i.e. which action to take in each decision cycle (e.g. year-based). It models the decision-making process considering all possible combinations of MR&R strategies for a network of bridges over a multi-year horizon. In comparison, the randomized MDP provides a probabilistic spectrum of MR&R actions instead of a deterministic solution in each decision cycle. To obtain implementable action items, an additional step of random sampling (simulation) of the developed probabilistic solution is required. Randomized MDP models are often used because their computational advantages are given that they are essentially continuous optimization problems. Conversely, the computational need for non-randomized MDP is much higher though it leads to the theoretically “exact” optimal solution. To handle this challenge and obtain an immediately usable solution, the proposed core model adopts a hybrid of the two MDP policies. It includes both non-randomized (discrete) decisions for the first year and randomized (continuous) decisions for the years after. In other words, it makes actual MR&R decisions for the current planning year and only estimates the expected future cost considering a probabilistic MR&R strategy. Such a hybrid MDP model satisfies agencies’ needs for both short-term and long-term asset management and more importantly, is much more computationally tractable for large-scale asset inventories.

Input parameters

τ ∈ T = {1,2,…,T}: year τ in a planning horizon from year 1 to T.k ∈ K: bridge facility k in the set of bridge facilities K. a ∈ A: MR&R action a in a set of actions A. s ∈ S: condition state s in a set of discrete conditions S. s_k: initial condition state of facility k at the beginning of the current year.

α: discount factor, α=1(1+i) where i is the annual interest rate.

Ck,aτ: agency cost of implementing MR&R action a on facility k in year τ.

Uk,sτ: user cost associated with facility k at condition state s in year τ.

Pk,aτ=[Pk,aτ(s,s′)]|S|×|S∣: transition matrix for facility k with the application of MR&R action a and Pk,aτ(s,s′) is the (s,s′) entry of it.

B_τ: annual budget in year τ.

Objective function

The objective function contains two components: user cost and agency cost for the first year and the user cost and the agency cost for the all following years. The system objective is to minimize the system costs, which is the sum of agency costs and user costs over a specified planning horizon. As we discussed previously, the core model is based on a hybrid of the non-randomized and randomized MDP policies. In the first year, MR&R actions (variable x) need to be determined deterministically, while in all following years the probabilities of actions (variable w) are determined. Thus, the objective function and constraints for the first year and the rest of the years are formulated separately shown in the following equation:

(1) min w , x ∑ k ∈ K [ ∑ a ∈ A C k , a 1 ⋅ ( ∑ s ∈ S U k , s 1 ) ⎵ current year agency costs + ∑ s ∈ S U k , s 1 ⋅ ( ∑ a ∈ A x k , a , s ) ⎵ current year user  cos ⁡ t s + ∑ τ ∈ T \ { 1 } ∑ a ∈ A α τ − 1 C k , a τ ⋅ ( ∑ s ∈ S w k , a , s τ ) ⎵ discounted agency costs in future years + ∑ τ ∈ T \ { 1 } ∑ a ∈ A α τ − 1 U k , a τ ⋅ ( ∑ a ∈ A w k , a , s τ ) ⎵ discounted user costs in future years ]

Decision variables

x_k,a,s ∈ {0,1}: binary variable with 1 assigning action a to facility k and 0 otherwise, for the current year, i.e. year τ = 1 only. wk,a,sτ∈[0,1]: represents the probability of facility k in state s and assigned with action a at the beginning of year τ > 1.

The timing of actions is assigned with the assumption that the agency cost, C, is incurred at the beginning of each year while the user cost, U and condition rating are evaluated at the end of each year. Therefore, the decision variable, wk,a,s0, represents the initial condition at the beginning of the first year. The Markov transition matrix of the first year P¹ is also known at the beginning of the first year while the transition process is completed at the end of each year.

Constraints

Constraints contain the state transition for each year, the budget limitation for each year and the state condition constraint for a group of bridge during the same year.

(5) ∑a∈A∑s∈Sxk,a,sPk,a1(s,s′)=∑a′∈Awk,a′,s′2,∀s′∈S,k∈K

(2) ∑a∈A∑s∈Swk,a,sτPk,aτ(s,s′)=∑a′∈Awk,a′,s′τ+1,∀s′∈S,k∈K,τ∈T\{1,T}

(3) ∑k∈K∑a∈A(∑s∈Sxk,a,s)⋅Ck,a1≤B1

(4) ∑k∈K∑a∈A(∑s∈Swk,a,sτ)⋅Ck,aτ≤Bτ,∀τ∈T\{T}

(6) 0≤wk,a,sτ≤1,∀s∈S,a∈A,k∈K,τ∈T\{1}

(7) xk,a,s∈{0,1},∀s∈S,a∈A,k∈K

(10) xk,a,s=0,∀s≠sk,a∈A,k∈K

(8) ∑a∈Axk,a,sk=1,∀k∈K

(9) ∑a∈A∑s∈Swk,a,sτ=1,∀k∈K,τ∈T\{1}

Constraint (2) describes the Markov transition process from the first year to the second year. The probability of each bridge staying at a certain condition state is dependent on the action and its corresponding transition matrix. Similarly, constraint (3) describes the similar process from year 2 to T. Constraints (4) and (5) impose budget for year 1 and year 2–T, respectively. Constraints (6) and (7) define variable types. Furthermore, we also add auxiliary constraints, which are unnecessary for the model definition but facilitate computation, i.e. constraints (8)–(10). Constraints (8) specify that all decision variables in the first year with a condition that is the actual initial condition of a bridge should equal to 0. Constraint (9) ensures that only one action can be selected for each bridge in the first year. Finally, constraint (10) stipulates that the sum of probabilities of taking all actions must equal to 1 for years 2 to T.

Case study

An illustrative case study is set up herein based on synthetic data to test and validate the proposed framework. Default values of a benchmark scenario are chosen for deterioration transition probabilities and costs as follows:Planning horizon: T = 20 year.Number of bridge facilities: K = 100.MR&R actions: A = {Repair, Do-nothing (DN)}.Discrete conditions: S = {Good,Bad}. Initial state: all bridges are assumed in a “Good” condition state.Discount factor and annual interest:

α=1(1+i)=11+0.06=0.9434

Annual budget: B_τ = $20,000 per year.

User cost ($/year): randomly generated following a normal distribution with a mean of $800 and standard deviation $100 across an inventory of 100 bridges; user cost only exists when the condition is bad.

Agency cost ($/repair action): randomly generated following a normal distribution with a mean of $1,400 and standard deviation of $200 across the inventory.

Total costs = user cost + agency cost (for all 50 years of the planning horizon).

Deterioration and repair mechanisms: Markov transition matrices where p = 0.4, p₁ = 0.3, p₂ = 0.1 are chosen as default values (Table 1). Note that realistically p > p₁ > p₂ should be hold.

In the benchmark case, we use the same deterioration models for all bridges (homogenous case) and we will perform a heterogeneous deterioration model case in the subsections to follow.

Benchmark scenario results – maintenance, repair and rehabilitation decision

We use the designed data presented above as a benchmark scenario. The core model is able to make deterministic MR&R decisions (discrete solution) only for the first year and estimate the expected cost (based on randomized MR&R decision strategy) in all remaining years, designated herein as the prediction results. The core model is in essence a mixed-integer program (MIP) in which the majority of the decision variables are continuous (i.e. randomized MR&R decision variables). It can be directly solved by commercial solvers or commonly used MIP algorithms. In this study, we coded the simulation process in R and the core MIP model in general algebraic modeling system and solved it using solver CPLEX. To obtain discrete MR&R action solutions for all periods in the planning horizon, a simulation model was used that samples the MR&R decisions and the condition ratings, based on the randomized solution of the prediction results. For example, the core model was solved for year 1 obtaining the MR&R decision x (binary variable). Then, the bridge condition rating was simulated in the second year based on the transition probability corresponding to the MR&R decision x and the initial condition rating in the first year. The MR&R decision was simulated by sampling the discrete distribution of the randomized MR&R solution. Based on the simulated MR&R action and new condition rating in the second year, a similar process can be applied to obtain realized values of the two variables for all the following years. A complete, discrete MR&R solution for all years can be obtained at the end year of such a process. Note that such simulation is able to provide a solution sample, which may not be exactly optimal (in comparison to a fully non-randomized policy MDP model). The simulation results (e.g. the mean of total cost) converge to the prediction results if the same simulation process is repeated a sufficient number of times.

Under the benchmark scenario setting, the optimization model is run with the simulated randomized agency and user cost input to solve for the 100-bridge inventory over a 20-year horizon. Results of some example bridges are illustrated in Figure 1. We found that the repair decision is directly dependent on the ratio between user cost and agency cost. To further investigation into the results, repair decisions of all 100 bridges are ranked from high to low based on this ratio and we group the bridges into four categories each containing 25 bridges; Table 2. The first category contains the bridges with the highest ratio, while the last category contains the bridges with the lowest ratio.

The bridge numbers in bold, red font are those that will never be “repaired” in the model result during the first 20 years and they all belong to Category IV. For Category I and Category II in which the user/agency cost ratio (U/A ratio) is high, the same decision mechanism is resulted – always “repair” when the bridge is in a “bad” condition state; Figure 1, where red squares stand for the action of “repair.” For Category III where the user/agency cost ratio is lower than Category I and Category II, we do not always “repair” when bridges become “bad”; Figure 2. For example, the bridge no. 90 will not be repaired in year 9 when it degrades to a “bad” condition state and so will bridge no. 61 in year 5 and after year 8. For Category IV in which the user/agency cost ratio is the lowest, more bridges will not be “repaired” when they are in “bad” condition; Figure 3.

Deterioration probability “epistemic” uncertainties impact the total cost

To investigate how uncertainties of deterioration model parameters affect the optimal decision and cost estimation, the transition probability p in the decision-making model was compared against the actual p that the bridges inherently follow to naturally deteriorate. To avoid complication, we test only the DN Markov matrix parameter p (the deterioration probability from “good” to “bad” when “doing nothing”) with the repair transition probabilities, which were kept the same as the benchmark case (p₁ = 0.3 and p₂ = 0.1).

We call the former p Decision-Making P, noted as DMP; and actual p Condition Generation P, noted as CGP. DMP is considered as an estimated deterioration rate that is used in the prediction model, while CGP is considered as the natural deterioration rate and is used in simulating the condition rating transition in each year. Homogenous for all bridges, we set both DMP and CGP in the range of 0.2–0.6, so there are a total of 25 combination scenarios of different DMP and CGP values. The 50-year total cost is calculated by summing up the first-year costs of each decision cycle in a 100-year rolling horizon.

In Table 3, the predicted costs in the second row are obtained directly from the optimization model under different p values (which we call decision-making probability – DMP). Under each DMP, the actual costs in that column are computed from simulation under each CGP from 0.2 to 0.6 and under the MR&R actions determined by the optimization model using the corresponding DMP value. The results in Table 3 reveal that for a certain CGP, the total cost is almost the lowest when DMP is equal to CGP, i.e. the numbers on the diagonal line, which represent the cases when the true deterioration probability happens to be exactly the same as the user parameter. This verifies our initial postulate that inaccurate information of p will lead to suboptimal solutions. This also highlights the importance of accurate deterioration modeling in asset management. If the deterioration probability parameter used for the MR&R decisions is different from the actual deterioration probability, it results in unreliable total cost. This happens inevitably because of data and model limitations and such uncertainty is the so-called epistemic uncertainty in the statistical modeling of the deterioration process.

Alternatively, when DMP was kept fixed, the total cost increases dramatically with CGP. This implies that the simulated (representing the actual) cost varies largely depending on the actual deterioration probability. Accurate cost estimation would be difficult without a good knowledge of actual deterioration probability. However, if we look at the results horizontally, the cost differences are much smaller. Therefore, we can conclude that the effects of deterioration model parameter p on the cost estimation highly outweigh the effects of the MR&R decisions in the optimization framework.

Impacts of user and agency costs on the total cost

In practice, agencies often make MR&R decisions dynamically on a rolling horizon basis. To emulate such a decision process, we solve the core optimization model in rolling decision cycles, each of which, for example, covers a 20-year planning horizon. We call such a decision process dynamic simulation and decision process (DSDP), which is capable to provide an MR&R solution for the immediate decision period while incorporates the long-term cost projection into the future for a certain cycle length. Figure 4 illustrates the DSDP as a sequential process in comparison with the prediction case in which conditions and costs during the entire planning horizon are simultaneously estimated upfront.

The results of DSDP under the benchmark parameter settings are obtained by solving for MR&R decisions for all 100 bridges over a 20-year horizon. For example, we solve the core model for year 1 obtaining the MR&R decision x (binary variable). Then we simulate the bridge condition rating in the second year based on the transition probability corresponding to the MR&R decision x and the condition rating in the previous year. Then we use this information as an input to solve the core model for year 2 and repeat this process year by year for all subsequent years.

Sensitivity analysis is of importance for assessing the impacts of the ratio between user cost and agency cost on the total cost. In reality, user cost and agency cost are likely to be correlated; for example, larger bridges with high user costs tend to cost more for repair. To emulate this observation, we assume the user and agency cost follow a multi-normal distribution across the bridge population. We randomly generate 9 data sets, using different mean values for user and agency cost across bridges. The parameter ρ represents the correlation level between agency and user cost. Higher ρ means a higher degree of correlation, where ρ = 1 indicates a full correlation. Table 4 lists input data and parameter values for the sensitivity analysis.

Figure 5 visualizes how total cost varies with the user cost. Starting from each year through the next 50 years is considered as a decision cycle. We run the model for a total of 50 decision cycles. As shown in the figure, when the correlation coefficient between user and agency costs is 0.6 and unit agency cost is fixed at $800, the total cost of each decision cycle increases with the user cost. Also, the variation tendency (curve shape) of the total cost for these three different user cost sets is similar. Besides, note that the total costs for the initial years are much smaller than the following years. This is because the initial condition states of bridges are all “good” and the action of “repair” is not needed. Therefore, the agency and user costs are both at a lower level in the initial years and the total cost ramps up in consecutive years and gradually becomes stable.

We also investigated the impact of correlation between user and agency costs on the total cost. In this experiment, the agency cost mean is fixed to $800 and the user cost mean (µ) is fixed to $1,400. In Figure 6, the total cost of each decision cycle will increase when the correlation coefficient increases. The underlying principle is harder to perceive in this case. If user and agency costs are fully correlated (ρ = 1), the user/agency cost ratio is almost fixed and the same for all bridges. This reveals that the effect of different bridges on the total cost is the same and no bridges save cost more than others. However, if ρ = 0.6, there will be some bridges having a higher user/agency cost ratio so that repairing these bridges will bring more benefit in terms of reducing the total cost. Therefore, the total system cost is lower in this case than that in ρ = 1.

Optimal budget

One of the key research questions of the LCCA analysis is how to determine the optimal budget. Through our experiment, we demonstrated that there is actually an optimal budget that can achieve the lowest total system cost. We perform a sensitivity analysis on an annual budget from $10,000 to $50,000. We run the core model, i.e. prediction model, with the objective function to minimize the user cost only and the annual budget to vary. Figure 7 shows that the optimum annual budget does exist at the lowest point of the total cost curve. When the annual budget is $30,000, we can get the minimum total cost of $1,189,218. Beyond this point, the reduction of user cost cannot offset the increase in agency cost (budget used). This implies that it is not “more spending the better” and there is a socially optimal point to which degree infrastructure should be maintained.

Concluding remarks and future research

The analytical framework in this paper provides a quantitative method to assess a number of factors and concerns, such as the impact of bridge deterioration model on system cost estimation and optimal decision, the trade-off between early intervention and preservation extend service life and reduce future cost, the costs and benefits of taking actions today and in future in terms of life cycle cost, the allocation of limited funds to a large number of bridge assets, the compromise between agency and user cost under a limited budget and the impact of uncertainties due to data and model limitations on system cost estimation and optimal decision.

Our experiments and case study revealed several findings. First, we showed the importance of the deterioration model parameter (i.e. Markov transition probability). The results are highly dependent on the deterioration model used. Inaccurate information of p will lead to suboptimal solutions and results in excessive total cost. Second, both agency cost and user cost of a single facility will have significant impacts on the system cost and correlation between them also influences the system cost. Third, the optimal budget can be found and the system cost is tolerant to budge variations within a certain range. The difference between the model estimation and the simulated cost is not statistically significant. Heterogeneity and variance of deterioration model parameter (i.e. transition probability p) across bridges offset each other reducing the total deviation with the predicted value, but lead to larger variance. Four, the case study results coincide with our intuition that the bridges with relatively higher user costs should have priority to be repaired given a limited budget. The model minimizes the total cost by optimizing the allocation of funds to bridges weighing the trade-off between user and agency costs.

The model and analytical framework, when packaged into bridge management systems, could be a useful tool for assisting stakeholders with the efficient allocation of available funding and decision-making for MR&R planning and budgeting. As an initial step, this study is a stepping stone for long-term research efforts into the more advanced and comprehensive analytical framework and the next generation of bridge management systems. Given the complexity of bridge management problems, structural deterioration mechanism, factor correlations and element interdependencies, uncertainties, heterogeneity, network connectivity and special agency concerns need to be further studied. The ultimate goal is to help the industry move toward data-driven bridge management practices to make better-informed and robust decisions.

One of the research questions that are of interest to the industrial is budget reliability. For example, what is the sufficient budget at 95% of a chance to maintain the network bridges at desired condition level? To answer the question, reliable and robust optimization theories can be applied. Extension of the current model could also be made to incorporate network connectivity concerns, for instance, repairing certain bridges at the same time to use an economy of scale or avoiding repair certain bridges at the same time to maintain network connectivity for some essential routes. Another future research direction would be optimizing the MR&R of the entire bridge system of heterogeneous and interdependent structural components. Most existing studies either focus on bridge deck only or ignore the interdependencies among elements and subsystems. Advanced “system of systems” modeling and simulation approach could be applied to improve system reliability and reduce the real total LCC.

Figures

Figure 1.

Condition state and repair decision for sample bridges in Category I and Category II

Figure 2.

Condition state and repair decision for sample bridges in Category III

Figure 3.

Condition state and repair decision for sample bridges in Category IV^a

Figure 4.

Illustration of the dynamic simulation and decision process (DSDP)

Figure 5.

Total cost curves in 50 decision cycles by solving DSDP model with different user cost mean values

Figure 6.

Total cost curves over the 50 decision cycles by solving DSDP model with different correlation coefficients

Figure 7.

Sensitive analysis results of annual budget levels and optimal budget

Table 1.

Markov transition matrices for do nothing (DN) and repair in the case study

Do nothing	Bad	Good	Repair	Bad	Good
Bad	1	0	Bad	p1	1−p1
Good	p	1−p	Good	p2	1−p2

Table 2.

Four categories of bridges classified by the ratio of user cost/agency cost

Category I		Category II		Category III		Category IV
Biggest rate		Second biggest rate		Second smallest rate		Smallest rate
Bridge no.	75	Bridge no.	10	Bridge no.	18	Bridge no.	22
Bridge no.	45	Bridge no.	70	Bridge no.	28	Bridge no.	8
Bridge no.	11	Bridge no.	88	Bridge no.	94	Bridge no.	42
Bridge no.	80	Bridge no.	55	Bridge no.	16	Bridge no.	29
Bridge no.	17	Bridge no.	27	Bridge no.	81	Bridge no.	47
Bridge no.	26	Bridge no.	67	Bridge no.	56	Bridge no.	43
Bridge no.	51	Bridge no.	13	Bridge no.	76	Bridge no.	5
Bridge no.	53	Bridge no.	37	Bridge no.	44	Bridge no.	40
Bridge no.	50	Bridge no.	2	Bridge no.	66	Bridge no.	91
Bridge no.	62	Bridge no.	4	Bridge no.	100	Bridge no.	74
Bridge no.	32	Bridge no.	48	Bridge no.	9	Bridge no.	24
Bridge no.	93	Bridge no.	73	Bridge no.	57	Bridge no.	15
Bridge no.	96	Bridge no.	63	Bridge no.	90	Bridge no.	97
Bridge no.	83	Bridge no.	25	Bridge no.	87	Bridge no.	38
Bridge no.	14	Bridge no.	49	Bridge no.	86	Bridge no.	3
Bridge no.	46	Bridge no.	82	Bridge no.	21	Bridge no.	64
Bridge no.	39	Bridge no.	85	Bridge no.	6	Bridge no.	34
Bridge no.	92	Bridge no.	33	Bridge no.	59	Bridge no.	78
Bridge no.	20	Bridge no.	30	Bridge no.	84	Bridge no.	69
Bridge no.	71	Bridge no.	41	Bridge no.	23	Bridge no.	77
Bridge no.	19	Bridge no.	95	Bridge no.	60	Bridge no.	65
Bridge no.	68	Bridge no.	31	Bridge no.	72	Bridge no.	35
Bridge no.	36	Bridge no.	58	Bridge no.	12	Bridge no.	7
Bridge no.	1	Bridge no.	54	Bridge no.	98	Bridge no.	89
Bridge no.	52	Bridge no.	99	Bridge no.	61	Bridge no.	79

Table 3.

Total cost of 50-year time horizon in different combination scenarios of DMP and CGP values

DMP CGP		DMP = 0.2	DMP = 0.3	DMP = 0.4	DMP = 0.5	DMP = 0.6
Predicted cost		740,885	1,006,881	1,226,706	1,407,163	1,521,997
Actual cost (simulation)	CGP = 0.2	739,995	742,165	745,480	789,306	1,121,201
	CGP = 0.3	1,002,979	1,007,332	1,009,216	1,030,996	1,248,843
	CGP = 0.4	1,229,312	1,230,125	1,228,215	1,238,869	1,356,369
	CGP = 0.5	1,418,983	1,417,435	1,414,033	1,408,375	1,448,594
	CGP = 0.6	1,573,440	1,568,551	1,570,593	1,556,277	1,524,880

Table 4.

Agency and user cost mean, standard deviation and correlation parameters

Cost items	Correlation	ρ = 0.6	ρ = 0.8	ρ = 1.0
Agency cost (for the repair action) ($)	Mean	800	800	800
	Standard deviation	100	100	100
User cost ($/year)	Mean (µ)	1,400/1,500/1,600	1,400/1,500/1,600	1,400/1,500/1,600
	S.D.	200	200	200