Structural systems must satisfy multiple performance and functionality requirements during their life cycle, withstanding safety-reducing degradation mechanisms and hazards. Intervention strategies must be planned accordingly to maintain structural integrity and minimize total life-cycle costs and risks, posing a complex optimization problem. Recent advances in multi-agent deep reinforcement learning (DRL) in conjunction with partially observable Markov Decision Processes (POMDPs) have shown great potential for determining optimal structural integrity management policies for systems with large state and action spaces compared to traditional decision practices. This paper tackles the maintenance optimization problem of aging bridges in seismic-prone areas, creating an updatable environment that embeds chloride-induced corrosion and state-dependent seismic fragility throughout the bridge life-cycle. The evolution of the environment is captured by a dynamic Bayesian network, and it is further integrated with decentralized multi-agent DRL algorithms to identify near-optimal lifecycle decisions under risk constraints. Results on a multi-component bridge system show the suitability of the developed framework for minimizing expected life-cycle costs, and for providing detailed and adaptive policies that significantly outperform traditional condition- and time-based maintenance plans.@en

Maintenance planning of engineering systems is typically posed as a discrete stochastic optimal control problem, as it refers to determining a series of distinct interventions that upkeep structural integrity. Advanced algorithmic schemes within the joint framework of Partially Observable Markov Decision Processes (POMDPs) and multi-agent Deep Reinforcement Learning (DRL) have been recently able to approximate well global optima for this complex problem, outperforming existing time- and condition-based decision strategies. Integral to their success is the hypothesis that system components represent individual agents who form cooperative policies to minimize a central life-cycle cost. Thereby, the policy output scales linearly with the number of components, alleviating the curse of dimensionality related to combinatorial choices. State complexity and long-term optimality are handled efficiently via deep learning and POMDP principles, respectively. However, the efficiency of multi-agent coordination can fade as the number of agents increases. To this end, we propose a new formulation: we pose the problem as a continuous-control dynamic resource allocation one, combining hierarchical DRL and mixed-integer programming. Moving from flat decentralized to hierarchical multi-agent decompositions allows us to improve further the policy output scalability. The new Adaptive Knapsack Hierarchical Resource Allocator (AK-HRA) DRL architecture distributes available resources within the system, creating local, independently solvable, multi-choice knapsack optimization problems. By design, AK HRA allows decision-makers to inscribe known hierarchical structures and local decision rules in their architectures, thereby enhancing control and interpretability over the solution space. The efficacy of the new approach is demonstrated in a multi-component reliability system subject to stochastic deterioration.@en