The application of Deep Reinforcement Learning (DRL) for the management of engineering systems has shown very promising results in terms of optimality and scalability. The interpretability of these policies by decision-makers who are so far mostly familiar with traditional approaches is also needed for implementation. In this work, we address this topic by providing a comprehensive overview of POMDP- and DRL-based management policies, along with simulation-based implementation details, for facilitating their interpretation. By mapping a sufficient statistic, namely a belief state, to the current optimal action, POMDP-DRL strategies are able to automatically adapt in time considering long-term sought objectives and the prior history. Through simulated policy realizations, POMDP-DRL-based strategies identified for representative inspection and maintenance planning settings are thoroughly analyzed. The results reveal that if the decision-maker opts for an alternative, even suboptimal, action other than the one suggested by the DRL-based policy, the belief state will be accordingly updated and can still be used as input for the remainder of the planning horizon, without any requirements for model retraining.@en

To preserve structural safety of deteriorating engineering systems through optimal maintenance, it is imperative to efficiently integrate structural health information with decision-making optimization frameworks. Although there may be abundance of available data, these are often uncertain and incomplete. In addition, joint inspection and maintenance (I&M) optimization is inherently complex due to high-dimensional state and action spaces, stochastic objectives, long planning horizons, and various constraints, among others. As shown recently, these computational challenges can be effectively addressed through optimization principles of Partially Observable Markov Decision Processes (POMDPs) and constrained Deep Reinforcement Learning (DRL). The POMDP framework provides a way of updating the decision-maker's perception about the system state by naturally incorporating the Value of Information (VoI) in the optimality equations. As such, optimal observation-gathering actions are those which guide maintenance decisions towards reduced life-cycle costs and risks. The role of VoI in DRL-driven I&M has also been shown to be central to the formation of policy gradients, which are necessary to obtain the optimal I&M plan with deep learning actor-critic architectures. Leveraging this property, a recently devised DRL architecture is further examined in this work, consisting of fully decoupled 'maintainer' and 'inspector' actors, which allow for greater efficacy and interpretability in multi-agent DRL settings. Several numerical analyses are carried out to assess the performance of the relevant architectures on stochastic systems with a varying number of components, multiple maintenance-inspection actions per component, and system-level failure risks.@en