Interpretation and analysis of deep reinforcement learning driven inspection and maintenance policies for engineering systems

Abstract

The application of Deep Reinforcement Learning (DRL) for the management of engineering systems has shown very promising results in terms of optimality and scalability. The interpretability of these policies by decision-makers who are so far mostly familiar with traditional approaches is also needed for implementation. In this work, we address this topic by providing a comprehensive overview of POMDP- and DRL-based management policies, along with simulation-based implementation details, for facilitating their interpretation. By mapping a sufficient statistic, namely a belief state, to the current optimal action, POMDP-DRL strategies are able to automatically adapt in time considering long-term sought objectives and the prior history. Through simulated policy realizations, POMDP-DRL-based strategies identified for representative inspection and maintenance planning settings are thoroughly analyzed. The results reveal that if the decision-maker opts for an alternative, even suboptimal, action other than the one suggested by the DRL-based policy, the belief state will be accordingly updated and can still be used as input for the remainder of the planning horizon, without any requirements for model retraining.