Machine learning techniques have been shown to outperform many rule-based systems for the decision-making of autonomous vehicles. However, applying machine learning is challenging due to the possibility of executing unsafe actions and slow learning rates. We address these issues by presenting a reinforcement learning-based approach, which is combined with formal safety verification to ensure that only safe actions are chosen at any time. We let a deep reinforcement learning (RL) agent learn to drive as close as possible to a desired velocity by executing reasonable lane changes on simulated highways with an arbitrary number of lanes. By making use of a minimal state representation, consisting of only 13 continuous features, and a Deep Q-Network (DQN), we are able to achieve fast learning rates. Our RL agent is able to learn the desired task without causing collisions and outperforms a complex, rule-based agent that we use for benchmarking. @en

Safety is the most important aspect of systems which have to perform collision-free motions in dynamic environments. Formal verification methods, such as reachability analysis, are capable of guaranteeing safety for a given model and given assumptions (e. g. bounded velocity and acceleration). However, certain assumptions can be violated by dynamic obstacles during the execution of the verified motion plan, exposing the system to potential collisions. To compensate for the invalidated verification, this paper introduces the Point of No Return (PNR) and the Point of Guaranteed Arrival (PGA) by incorporating invariably safe sets. These concepts allow one to divide the planned trajectory into safe sections and safety-critical passageways. For the former, we are able to provide safety guarantees for an infinite time horizon. For the latter, we present a method to minimize such safety-critical passageways prior to execution and thus reduce the risk of potential collisions if assumptions are violated during execution. The safety benefits are highlighted by a numerical example of overtaking maneuvers of self-driving vehicles.@en