Air traffic demand has increased at an unprecedented rate in the last decade (albeit interrupted by the COVID pandemic), but capacity has not increased at the same rate. Higher levels of automation and the implementation of decision-support tools for air traffic controllers could help increase capacity and catch up with demand. The air traffic control problem can be effectively modelled as a Markov game, where a team of aircraft (the agents) interact in the airspace (the environment) and cooperatively take resolution actions to achieve a common goal: safe separation in the most efficient way. As in any Markov game, the optimal policy for the team could be learnt through trial and error in a simulated environment using reinforcement learning algorithms. In this paper, we use the soft actor-critic algorithm to unravel the optimal air traffic control policy. Unlike some previous works, we propose a global (i.e., shared) reward that encourages cooperative behaviour. Furthermore, we propose a versatile policy model capable of performing heading, speed, and/or altitude resolution actions. We also demonstrate that the policy is robust and can maintain safe separation even in the presence of uncertainty regarding aircraft position, delays in implementing resolution actions, and wind. The findings of this paper also suggest that there is still significant room for improvement when controlling three degrees of freedom at the same time.@en

The vertical trajectory plan (altitude and speed) corresponding to the descent phase of a modern airliner is computed by the on-board flight management system while the aircraft is still in cruise. As long as the constraints on the arrival procedure allow, this system plans for an idle descent and the exact location of the (optimal) top of descent (TOD) is determined in this process. In busy terminal airspace, however, air traffic control officers – motivated by the needs to maintain a safe and expeditious flow of aircraft – might require to start the descents before the TOD computed by each particular arriving aircraft. In such situations, most flight guidance systems aim to intercept the original altitude plan from below, by using a shallower descent angle while keeping the speed plan, requiring in this way, additional thrust. This leads, consequently, to higher fuel consumption figures. The objective of this paper is threefold. Firstly, it characterises and quantifies these fuel inefficiencies for an Airbus A320, using accurate aircraft performance data and a trajectory computation software from the manufacturer. Secondly, it proposes a methodology to automatically identify early descents and to extract the key parameters required to compute the fuel inefficiencies by only observing ADS-B (automatic dependent surveillance-broadcast) data. Finally, the method is applied to a case study with 4,139 real ADS-B trajectories in Amsterdam-Schiphol (The Netherlands) terminal airspace; showing that early descents are very frequent and that they increase the fuel consumption by a 5%, in average. @en