Learning-Based Multi-UAV Flocking Control With Limited Visual Field and Instinctive Repulsion

Learning-Based Multi-UAV Flocking Control With Limited Visual Field and Instinctive Repulsion

Bai, Chengchao (Harbin Institute of Technology)
Yan, Peng (Harbin Institute of Technology)
Piao, Haiyin (Northwestern Polytechnical University; SADRI Institute)
Pan, W. (TU Delft Robot Dynamics; The University of Manchester)
Guo, Jifeng (Harbin Institute of Technology)

2024

This article explores deep reinforcement learning (DRL) for the flocking control of unmanned aerial vehicle (UAV) swarms. The flocking control policy is trained using a centralized-learning-decentralized-execution (CTDE) paradigm, where a centralized critic network augmented with additional information about the entire UAV swarm is utilized to improve learning efficiency. Instead of learning inter-UAV collision avoidance capabilities, a repulsion function is encoded as an inner-UAV 'instinct.' In addition, the UAVs can obtain the states of other UAVs through onboard sensors in communication-denied environments, and the impact of varying visual fields on flocking control is analyzed. Through extensive simulations, it is shown that the proposed policy with the repulsion function and limited visual field has a success rate of 93.8% in training environments, 85.6% in environments with a high number of UAVs, 91.2% in environments with a high number of obstacles, and 82.2% in environments with dynamic obstacles. Furthermore, the results indicate that the proposed learning-based methods are more suitable than traditional methods in cluttered environments.

Autonomous aerial vehicles
Collision avoidance
Deep reinforcement learning (DRL)
flocking control
inter-unmanned aerial vehicle (UAV) collision avoidance
limited visual field
Optimization
Reinforcement learning
Sensors
Training
UAVs
Visualization

http://resolver.tudelft.nl/uuid:9c9e7446-41fa-4ba4-84c3-03575873e5b0

https://doi.org/10.1109/TCYB.2023.3246985

2023-09-08

2168-2267

IEEE Transactions on Cybernetics, 54 (1), 462-475

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2024 Chengchao Bai, Peng Yan, Haiyin Piao, W. Pan, Jifeng Guo