Reinforcement learning, especially deep reinforcement learning, has made many advances in the last decade. Similarly, great strides have been made in multi-agent reinforcement learning. Systems of cooperative autonomous robots are increasingly being used, for which multi-agent reinforcement learning can be used as a training method. However, the curse of dimensionality remains a problem for the computational speed of the learning algorithm and the bandwidth of communication channels. This research will focus mainly on reducing the problem of overloading communication channels by trying to reduce the number of communications. This is possible since it is usually unnecessary for every agent to communicate with every other agent constantly.

To do this, we use some ideas by Daniel Jarne Ornia. The first is to reduce communications in a multi-agent reinforcement learning system by treating it as an event-triggered control problem. This method uses so-called robustness surrogates as an equivalent to a Lyapunov function to determine if a communication can be skipped without decreasing the performance more than some tolerance. The second is a method to increase the observational robustness of a policy by using lexicographic reinforcement learning.

We aim to combine these ideas and trade the additional observational robustness for decreased communications. We also want to test whether additional observational robustness can help mitigate the sim-to-real gap. We implement this method for the multi-agent deep deterministic policy gradient algorithm and perform tests on a variant of the predator-prey domain in increasingly more realistic simulations.

We found that the combination of this robust policy and the robustness surrogates method does enable the agents to achieve the same return while communicating less. Unfortunately, our research shows that the observational robustness obtained using lexicographic reinforcement learning, does not help mitigate the sim-to-real gap.

Control engineering researchers are increasingly embracing data-driven techniques like reinforcement learning for control and optimisation. An example of a case where reinforcement learning could be useful is the synthesis of near-optimal sampling strategies for self-triggered control. Self-triggered control is an aperiodic control method that aims to reduce the number of communications between the controller and sensors in a control loop, by predicting when some triggering condition is met and only transmitting a sample accordingly. Recent research has shown that greedily following the proposed sampling times can result in sub-optimal long-term average inter-sample times. Abstraction-based methods have been able to synthesise sampling strategies that result in better long-term average inter-sample times, by allowing for early sampling and considering the proposed sampling times as deadlines. However, these abstraction-based methods suffer greatly from the curse of dimensionality in the form of combinatorial explosion, which limits their practicality for more complex systems. This thesis proposes a novel deep reinforcement learning tool for finding near-optimal sampling strategies for self-triggered control of LTI systems. The proposed tool is evaluated and compared to a state-of-the-art abstraction-based method. The proposed tool is shown to match the performance of the abstraction-based method for smaller systems, while still achieving good results on more complex systems that prohibit the use of abstraction-based methods.

Over the last years advantages in autonomous agent systems and technology have created many potential applications for large numbers of collaborating robots in the field of surveillance, mapping, mining, farming and (space) exploration. The underlying principle that enables robots to collectively solve complex tasks is that of minimal interference: the basis of swarm robotics. In nature, swarms of insects use stigmergy, communication through environment marking, to connect individual and collective behavior. Many have tried to implement this stigmergic principle in swarm robotics, though it remains a challenge to implement stigmergy in robotic systems suited for real applications. In this thesis we present a biologically inspired minimalistic design for a foraging robotic swarm based on stigmergy. Our self-guiding swarm requires only very restricted robot capabilities: Robots do not require global or relative position measurements; the swarm is fully decentralized; and the robots need no infrastructure in place. Additionally, the system only requires one-hop communication over the robot network, we do not make any assumptions about the connectivity of the communication graph and the transmission of information and computation is scalable versus the number of agents. All this is realized by letting the agents in the swarm act as foragers or as guiding (beacons). We analyse the characteristics and performance of our swarm by our own developed so called 'particle'-simulator and the realistic \textit{Webots} simulator using a swarm of Elisa-3 robots. We show how the swarm self-organizes to solve a foraging problem over an unknown environment and converges to trajectories around the shortest path. In addition, we study directions of future improvements of the swarm design regarding the minimization of resources, optimality of the created paths and the convergence speed. At last, we derive formal results regarding the reachability, coverage time and hitting times of the swarm.

Persistent surveillance is the act of covering an environment persistently, as fast as possible. By exploiting the intelligence of the swarm, it is possible to create a swarm robotic persistent surveillance method that can deal with unknown dynamic environments without the need for complex computations or excessive storage. Using stigmergy, which is communication via the environment, robots drop pheromones to signal that a specific location is covered. Other robots sense these pheromones and avoid going there. This is, in essence, the inverted stigmergic behaviour of ants. While ants deploy pheromones to attract other ants to that location, our robots repel other robots by deploying pheromones.
This thesis proposes a stigmergic swarm robotic persistent surveillance method that can deal with unknown environments and dynamic obstacles. Stigmergy is used as the sole communication mean. Additionally, an extension is included in the model that renders the model more efficient with respect to the pheromone usage. Subsequently, to demonstrate the potential for real-life application, the model is simulated in Webots, an open-source 3-D Robot simulator.
Concludingly, this thesis demonstrates that stigmergy lends itself perfectly for persistent surveillance. It minimizes the computations and memory storage needed, while ensuring performance. The proposed model outperforms current literature and deploys pheromones more efficient. The model is inherently robust and flexible.