Arguably the main goal of artificial intelligence is to create agents that can collaborate with humans to achieve a shared goal. It has been shown that agents that assume their partner to be optimal can converge to protocols that humans do not understand. Taking human suboptimality into consideration is imperative to perform well in a coordination task. One way to achieve this is imitation learning, where you train an agent on recorded data from a human playing optimally. I created several agents using different implementations of behavioral cloning, by reducing the dataset to state-action pairs and training a neural network on this. To evaluate their performance, I used an environment that poses a coordination challenge based on the popular game Overcooked. Neither expanding nor reducing the feature space that the agents are trained on yielded any significant improvement in the performance of the agents. In fact, expanding the feature space to include some historical data made the agent less generalizable and especially failed to perform when paired with agents with unfamiliar strategies. These limitations were mostly posed by the available dataset, which was not big enough to support more features and of too low quality of gameplay to create agents that perform exceptionally well.

Overcooked, an immersive multiplayer video game centered around cooperative cooking challenges, provides the roots for this research project. The study focuses on designing and evaluating a hand-authored controller in comparison to controllers implemented using various machine learning techniques, such as Population Based Training, in the context of a simplified version of the game. The main objective is to assess the cooperation between the hand-authored controller and a human controlled agent, with a particular emphasis on the coordination and minimizing errors during runs of fixed time length. A testing method has been designed in order to more accurately evaluate the performance. Through the implementation of a specialized controller utilizing techniques such as Behavior Trees and Decision Trees, the controller was able to successfully accomplish the task of delivering soups. An analysis was made to examine the performance of the hand-authored controller in comparison to the results obtained in other research papers that use the same simplified version of the game. After the results have been obtained, a clear difference was visible. The scripted AI performed better when paired with himself than one created with population based training that was paired with himself. However, the scripted AI was not better than a coupled planning algorithm that was paired with himself, but the computations were easier and faster for the scripted AI. When paired with a human, the overall performance decreases, but increases only for one specific map.

Cooperative AI is AI designed to cooperate with humans. One example of such an AI, made using planning algorithms, was studied in a paper from 2019 which used a simplified version of the video game Overcooked for evaluation. However, only limited evaluations were possible due to the long runtime and heuristic optimisations made. This paper will attempt to increase the amount of evaluations while removing an important optimisation limiting the functionality of the AI: the omission of counters. In the end it will find ways of reducing the runtime and increasing performance, which together allow for the addition of counters in a few instances. The final results with counters suggest there is more to gain upon removal of a select few heuristic optimisations and improvement of the human behavioural clone used to simulate a human.

The popular video game "Overcooked" is a great example of a task requiring complex planning and cooperation with other players. This game is used as the inspiration for an environment for evaluating AI, called "Overcooked-AI". This paper implements a centralized critic into the Overcooked-AI environment's implementation of the PPO algorithm and compares the results with the decentralized critic approach when it comes to cooperation with human-like agents and computational efficiency. \\

The centralized critic approach gives similar results compared to the decentralized critic approach, both in self-play and when playing with human-like agents. This is probably due to the decentralized critic approach already having full access to the entire observation space, and no hyperparameter tuning being done due to a lack of time.

In ad-hoc cooperative environments, the usage of artificial intelligence to take supportive roles and work in collaboration with humans has proven to be of great benefit. The objective of this research is to evaluate the use of population-based training for reinforcement learning agents in a simplified version of the multiplayer game - Overcooked. The method used to answer that question involves evaluating the performance of the agents when paired with a human proxy and their learning curves on different layouts. Based on the employed method, it was concluded that both PBT and other self-play agents display notable underperformance when compared to human proxies and agents trained using human data. Moreover, while the inclusion of mutated agents enhanced sample efficiency in layouts with minimal collision risks, its effect on the final performance of PBT in those layouts was negligible. However, this approach managed to improve performance in layouts where collisions were the primary limiting factor.

Operation and maintenance of the built environment have a major effect on socioeconomic stability and sustainability. A significant part of our built world approaches or has well exceeded its designated structural life. As engineers, we need to find efficient ways to extend this life while maintaining acceptable levels of safety and performance. In this direction, smart and adaptive life-cycle inspection and maintenance planning are of paramount importance to reduce costs, increase structural reliability, and minimize resource-intensive interventions. Deep reinforcement learning provides a novel approach to strategize these decisions for systems subject to uncertainties and deterioration.

The goal of this project is to determine optimal life-cycle inspection and maintenance policies for road networks. Road maintenance and inspection planning is a complex task, involving a multitude of different objectives, such as the minimization of lifecycle cost and CO2 emissions (both from the maintenance works and vehicle emissions). However, current literature has underestimated the importance of optimizing for multiple objectives, and doesn’t consider the environmental impact during planning. In the current project, we aim to optimize the maintenance and inspection schedule with respect to minimizing the maintenance costs, carbon emissions and vehicle owners costs at the same time. To achieve that, a multi-objective road network environment will be modelled and multiple multi-objective Deep Reinforcement learning approaches will be applied and compared to traditional life-cycle management policies.

In the field of cooperative AI, an environment is created called Overcooked AI based on the popular Overcooked game. Originally the environment is used to study deep reinforcement learning, on the other hand it also allows for cooperative planning methods of which the paper will focus on. These methods include coupled based planning with replanning and model-based planning. This research paper attempts to reproduce the results the Overcooked AI environment developers obtained and to improve the Coupled Planning algorithm to gain higher results. In particular, experiments were performed against themselves and a human model for the planning methods, and an improved coupled planning algorithm, in which the failures are handled by deviating from optimal play, under different game steps. And a study on collision failures is performed. The results concluded that extrapolation of results are sub-optimal and that collision failures can be significantly reduced by handling collision differently; walking into the opposite direction.

Agents trained through single-agent reinforcement learning methods such as self-play can provide a good level of performance in multi-agent settings and even in fully cooperative environments. However, most of the time, training multiple agents together using single-agent self-play yields poor results as each agent tries to learn how to perform their task while their teammates are also learning. Thus, training models to reach an optimal behaviour in such situations becomes a challenging, if not impossible issue to overcome. One possible solution to deal with this problem is to facilitate a centralized training process in which the policies of all agents are evaluated by a centralized critic that has access to the observations and actions of all the agents in the environment. By using this approach, the environment becomes stationary and the agents learn in a similar way to using a single-agent algorithm in settings where only one agent needs to be trained. In this paper, we test whether by using a multi-agent reinforcement learning algorithm with centralized critics, as opposed to single-agent ones, we would obtain an agent that generalizes better to new partners in a collaborative environment such as Overcooked, where coordination is critical for good performance. The results display a similar performance between the two algorithms when evaluated through self-play and slightly better or worse results when paired with the human model, representing a mediocre agent, depending on the map. Thus, the multi-agent, centralized critics algorithm used in this study did not train agents that generalize better to new partners. However, the training metrics clearly indicate that the centralized critics method makes the agents learn and converge twice as fast as its single-agent version.

In an ad-hoc teamwork environment, artificial intelligence agents have the potential to take on supportive roles and complete tasks in collaboration with human players. The following paper investigates the use of employing population-based training (PBT) for reinforcement learning agents in the multi-player game Overcooked. In addition to this, the research examines whether the incorporation of highly mutated agents, which serve to introduce noise into the initial population, could enhance the final performance of PBT. As the method used to answer the previous inquiries, the learning curve of a selected PBT agent is first evaluated and its final performance with a human proxy then examined within different layouts of the game. Following this method, it was concluded that PBT, and other self-play agents, have the tendency to drastically underperform against the human proxy and agents that are trained based on human data. Furthermore, while incorporating the mutated agents increased sample efficiency in layouts with low risk of collisions, it had negligible effect on the final performance of PBT with the human proxy.

Most cooperative games are tackled by creating a team of agents who are optimised for each other and the problem. Creating an agent who can play in a variety of teams without any foreknowledge of its partner is a different challenge. These AI systems could useful for human-AI interaction as different people bring a lot of variance into the system. The training method synchronous K-level reasoning best response (SyKLRBR) tries to tackle this problem by creating agents based on grounded information. This research tested the potential of SyKLRBR in human-AI research evaluating the performance of its agent in the cooperative game Overcooked. The agents were able to obtain consistent scores against several unseen strategies, suggesting that SyKLRBR is able to create robust agents. Where SyKLRBR shows potential for medium cooperative settings this paper also discusses its great weakness for highly cooperative problems.