In the context of open-world scenarios in autonomous vehicles (AVs), previously unseen classes may arise. To address this, effective extraction of well generalizable features is essential for AV downstream tasks, especially in the context of zero-shot learning. This can be achieved using transformers, and Swin Transformer in particular (vision backbone of most Vision-Language Models). However, to enable on-board applications, knowledge distillation must be utilized to create a lightweight model capable of real-time processing. We explore self-supervised knowledge distillation, given that AV datasets need to generalize to previously unseen classes. Our contributions include adapting existing CNN-to-CNN output-based self-supervised knowledge distillation algorithms to Transformer-to-CNN for benchmarking and enhancing them with a cross-architecture loss function. By leveraging DisCo, the best performing output-based self-supervised knowledge distillation method, and using EfficientNetB0 as the student model, we achieve a 3.9% relative improvement in top-1 accuracy over the supervised Swin-T teacher on our modified ImageNet for open-world classification, up to 5.0% with our loss.

In this thesis, an autonomous multi-robot system for indoor exploration in limited network environments is proposed. The specific use case is search and rescue where the operators must have access to the most up-to-date information, necessitating the requirement for communication maintenance. This requirement is satisfied in a novel way by using signal strength measurements collected by the robots to update the network model, thus reducing the overly conservative nature of current approaches. The network model chosen is an adaptation of existing path loss models as they balance complexity and performance given the information available during multi-robot exploration.

The proposed system consists of a central server with multiple robots which perform tasks that are readily distributable, whereas the central server performs complex iterative optimizations such as merging the local maps and running task assignment. While fully distributed architectures offer theoretical benefits such as scalability and robustness, in practice, these benefits are currently not achieved due to the complexity of splitting iterative optimizations such as mapping and task assignment among multiple nodes. This results in distributed architectures requiring more time and bandwidth to solve these problems, which is why this work implements a central architecture. While iterative optimizations are more efficient when performed centrally, scalability is limited due to the branching of possible options during task assignment. To combat this, an action space formulation, called an action graph, is proposed that is unique to each robot and reduces the number of actions by merging similar ones.

Both the dynamic network prediction model and the action graph formulation are shown to hold promise by experimentation. However, more work is needed before real-world use is feasible.

4D millimeter-wave radar is increasingly important in advanced driver-assistance systems due to its ability to capture Doppler/velocity information and robustness in low-light or adverse weather conditions. Unlike traditional 3D radar, 4D radar provides elevation information, enhancing 3D spatial perception. Most of the perception tasks using 4D radar tensors focus on target classification using bounding boxes. In contrast, although semantic segmentation is typically used to process images and LiDAR point clouds, it has not been well explored for 4D radar tensors.

This MSc thesis aims to bridge the gap in using 4D radar tensors for semantic segmentation and in generating the required labels for supervision. Specifically, it proposes an automatic approach for generating multi-class point-wise labels for automotive datasets by leveraging the complementary information from the synchronized camera and LiDAR data. Then, 4D radar tensors are used as inputs, supervised by the generated labels, to design a radar semantic segmentation network. Promising results are shown by applying both developed parts to the publicly shared RaDelft dataset. The automatic labeling process demonstrates satisfactory quantitative and qualitative results compared with manual labeling results obtained on randomly chosen scenes. The outputs of the semantic segmentation network achieve more than 65% in overall detection probability, improving by +13.1% in terms of vehicle class detection probability, and reducing 0.54 m in terms of Chamfer distance compared to the variants inspired by the literature.

Detecting nearby vehicles involves utilizing data from various sensors installed on a car as it moves. Common sensors for identifying nearby vehicles include LiDAR, cameras, and RADAR. However, all of these sensors suffer from the same issue -- they cannot detect an approaching vehicle that is not yet visible. Hence, this thesis explores the potential of using a microphone array -- an array of sensors capable of detecting vehicles that are out of sight. Exploring prior research on detecting obstructed vehicles using sound reveals an existing model capable of detecting nearby vehicles approaching from behind blind corners. However, as the local geometry around the ego vehicle affects the perceived sound patterns, this model was only designed to work within a specific set of T-junctions. Therefore, the thesis aims to take a step further and develop a detection model capable of detecting vehicles behind blind corners in environments not included in the training set of the deployed model. This is challenging for multiple reasons. First, literature review revealed a lack of suitable datasets comprising sounds from approaching vehicles behind blind corners within various road junctions. In addition, microphones, like other sensors, come with limitations. Sound inherently provides less spatial information compared to commonly used sensors in autonomous driving, such as LiDAR or cameras. Considering sound propagation variations in different road junction geometries, building a model adaptable across diverse junction types presents a challenge. To overcome the data scarcity and sound's inherent spatial limitations, the study investigates the potential of employing simulated acoustic responses within artificial road environments as training data for real-world vehicle detection. Simultaneously, to complement the sounds inherent advantage of detecting objects that are out of sight, the thesis proposes to use a Bird's Eye View (BEV) encoding of the top-down map from the driving vehicle's perspective. Having an encoding of the top-down map of the current driving environment would allow a detection model to expect sound signatures commonly observed within a given setting. Overall, the assessment of acoustic simulations could not outline a singular configuration of simulation properties allowing realistic sound propagation for any kind of considered junctions when hearing an approaching vehicle. However, it was observed that the utilization of specific simulation parameters can result in realistic sound propagation within the given junction. Subsequently, evaluating a novel BEV encoding within the newly proposed acoustic detection pipeline demonstrated either equivalent or superior performance compared to a model relying solely on sound. Overall, this research underscores the potential of incorporating BEV encoding in non-line-of-sight acoustic detection and suggests the promise of acoustic simulations within the field. This study contributes to advancing the integration of sound as an additional data modality in vehicle detection.

A scenario-based testing approach can reduce the time required to obtain statistically significant evidence of the safety of Automated Driving Systems (ADS). Identifying these scenarios in an automated manner is a challenging task. Most methods on scenario classification do not work for complex scenarios with diverse environments (highways, urban) and interaction with other traffic agents. This is mirrored in their approaches which model an individual vehicle in relation to its environment, but neglect the interaction between multiple vehicles (e.g. cut-ins, stationary lead vehicle). Furthermore, existing datasets lack diversity and do not have per-frame annotations to accurately learn the start and end time of a scenario. We propose a method for complex traffic scenario classification that is able to model the interaction of a vehicle with the environment, as well as other agents. We use Graph Convolutional Networks to model spatial and temporal aspects of these scenarios. Expanding the nuScenes and Argoverse 2 driving datasets, we introduce a scenario-labeled dataset, which covers different driving environments and is annotated per frame. Training our method on this dataset, we present a promising baseline for future research on perframe complex scenario classification

VPR describes a task where an agent (e.g., a robot) attempts to recognize its current location by comparing the incoming visual data from its sensor(s) (query images), usually a camera, to geotagged reference images. Both query and reference images are described using a feature extractor, and the query descriptor is matched to its closest reference descriptor in the feature space. Within VPR there are many different VPR techniques that have been proposed throughout the years with many different types of architecture and trained on different datasets. With the many test datasets available, there exists no VPR technique that is able to reach state-of-the-art performance on all these datasets. For this reason, existing work has argued it can be beneficial to utilize an ensemble-based method to combine multiple VPR techniques and achieve better VPR performance. Some of these Ensemble-based methods have already been proposed. These ensemble-methods combine individual VPR techniques and weigh their predictions using these same predictions to give an indication of their confidence. This calculation, however, is strictly based on predictions obtained from applying the VPR techniques on test data at inference time. Generally within VPR research, the dataset that was used to train the VPR technique is often different from the dataset it is tested on. This means there is a domain shift between the training and test data. This domain shift is not taken into account when weighting the predictions of VPR techniques in an ensemble using these existing methods. In this work, we analyze how this degree of domain shift between train and test data, which can be observed by looking at the relative location of descriptors in the feature space, impacts downstream VPR performance. Intuitively, one would expect better VPR performance in a situation where the degree of train-test domain shift is minimal. Our analysis shows that this is indeed the case. We propose two different methods that utilize this degree of domain shift to calculate the weights given to the VPR techniques in an ensemble. First, we propose a generative method. Here weights are given to the VPR techniques based on the likelihood that the query sample originated from the same distribution as the training dataset of the technique and is in distribution. This way each individual technique is given a weight. Secondly, we propose a discriminative method. Here weights are given to the training datasets used to train the techniques in the ensemble. These training datasets are given weights based on relative proximity to a query sample in the feature space, an indicator for the degree of domain-shift between the training dataset and the query sample. all VPR techniques are given the weight corresponding to their training dataset. We compare these proposed approaches to other ensemble-based baselines and individual VPR techniques. The quantitative results show that our proposed methods generally outperform the ensemblebased baselines and the individual VPR techniques. We also propose further future work. One of the generative methods still delivers lower performance than could be possible, caused by applying this method to high-dimensionality descriptors. A solution for this issue should lead to higher VPR performance using this method. Additionally, we suggest future work to expand on the datasets used in this research, to strengthen the claims made, and verify that results and trends found to hold up when testing and training using other datasets.

An event-based camera enables capturing a video at a high temporal resolution, high dynamical range, reduced power consumption and minimal data bandwidth while the camera has minimal physical dimensions compared to a frame-based camera with the same vision properties. The limiting factor, however, of an event-based camera is the spatial resolution which ranges between 40 × 40 and 1280 × 960. To counter this deficiency, a method is researched to super resolve event-based vision in order to enhance spatial resolution. A selection of different neural network types and configurations are researched in a step-by-step fashion. Subsequent experiments tested the selected networks on their ability to process event-based data and extract features from it. Followed by experiments that exploited the limitations of the networks to super resolve at different ratios, lengths of eventstreams and more complex event-based data. Results of various experiments showed that a network configuration that utilizes a transformer architecture was best able to super resolve event-based vision. This type of network leverages the ability to extract features based on dependencies between events which aligns with the characteristics of event- based vision. Based on the obtained results from the exper- iments, a pipeline is proposed to super resolve event-based vision and consists of a combination of a transformer network, multilayer perceptrons and a k-nearest-neighbor algorithm. Using this pipeline, eventstreams can be super resolved in the spatial resolution at a scaling ratio of 4. Visually, these super resolved eventstreams resemble more detailed and enhanced version to the low-resolution input. This proposed pipeline can be considered as a starting point in further research toward the super-resolution of event-based data and thereby contributes to the extension of application possibilities of event-based vision.

Adaptive Cruise Control (ACC) relieves human drivers’ tasks by taking over the control of the throttle and braking of the vehicles automatically. However, it has been demonstrated in many empirical studies that current production ACC systems fail to guarantee string stability. It is believed that if vehicles can take the longitudinal dynamics further downstream into account and react to the propagating disturbance earlier, the string stability in the platoon may be improved. Instead of relying on inter-vehicle communication technologies, the ego-vehicle should be able to detect the second leading vehicle by leveraging the power of on-board sensors. Still, the second leader measurements can be highly erroneous. Therefore, it is important to consider the entailed measurement uncertainties when designing and evaluating such ACC systems. This study proposes several ACC systems which possess the property of multi-anticipation and uncertainty handling.

The possible sensor technology which can collect the second leader measurements is first investigated. Based on the considered setup, the measurement uncertainties are modelled to reflect the real-world conditions. The ACC system architecture and control system design method are then proposed. Deep reinforcement learning is applied for the controller design in light of its great potential in describing the complex non-linear control task and handling the uncertainties. Kalman filters and recurrent policies with a Long-Short-Term-Memory network are applied to cope with uncertain measurements. The first method estimates the state information before feeding it back to the controller agent, while the latter incorporates the state estimator into the controller to actively consider the uncertainties while making decisions.

A numerical simulation approach is adopted to theoretically assess the performance of the proposed ACC systems. A traffic disturbance event and multiple levels of measurement noise are considered in the experiment. To analyze the performance in terms of string stability and ride comfort and understand the car-following behavior mechanism resulted from the proposed systems, a quantitative analysis framework is developed.

The evaluation results demonstrate the applied learning-based approach succeeds to train ACC control policies which can ensure string stability. It is also found that the multi-anticipation ability significantly improves the string stability and ride comfort performance. In the scenarios with measurement noise, systems using the tuned Kalman filters exhibit the ideal level of string stability performance. However, ride comfort cannot be guaranteed in scenarios with large measurement noise. On the other hand, systems using recurrent policies can better ensure ride comfort performance while maintaining string stability at certain levels. Based on the results, the performance limits of the proposed ACC systems in the handling of measurement uncertainties are explored. In addition, with the different policy training setups, the trade-off between these two performance aspects is shown.

The findings of this study are anticipated to trigger the development of advanced multi-leader ACC system by automakers, sensor manufacturers, and traffic engineers. Future work can be directed to an enhanced controller design. Robustness of the systems with respect to other sources of measurement uncertainties, more types of traffic disturbance, and platoon heterogeneity is worth further design consideration and analysis.

Drone detection and tracking systems are nowadays a requirement in most public, private and political events, because of the increasing risk of unintentional or malicious misuse of these platforms. Moreover, in order to ensure adequate protection, full spatial coverage is a must for every such system. However, the research literature focuses on staring radars that have a limited field of view, but which yield rich target information via time-frequency distributions that facilitate the target recognition task. In this thesis, surveillance radars that offer full spatial coverage are presented, albeit their usage for classification is made more complex because of the rotating nature of their antennas which limits the dwell time on targets.

Additionally, due to the incredible fast growth of the drone market, novel counter-drone radars that are able to jointly localize and classify small targets while on-the-move now represent a highly in-demand remote sensing system. Nonetheless, surveillance sensors anchored on moving vehicles are a brand-new technology that is currently being developed. This work therefore investigates surveillance systems in a novel scenario, and presents the technological challenges alongside the proposed solutions to achieve reliable object detection via grounded counter-drone radars on-the-move. Specifically, the required pre-processing steps to remove the clutter from the data while the radar is rotating and moving on the ground are developed and discussed.

In the end, the joint detection and classification problem is traditionally solved separately by different algorithms due to the computational complexity of the task. This thesis project presents a novel framework that localizes and labels drones in an unified pipeline under the umbrella of object detection via computer vision, and that is able to operate while being static or on-the-move. Thus, an end-to-end radar data processing architecture that is robust against homogeneity constraints and based on You Only Look Once (YOLO) model is used to perform object detection in real-time. In brief, this work opens new avenues towards multi-class and multi-instance plot-based target detection and classification by transferring cross-disciplinary algorithms from computer vision into remote sensing.

Knowledge Distillation (KD) is a well-known training paradigm in deep neural networks where knowledge acquired by a large teacher model is transferred to a small student. KD has proven to be an effective technique to significantly improve the student's performance for various tasks including object detection. As such, KD techniques mostly rely on guidance at the intermediate feature level, which is typically implemented by minimizing an L_p-norm distance between teacher and student activations during training. In this paper, we propose a replacement for the pixel-wise independent L_p-norm based on structural similarity (SSIM). By taking into account additional contrast and structural cues, more knowledge within intermediate feature maps can be preserved. Extensive experiments on MSCOCO demonstrate the effectiveness of our method across different training schemes and architectures.
Our method adds only little computational overhead, is straightforward to implement and at the same time it significantly outperforms the standard L_p-norms. Moreover, more complex state-of-the-art KD methods using attention-based sampling mechanisms are outperformed, including a +3.5 AP gain using a Faster R-CNN R-50 compared to a vanilla model.

Pedestrian trajectory prediction is essential for developing safe autonomous driving systems. Such trajectories depend on various contextual cues, among which surrounding objects.

This work proposes the first pedestrian trajectory prediction method in the 2D on-board domain that models interactions between the pedestrian and surrounding static- and dynamic- contextual objects using a graph-based approach. Our two-stream model separately encodes past motion history and interactions. The encoded information from both streams is fused and decoded to generate future pedestrian trajectories. The interactions are modeled using spatial graphs, which are temporally connected using a Gated Recurrent Unit. The graph nodes represent the pedestrian and contextual objects, and the edges represent the interaction importance between nodes.

In experiments on the PIE and JAAD_full dataset, it is shown that our graph-based interaction-aware trajectory prediction method outperforms all considered baselines on nearly all metrics. Moreover, the performance gain on JAAD_full is most significant for the close-by pedestrians. Finally, modeling the interactions with all considered contextual objects, i.e. vehicles, crosswalks, and traffic lights, improves trajectory prediction performance most compared to only using a subset of these objects.

This thesis proposes a novel algorithm, Wi-Closure, to improve computational efficiency and robustness of map matching in multi-robot SLAM. Current state-of-the-art techniques connect maps with inter-robot loop closures, that are usually found through place recognition. Wi-Closure decreases the computational overhead of these approaches by pruning the search space of potential loop closures, prior to evaluation by a typical place recognition algorithm. Wi-Closure achieves this by identifying where trajectories are close to each other through sensing spatial information directly from the wireless communication signal. Then, place recognition is only performed on scans taken at locations close to each other. Wireless sensing provides information even when operating in non-line-of-sight or without existing communication infrastructure. The validity of Wi-Closure is demonstrated in simulation and hardware experiments. Results show that using Wi-closure greatly reduces computation time, by 54% in simulation and by 77% in hardware, compared with a multi-robot SLAM baseline. Importantly, this is achieved without sacrificing accuracy. Using Wi-Closure reduces absolute trajectory estimation error by 99% in simulation and 89% in hardware experiments. This improvement is due in part to Wi-Closure’s ability to avoid catastrophic optimization failure that typically occurs with classical approaches in challenging repetitive environments.

This thesis research proposes a new method for a controlling an agricultural robot using computer vision. The robot has to follow and simultaneously reel in a hose, which lies on a grass field. The hose that has to be followed, is attached to the robot itself. The trajectory of the hose is captured by a monocular camera and is extracted from the image by using a vision pipeline. The vision pipeline consists out of semantic segmenting in combination with a clustering algorithm and polynomial regression to find the trajectory of the hose. This trajectory is used as an input for the PID controller to control the motion of the robot. The presented method of controlling an agricultural robot by using vision control is tested and validated and performs sufficient, when the robot drives with a velocity of 0.2 m/s and when the hose is not partial occluded by long grass.

Underwater position estimation is challenging due to the absence of Global Navigation Satellite System (GNSS) signals. Underwater vehicles are typically equipped with a Doppler Velocity Log (DVL) that measures the velocity relative to the seafloor. Aside from the velocity, the DVL also measures the range of each of the four beams. When compared against a bathymetry height map, these measured ranges provide additional information enabling improving position estimates. Unfortunately, surveys often occur in areas where detailed bathymetry maps are unavailable. In these cases, bathymetry Simultaneous Localization and Mapping (SLAM) could be used to improve the position estimates compared to the velocity integration position. With SLAM, the map is being estimated during the mission while at the same time using the map for position determination.

In this thesis, reduced rank Gaussian processes (GPs) are used as map representation for SLAM. The downside of regular GPs is a time complexity of O(n^3) , reduced rank GPs improve the computation performance. GPs provide Gaussian distributions at any point, leading to a neat integration with probabilistic SLAM algorithms. To the best of the author’s knowledge, this has not been used in bathymetry SLAM. This report investigates how reduced rank approximated GPs, representing the bathymetry, can be integrated into a SLAM algorithm to improve the position estimates of an underwater vehicle equipped with a DVL and low-quality gyroscopes.

A squared exponential kernel is used as GPs model of the bathymetry. The reduced rank approximation is vital for real-time SLAM performance. This map representation is integrated with a Rao Blackwellized particle filter (RBPF) that estimates both the underwater vehicle’s trajectory and the bathymetry map.

The SLAM algorithm is evaluated using data from an underwater vehicle operated at the surface such that a GNSS reference position is available. Experiments of the SLAM algorithm show a reduced position error compared to the GNSS reference. The resulting algorithm has a computation time of up to 30 times faster than the Autonomous Underwater Vehicle (AUV) collects data while improving position estimates. This concludes that the RBPF using reduced rank GPs is capable of onboard improved position estimation on underwater vehicles.

Lane detection represents a fundamental task for automated/autonomous vehicles. Current lane detection methods do not provide the versatility of real-time performance, robustness,and accuracy required for real-world scenarios. The reasons include lack of computing power while being portable and inability to observe the continuity and structure of lane lines over a sequence of images. An investigation into the present methods in the literature reveals that deep learning networks cannot focus on relevant images and critical parts of the images. The neural networks implemented with max-pooling operations can cause a loss of information at a granular level of the image during the downsampling of images. Obtaining a fixed set of lane locations will restrict the number of lane lines detected. It hinders the generalisability of the network. This thesis aims to introduce a novel spatio temporal method that can focus on lane lines and key features to increase the robustness and accuracy of lane line detection.The spatio temporal attention network based on Long Short Term Memory (LSTM) units tested on the tvtLane dataset provided an accuracy of 98.1443%, the precision of 0.8873, and F1-score of 0.9108. The precision and F1-measure are the highest when compared to state-of-the-art lane detection networks. The spatio temporal FC attention network produces better accuracy and precision on TuSimple dataset than state-of-the-art networks with 98.2078% and 0.8861, respectively. Testing on the LLAMAS dataset, the network achieved an average precision of 0.8028 and corner recall of 0.7183. The results show the robustness and high accuracy on two different datasets with unique distributions. Although the network is trained on datasets with a maximum of five lanes, it can detect more than five lanes in an image.The network is also able to detect lanes on the unseen Netherlands dataset.

Suction based robotic actuators have potential for the bin-picking industry, but are currently not usable due the needed speed, accuracy and ability to handle novel and adversarial objects. An evaluation of the state of the art grasp pipeline developed by Mahler et al. [1] for detecting grasps on novel objects lead us to split the problem of robotic grasp generation into a global and a local component. The state of the art solution had the ability to fill the role of global evaluator leaving a local evaluator to be developed. This local grasp evaluator was obtained by training a neural network on a suction based grasp dataset, which was created using a newly developed annotation tool. The proposed grasp pipeline obtained by combining these two showed a 95.27% pick success rate for a random setup and a 90.28% success rate for solely novel objects.

3D human pose estimation is a widely researched computer vision task that could be applied in scenarios such as virtual reality and human-robot interaction. With the lack of depth information, 3D estimation from monocular images is an inherently ambiguous problem. On top of that, unrealistic human poses have been overlooked in the majority of papers since joint detection is the only focus. Our work consists of two parts, an end-to-end 2D-3D lifting pipeline and a novel kinematic human model integrated approach. We start with Pose Estimation using TRansformer (PETR), an approach that does not require temporal information and has the attention mechanism to model the inter-joint relationship from RGB images. In the approach with human model, we emphasize pose similarity rather than focusing on joint detection. We propose a new metric, called Mean Per Bone Vector Error (MPBVE), that evaluates poses regardless of a human body’s gender, weight, or age. We introduce Pose Estimation on Bone Rotation using Transformer (PEBRT), a novel approach that regresses rotation matrices for 16 human bones, assuming labeled 2D poses as input. Our human model encapsulates joint angle and bone length constraints. Existing methods treat these constraints as an additional loss term, which does not guarantee realistic final out- puts. Our method does not require temporal information or receptive fields to generate kinematically realistic human poses. We demonstrate that PEBRT is capable of delivering comparable results on Human3.6M to existing methods.

Least Squares Support Vector Machines (LS-SVMs) are state-of-the-art learning algorithms that have been widely used for pattern recognition. The solution for an LS-SVM is found by solving a system of linear equations, which involves the computational complexity of O(N^3). When datasets get larger, solving LS-SVM problems with standard methods becomes burdensome or even unfeasible. The Tensor Train (TT) decomposition provides an approach to representing data in highly compressed formats without loss of accuracy. By converting vectors and matrices in the TT format, the storage and computational requirements can be greatly reduced. In this thesis, we develop a Bayesian learning method in the TT format to solve large-scale LS-SVM problems, which involves the computation of a matrix inverse. This method allows us to include the information we know about the model parameters in the prior distribution. As a result, we are able to obtain a probability distribution of the parameters, which enables us to construct confidence levels of the predictions. In the numerical experiment, we show that the developed method performs competitively with the current methods.

A common problem in robotics is the simultaneous localization and mapping (SLAM) problem. Here, a robot needs to create a map of its surroundings while simultaneously localizing itself in this map. An unknown environment is assumed. Traditionally, it has been approached through filtering solutions. This paradigm has shifted to pose graph optimization (PGO). This method scales well with large maps and is fast and accurate. Furthermore, it is especially suited to the distributed SLAM problem as existing distributed optimization methods can be leveraged. One such method is the alternating direction method of multipliers (ADMM), which has been used in distributed PGO. ADMM has a simple implementation and can achieve high accuracies in distributed PGO. A solution of good quality can be acquired in a few iterations with ADMM. However, ADMM is slow to converge to high accuracies. This thesis introduces an algorithm which implements Nesterov acceleration in the ADMM algorithm for distributed PGO in SLAM problems. Such an implementation will be novel. To create the proposed Nesterov accelerated ADMM (N-ADMM) algorithm, the current literature is adapted and extended based on the choices made in this thesis. The main research question is how to make these choices. The proposed N-ADMM algorithm is implemented in C++ and compared with unaccelerated ADMM and the state of the art. N-ADMM has shown better performance in some scenarios. To further research what these scenarios are characterized by, two models are introduced to create new datasets of which the parameters can be controlled. The effects of graph size and bad initial guesses are investigated.

Learning from demonstration is a technique where the robot learns directly from humans. It can be beneficial to learn from humans directly because humans can easily demonstrate complex behaviors without being experts in demonstrating required tasks. However, it can be challenging to gather large amounts of data from humans because humans often get tired, bored, or lose focus while giving many demonstrations. Therefore, the learning algorithms require to be as data-efficient as possible. To solve sequential tasks using few human demonstrations, the Transporter Network was introduced.

The Transporter Network consists of two networks, namely the pick network and the place network. Both these networks have a fully convolutional architecture which has the property of being translationally equivariant. Translational equivariance is well suited for estimating pick locations because the predicted pick location changes corresponding to the change in the object's position in the image. However, the performance of the transporter network on sequential tasks depends on the receptive field of the fully convolutional network because each pixel in the output should depend on each pixel in the input image. This correlation is important for the network to make predictions based on the overall configuration of objects in the input image. A larger receptive field would give a prediction by correlating large amount of pixels in the input image, however increasing the receptive field could result in loss of resolution which in turn leads to information loss in the latent space. Therefore, it is essential to select an appropriate size for the receptive field such that there is minimal loss of resolution in the latent space of the fully convolutional network. This limitation of selecting the appropriate size of the receptive field was also observed in the preliminary experiments on the pick network of the Transporter Network.

In this work, a SEquential Attention Network (SEA Net) is introduced to overcome the dependence of the Transporter Network on the receptive field size for solving sequential task. SEA Net is a variation of the pick network of the Transporter Network. SEA Net works under the assumption that the set of objects to be picked are known beforehand. Based on this assumption, the SEA Net learns to predict two things: 1) The pick location of all the objects. 2) "what" object to pick as a function of the current configuration of the objects in the environment. SEA Nets can be used independently to predict pick location sequentially and also be used as a replacement of the pick network in the Transporter Network. SEA Net is evaluated on two datasets, namely synthetic dataset and simulated robot dataset. The synthetic dataset has a top-down view of simple shape blocks. In contrast, simulated robot datasets are extracted using a pre-trained policy on the Mujoco simulators. The Object Keypoint Similarity (OKS) metrics is used to score the distance between the predicted pick point and the ground truth. The OKS Metric uses a standard deviation threshold (σ(threshold)). A strict standard deviation threshold (σ(threshold)) of 2 pixels and a lenient threshold (σ(threshold)) of 20 pixels is used because a strict threshold will evaluate the resolution of the prediction. In contrast, the lenient threshold will assess if the predicted pick point is, at least on the desired object. The model achieved an overall accuracy of 64% for a strict standard deviation threshold (σ(threshold)) of 2 pixels and accuracy of 85% for a lenient standard deviation threshold (σ(threshold)) of 20 pixels on the OKS metrics, on the synthetic dataset. The model could perform on a simulated robot dataset with an accuracy of 82% on a strict standard deviation threshold (σ(threshold)) of 2 pixels.