Critical to the safe application of autonomous vehicles is the ability to accurately predict the future motion of agents surrounding the vehicle. This is especially important - and challenging - in urban traffic, where vehicles share the road with Vulnerable Road Users (VRUs) such as pedestrians and cyclists. However, the majority of the existing on-board prediction datasets focus on predicting future trajectories of vehicles. We therefore present the View-of-Delft Prediction dataset, an extension of the recently-released urban View-of-Delft (VoD) dataset. The proposed dataset contains a large proportion of VRUs and has a good class balance, consisting of 844 prediction scenarios in the city of Delft, with 228 prediction instances for vehicles, 159 for cyclists, and 444 for pedestrians in dense urban traffic. Since state-of-the-art trajectory prediction approaches are primarily developed on car-dominated traffic with little interaction with VRUs, we analyse if the same methodology is suitable for mixed-traffic urban environments with VRUs and vehicles in close proximity. As our baseline for this analysis, we select the graph-based PGP model, for which we propose the addition of encoding motion of surrounding cyclists separately to facilitate its application in dense urban traffic. Since PGP relies on the lane graph topology, we provide novel rich map annotations for the VoD dataset, including lane polylines. Our analysis shows that there is a significant domain gap between the vehicle-dominated nuScenes and VRU-dominated View-of-Delft Prediction datasets, as training only on nuScenes results in a 107.79% higher minADE10 on the VoD Prediction test set than training the model on VoD Prediction. Furthermore, we modify the model by adding target agent class information, to make it suited for multi-class trajectory prediction. Our analysis shows that this yields a significant performance improvement of 13.92% in minADE10 for a six-second prediction horizon. The View-of-Delft Prediction dataset will be publicly released, enabling novel research on urban mixed-traffic trajectory prediction.

For an Autonomous Vehicle (AV) to traverse safely in traffic, It is vital it can anticipate the behavior of surrounding traffic participants using motion prediction. Current motion prediction approaches can be categorized into object-centered and object-agnostic methods and are primarily based on deep learning. The former relies on a human-engineered pipeline of object detection and tracking, of which the errors can accumulate in the motion predictions. The latter does not rely on this pipeline; however, it lacks the ability to learn object representations causing blurriness and object disappearances for longer-term predictions which forms a safety hazard. This thesis proposes two methods to improve the performance of the object-agnostic sequence-to-sequence Occupancy Grid Map (OGM) prediction networks, trained on the Waymo Open Perception dataset. The first method uses inter-pixel loss functions, i.e. the SSIM and Sinkhorn losses, instead of the ubiquitously used per-pixel losses, to train the PredRNN++ Occupancy Grid Map (OGM) prediction network. Inter-pixel losses take into account the spatial relations between grid cells during the evaluation of OGMs, whereas per-pixel losses evaluate each grid cell’s value independently. The quantitative results demonstrate that using inter-pixel losses can improve short term predictions with a prediction horizon of T = 5 for the Mean Squared Error (MSE) by 4.3%, Image Similarity (IS) by 7.8%, Average Precision (AP) by 0.3%, metrics. For the longer term, T = 15, the predictions improve for the MSE by 5.0%, IS by 20.5%, AP by 0.1%, and Accuracy by 0.6%. Furthermore, the use of inter-pixel losses reduces blurriness and object disappearances. The second method is based on multi-task learning. By training the PredRNN++ to perform the prediction task together with the semantic segmentation task on the predicted OGMs, it is expected to learn object representations which it uses to improve the prediction quality. The quantitative results show that multi-task learning does not improve the OGM predictions. However, some qualitative results show that multi-task learning reduces blurriness and object disappearances.