We present a versatile NeRF-based simulator for testing autonomous driving (AD) software systems, designed with a focus on sensor-realistic closed-loop evaluation and the creation of safety-critical scenarios. The simulator learns from sequences of real-world driving sensor data and enables reconfigurations and renderings of new, unseen scenarios. In this work, we use our simulator to test the responses of AD models to safety-critical scenarios inspired by the European New Car Assessment Programme (Euro NCAP). Our evaluation reveals that, while state-of-the-art end-to-end planners excel in nominal driving scenarios in an open-loop setting, they exhibit critical flaws when navigating our safety-critical scenarios in a closed-loop setting. This highlights the need for advancements in the safety and real-world usability of end-to-end planners. By publicly releasing our simulator and scenarios as an easy-to-run evaluation suite, we invite the research community to explore, refine, and validate their AD models in controlled, yet highly configurable and challenging sensor-realistic environments.@en

This work addresses cross-view camera pose estimation, i.e., determining the 3-Degrees-of-Freedom camera pose of a given ground-level image w.r.t. an aerial image of the local area. We propose SliceMatch, which consists of ground and aerial feature extractors, feature aggregators, and a pose predictor. The feature extractors extract dense features from the ground and aerial images. Given a set of candidate camera poses, the feature aggregators construct a single ground descriptor and a set of pose-dependent aerial descriptors. Notably, our novel aerial feature aggregator has a cross-view attention module for ground-view guided aerial feature selection and utilizes the geometric projection of the ground camera's viewing frustum on the aerial image to pool features. The efficient construction of aerial descriptors is achieved using precomputed masks. SliceMatch is trained using contrastive learning and pose estimation is formulated as a similarity comparison between the ground descriptor and the aerial descriptors. Compared to the state-of-the-art, SliceMatch achieves a 19% lower median localization error on the VIGOR benchmark using the same VGG16 backbone at 150 frames per second, and a 50% lower error when using a ResNet50 backbone.@en

Motion prediction and planning are key components to enable autonomous driving. Although high definition (HD) maps provide important contextual information that constrains the action space of traffic participants, most approaches are not able to fully exploit this heterogeneous information. In this work, we enrich the existing road geometry of the popular nuScenes dataset and convert it into the open-source map framework Lanelet2. This allows easy access to the road topology and thus, enables the usage of (1) spatial semantic information, such as agents driving on intersecting roads and (2) map-generated anchor paths for target vehicles that can help to improve trajectory prediction performance. Further, we present DMAP, a simple, yet effective approach for diverse map-based anchor path generation and filtering. We show that combining DMAP with ground truth velocity profile information yields high-quality motion prediction results on nuScenes (MinADE5=1.09, MissRate5,2=0.18, Offroad rate=0.00). While it is obviously unfair to compare us against the state-of-the-art, it shows that our HD map accurately depicts the road geometry and topology. Future approaches can leverage this by focusing on data-driven sampling of map-based anchor paths and estimating velocity profiles. Moreover, our HD map can be used for map construction tasks and supplement perception. Code and data are made publicly available at https://felixhertlein.github.io/lanelet4nuscenes.@en