Accurate sensor calibration is a critical challenge in the development of automated vehicles, especially in dynamic and modular sensor configurations. Traditional target-based methods, while precise, are limited in scalability and adaptability. In this work, we propose a modular, targetless, ego-motion-based calibration framework for multi-modal sensors, including a monocular camera, LiDAR, and 4D radar. The framework leverages odometry trajectories for extrinsic calibration, incorporating temporal alignment, trajectory scaling, and both pairwise and joint optimization techniques to achieve robust and accurate sensor alignment. Experimental validation using the View-of-Delft (VoD) dataset demonstrates the framework’s
robustness across diverse sensor setups, adaptability to real-world conditions. Our results underscore the potential of scalable, targetless calibration approaches to enhance the reliability and flexibility of automated systems, supporting implementation in real-world scenarios.

3D semantic understanding is essential for a wide range of robotics applications. Availability of datasets is a strong driver for research, and whilst obtaining unlabeled data is straightforward, manually annotating this data with semantic labels is time-consuming and costly. Recently, foundation models have facilitated open-set semantic segmentation, potentially aiding automatic labeling. However, these models have largely been limited to 2D images. This work introduces Label Any Pointcloud (LeAP), which leverages 2D Vision Foundation Models (VFMs) to automatically label 3D data with any set of classes in any kind of application. VFMs are used to create image labels for the desired classes which are then projected to 3D points. Using the Bayesian update, point-wise labels are combined into voxels to improve label consistency, and label points outside the camera fustrum. A novel Cross-Modal Self-Training (CM-ST) approach further enhances label quality. Through extensive experiments, we demonstrate that our method can generate high-quality 3D semantic labels across diverse fields without any manual 3D labeling. Models adapted to new application domains using our labels show up to 3.7× (12.9 → 47.1) mIoU improvement compared to the unadapted baselines. This ability to provide labels for any domain can help accelerate 3D perception research.

Automotive radars are increasingly used in automated driving systems due to their cost effectiveness, ease of integration, and ability to withstand adverse weather conditions. Semantic segmentation for radar point clouds is a crucial step in radar pre-processing, which can be used on almost all downstream tasks of radars, such as detection and tracking. Radar point clouds are noisier compared to LiDAR point clouds due to sensor noise and the multi-path propagation, which makes the segmentation task for radar more challenging. In this paper, we address the problem of segmentation in noisy radar point clouds in terms of ghost target vs. real detection, moving vs. static objects as well as semantic segmentation of moving road users. We demonstrate how these three tasks can be performed in a single, unified pipeline using an auto-labeled radar dataset. Our approach, called Real, Moving, and Semantic Segmentation Network (RMSNet), is able to output point-wise labels for all three tasks simultaneously. On our dataset, RMSNet attains an IoU of 82.5% for real detection segmentation, 66.9% IoU for moving object segmentation, and 64.9% mIoU for semantic segmentation. We also did a live demonstration, using the model on two different radars. The result shows that the model is capable of running inference in real-time and demonstrates good ability in model generalization.

Autonomous driving systems require robust and reliable perception across diverse environmental conditions, yet current approaches to 3D semantic occupancy prediction face challenges in adverse weather and lighting. In this paper, we present the first study on fusing 4D radar and camera for 3D semantic occupancy prediction. This fusion offers significant advantages for robust and accurate perception as 4D radar provides reliable range, velocity, and angle information, even in challening conditions, complementing rich semantic and texture details from cameras. Additionally, we demonstrate that incorporating depth cues with camera image pixels supports the process of lifting 2D images to 3D, enhancing the accuracy of scene reconstruction. Secondly, we introduce a fully automatically labeled dataset specifically designed for training semantic occupancy models, demonstrating its ability to substantially reduce the need for costly manual annotation. Our results highlight the robustness of 4D radar in a wide range of challenging scenarios, showcasing its potential to advance perception for autonomous vehicles.