In this study, we perform human identification using accumulated radar point clouds in an outdoor scene. We employ PointNet as classification network and explore the impact of adding radars' non-spatial features as input, namely doppler velocity and radar cross section (RCS). Furthermore, we encode time as an additional time identity dimension to each point within the accumulated point cloud. We examine the effects of normalizing the RCS values, canonicalizing the spatial dimensions of the point cloud, as well as normalizing the doppler velocity with respect to this canonicalization. We examine three different PointNet configurations to understand the impact of the TransformNet blocks (T-Net) within the PointNet architecture on our six-dimensional radar data input. We have created a realistic outdoor dataset for training and evaluation purposes. Our approach of using the unnormalized six-dimensional radar data on the PointNet architecture without the two T-Net blocks achieves the highest performance of 73.4 % on our test set.

Multi-class road user detection using the next- generation, 3+1D (range, azimuth, elevation, and Doppler) radars has been shown feasible, thanks to the increased density of their point clouds and the inclusion of elevation information. However, object detection networks using LiDAR (64-layer) point clouds still dominate the performance metrics. In this work, we explore the potential of fusing a 3+1D radar point cloud and a monocular image to further close this performance gap in 3D object detection. We propose a generic and modular fusion architecture to extract both spatial and semantic cues from an RGB image to complement the radar point cloud. In a two-stage approach, we first generate a 3D point cloud representation of the input monocular image appended with semantic information through our proposed RAID (RAdar guided Instance-aware Depth) network, which takes monocular depth map and panoptic masks predicted from any pre-trained state-of-the-art networks, and a radar depth map as input. We then append the resulting point cloud to the 3+1D radar point cloud in a straightforward fusion scheme and train a point cloud-based object detection network. Results on the View-of-Delft dataset [1] show that our fusion approach significantly outperforms multiple state-of-the-art radar-camera fusion methods (proposed fusion vs. best baseline: 53.6 mAP vs. 50.8 mAP), and yields comparable performance to a network trained on LiDAR input when evaluated in the safety-critical driving corridor (80.5 mAP vs. 81.6 mAP).

In order to achieve redundancy and improve the robustness of an autonomous driving system, radar is a suitable choice for road user detection task in severe working conditions (e.g. darkness, bad weather). However, the real-time multi-class radar based road user detection algorithm is less explored compared with camera and LiDAR solutions. To fill this gap, the current thesis proposes a pipeline for radar based road user detection task, which is able to detect pedestrians, cyclists and cars by a single radar sweep. The pipeline effectively utilizes the advantages of radar low-level data by using it as region descriptor and combining it with radar high-level data for region proposal. A novel convolutional neural network structure called LLTnet is designed, in combination with proper pre-processing and post-processing stages. Ensemble learning is used to further improve the inter-class detection accuracy. The LLTnet itself performs radar targets segmentation. If needed, its output can be fused into object-level detection. To better train the network, a real-life dataset containing different moving road users is created during the study by a moving test vehicle, which simulates the real-life urban driving scenarios. After the network is trained, it is firstly evaluated by target-level metrics, such as the classification accuracy and F1 score. Then object-level metrics are used for object-level evaluation, such as the precision, recall and intersection over union (IoU).
Comprehensive experiments are performed which not only evaluate the performance of the proposed model but also test the importance of different stages and features, such as the importance of the ensemble learning and the validity of adding low-level data. The proposed pipeline improves the target-level F1 score from 0.59 of the baseline to 0.64 using LLTnet without ensemble learning. By adding the ensemble learning stage, the target-level F1 score is further improved from 0.64 to 0.70. The object-level recall of pedestrian class greatly improves from 0.37 to 0.68. The validity of adding low-level data to the algorithm is verified by bypassing the low-level data branch of the network. With only high-level branch, the target-level F1 score drops from 0.70 to 0.60. Furthermore, the trained model also shows good generalization ability on unseen data.