Angle-insensitive Human Motion and Posture Recognition Based on 2D FMCW MIMO Radar and Deep Learning Classifiers

Abstract

Nowadays, the aging problem is shaking the root of the healthcare system in many countries, an automatic human activity recognition (HAR) is seen as a promising solution to that problem. In particular, radar-based HAR attracts people’s attention thanks to its respect for privacy and functionality in poor lighting conditions. With a lot of research paying attention to this topic, there is still a lack of conclusive and practical methods. In particular, it is realized that dynamic motions at large aspect angles close to 90◦ or static postures have not been investigated in-depth as a part of the radar-based HAR problem. To extensively investigate this type of problem, we propose to use mm-wave FMCWMIMO radar to obtain accurate information of the human subject.

This thesis work aims to fully exploit the six dimensions of information provided by an imaging radar: range, azimuth, elevation, velocity, power and time. Two complementary data representations- point cloud and spectrogram- are utilized to represent these dimensions of information. A signal processing flow is implemented to generate the desired data representations. A hierarchical pipeline consisting of three cascaded deep learning-based classification modules is proposed to process the input data. Particularly, human orientation classification is achieved through the so-called "T-Net" network learning the geometric distribution of point clouds. The positive contribution of each module in the proposed pipeline is validated via an ablation study. The superior performances of the proposed pipeline are also established by comparing with those of the state-of-the-art baselines. The robustness of the proposed pipeline concerning a noisy environment is also discussed. It is also presented that the size