Today, with increasingly aging population, healthcare systems in many countries need to improve their effectiveness, and the automatic HAR technology can be beneficial. This can provide early diagnosis of changes in behavioral patterns in the home environment, without hospitalization, and detect critical events such as falls in a timely manner. In this area, radar-based HAR solutions are attracting the researchers' attention because no optical images are captured by radars, and thus respect of privacy and functionality in darkness can be guaranteed. Furthermore, no sensors need to be worn by the person being monitored.
Most previous work related to radar-based HAR employs image-like data representation such as spectrograms, range profiles, and snapshots of point clouds, and the information contained in these data representation is limited. For instance, spectrograms and range profiles cannot reflect the body shape of the subjects, while snapshots of point cloud do not contain Doppler or intensity information.
To overcome the limitation of these data representations, we propose to utilize the data from a mm-wave FMCW MIMO radar to create a novel data representation of point cloud with Doppler and intensity values, plus temporal information to achieve accurate HAR.
Specifically, this thesis work focuses on the high dimensional radar point clouds and on a pipeline to generate and process this novel data representation. The proposed method combines the spatial information of point clouds with other features like Doppler, intensity/SNR, and time, expanding each point from 3D coordinates to a 6D vector.
Hence, the movement of every part of the body can be expressed by those points. A module consisting of adaptive noise cancelation, frame selection, and resampling is proposed to process point clouds to match the input of subsequent classifiers.
Considering that the core of the self-attention concept matches well the information within point clouds, we investigate three self-attention based models as classifiers. These models can learn the spatial distribution of point clouds with their extra features with self-attention mechanism. The best combination of different input features, the positive contribution of the proposed adaptive noise cancelation method, and the performance of these three models are studied with experimental data from the MMActivity dataset and a purposely collected TU Delft (TUD) dataset.