Motivated by the desire to bring intelligent processing at the Edge, enabling online learning on resource- and latency-constrained embedded devices has become increasingly appealing, as it has the potential to tackle a wide range of challenges: on the one hand, it can deal with on-the-fly adaptation to fast sensor-generated streams of data under changing environments and on the other hand, it can address a variety of challenges associated with offline training in the cloud, such as incurred energy consumption of sensor data transfers and extra memory storage for the training samples, but also data privacy and security concerns. Concurrently, maintaining low-latency and power-efficient inference is paramount for edge AI computing systems, and thus learning/adapting online with minimal incurred overhead is crucial.

In this work, we propose EON-1, an Edge ONline Learning SCNN (Spiking Convolutional Neural Network) processor with 1-bit synaptic weights, 1-spike per neuron and 1-neuron updated per input, which we have benchmarked for both ASIC and FPGA platforms. Our key contribution is proposing a binary and stochastic SDTP rule which, benchmarked in an ASIC node, achieves less than 1% energy overhead for inference. To our knowledge, our solution incurs the least energy overhead for inference, compared to state-of-the-art solutions, showing a better efficiency by at least a factor of 10x. We also report 94% and 77.65% accuracy on the MNIST and Fashion-MNIST classification tasks, and we achieve 0.09pJ/SOP and 1.5pJ/SOP energy efficiency during inference and learning, respectively. We extend our solution to demonstrate a practical use-case of performing inference in real-time UHD videos while coping with streaming data and we showcase 60 FPS UHD video processing.

Background & Objective
Cardiovascular diseases (CVDs) are the leading cause for death globally nowadays. Pulse wave velocity (PWV), a marker of arterial stiffness, is an important predictor of CVD risk. In precedent work, carotid artery data was collected with ultrasound to estimate the PWV with a digital signal processing (DSP) pipeline. As a potential alternative to the DSP-based approach, this thesis studies the applicability of machine learning(ML) for the estimation of carotid artery motion (diameter, distension, etc.) and explores to what extent neural networks can exploit the ultrasound data to extract relevant biomarker information.
Methods:
This thesis proposes a ML pipeline that processes the ultrasound data in a different perspective than the DSP approach. The ML pipeline consists of four modules (neural networks & post-processing) to: 1) segmentation based on cardiac cycle (CC), 2) detect the region of interest (ROI) of artery in the ultrasound data, 3) tracking the artery diameter and 4) post processing to estimate cardiac parameters e.g. pulse arrival time (PAT), an essential part of PWV estimation. Exploiting the features of the artery-lumen structure and time-evolving characteristics of collected ultrasound signal, the designed ML pipeline can acquire cardiac markers spatially and temporally with irregular kernels and sliding mechanism, decompose the complicated estimation into compact sub-modules.
Results:
The results show that the ML approach can successfully estimate the artery diameters and reserve important waveform features (max-slope moment) of the artery diameter, and can infer the CC markers without ECG data as a segmenting event for heart cycles. Thus, the PAT can be computed as the time difference of the max-slope moments of inferred artery diameter and detected CC markers. According to the numerical results, the PAT can be estimated on the average for an ultrasound data recording (120s), and the correlation coefficient of label PAT (computed from estimated parameters of DSP pipeline and ECG data) and estimated PAT (ML pipeline) is 0.8250. This indicates a good correlation and hence proves the effectiveness of the mean PAT estimation.
Conclusion:
In conclusion, the proposed ML pipeline can effectively estimate mean PAT, and demonstrate the feasibility to estimate PWV as a relevant cardiovascular marker using only ultrasound data of carotid arteries. Apart from the PAT, the heart rate can also be possibly tracked via intermediate results of the ML pipeline (CC markers). From the future perspective, the potential of phase information in the raw ultrasound data and further optimization are worth exploring, and the extension to hardware (e.g. chips, embedded system) can be implemented as a practical application.

In the recent past, real-time video processing using state-of-the-art deep neural networks (DNN) has achieved human-like accuracy but at the cost of high energy consumption, making them infeasible for edge device deployment. The energy consumed by running DNNs on hardware accelerators is dominated by the number of memory read/writes and multiplyaccumulate (MAC) operations required. As a potential solution, this work explores the role of activation sparsity in efficient DNN inference. As the predominant operation in DNNs is matrix-vector multiplication of weights with activations, skipping operations and memory
fetches where (at least) one of them is zero can make inference more energy efficient. Although spatial sparsification of activations is researched extensively, introducing and exploiting temporal sparsity is much less explored in DNN literature. This work presents a new DNN layer (called temporal delta layer) whose primary objective is to induce temporal activation sparsity during training. The temporal delta layer promotes activation sparsity by performing delta operation facilitated by activation quantization and l1 norm based penalty to the cost function. During inference, the resulting model acts as a conventional quantized
DNN with high temporal activation sparsity. The new layer was incorporated as a part of the standard ResNet50 architecture to be trained and tested on the popular human action recognition dataset (UCF101). The method caused 2x improvement in activation sparsity, with 5% accuracy loss.