Design of Novel AI-based Periphery for CIM Architecture

Abstract

With the prosperity of the Internet of Things (IoT) and artificial intelligence (AI), more and more edge devices have been deployed to enable intelligent applications. However, due to the limited energy budget and computation resources, it is challenging to deploy deep neural networks on edge devices directly. The energy consumption of the memory transfer between the processor and memory is a significant part of the total energy consumption, which is the bottleneck of traditional von Neumann architecture. Computation in memory (CIM) is a promising architecture that can significantly reduce this energy consumption by minimizing the data movement between the processor and memory. By utilizing the novel non-volatile memory (NVM) technology, the computation can be performed in the memory in analog domain, which can reduce the energy consumption and latency of the computation. However, the existing periphery of CIM architecture, which is responsible for converting the result of the computation in memory to the digital domain, is not energy-efficient and has a large area overhead. In this thesis, we propose a novel AI-based periphery for CIM architecture to address these issues. The proposed periphery is based on the neural network, which can be trained to convert the crossbar output to the digital domain with high accuracy. Multiple inputs, up to four crossbar columns with different significant levels, can be processed simultaneously and output the shift-and-add result in the digital domain in one shot, without the need for multiplexers or post-processing circuits. Moreover, a new coding scheme called natural Palindromic Coding (NPC), is proposed to encode the output of the neural network, which aims to mitigate the spectral bias of the neural network, where high-frequency components, like the least significant bit (LSB), are underrepresented. Benchmarking results show that the proposed AI-based periphery, with 4 parallel 4-bit inputs and 8-bit output, can achieve 81.16% accuracy or 99.5% accuracy with 1 unit step tolerance. Under the MNIST dataset, the proposed NPC can achieve an accuracy of 96.80%, compared with 97.07% of software inference. In addition, the proposed periphery can achieve 31.77× energy saving and 9× area saving compared with the state-of-the-art neural-network-based periphery, or up to 1.65× energy saving and 2.70× area saving compared with the state-of-the-art SAR-ADC-based periphery.