This paper, in answering the question ”Can effi- cient on-device spectrum sensing be achieved on microcontrollers?”, presents a simple yet compre- hensive approach to signal classification using Con- volutional Neural Networks (CNNs) optimized for deployment on resource-constrained devices. Us- ing data generated via MATLAB’s Wireless Tool- box, as well real world data obtained from testbeds, we created a robust dataset of 9000 samples for training our model. The steps we took while de- veloping a CNN model that performs efficiently on microcontrollers include: data augmentation (pre- processing), model compression and quantization. The model significantly outperformed baseline ac- curacy metrics and maintained competitive infer- ence times, despite the hardware limitations of mi- crocontrollers. This reinforces the idea that Deep Learning has great potential in signal classification. Our research has the potential of being applied to smart homes, IoT networks, industrial automation, and public safety, where our optimized model facil- itates efficient spectrum utilization and minimizes interference.