Recently, deep learning has emerged as a powerful technique for side-channel attacks, capable of even breaking common countermeasures. Still, trained models are generally large, and thus, performing evaluation becomes resource-intensive. The resource requirements increase in realistic settings where traces can be noisy, and countermeasures are active. In this work, we exploit mimicking to compress the learned models. We demonstrate up to 300 times compression of a state-of-the-art CNN. The mimic shallow network can also achieve much better accuracy as compared to when trained on original data and even reach the performance of a deeper network.@en