Multi-scale convolutional neural network for pixel-wise reconstruction of Van Gogh’s drawings

Abstract

This paper investigates the reconstruction of Van Gogh’s drawings which have been degraded in the course of time due to aging problems, like ink fading and discoloration. Learning to predict the past and original appearances of degraded drawings can help to envisage how the artist’s work may have looked at the time of creation. In this paper, we use reproductions as reference information for the past appearances of drawings and consider the reconstruction of drawings as a pixel-wise prediction problem. We present an approach to automatically predict the past appearances of drawings. This approach brings together methods from multi-resolution image analysis and deep convolutional neural networks (CNNs) for addressing the task of pixel-wise prediction. Our experiments first investigate how scale affects prediction performance of the proposed multi-scale CNN framework and then demonstrate the reconstruction capability of the multi-scale CNN framework. The results demonstrate that the predictive reconstruction of degraded images is a feasible endeavor.