Unpaired day-night domain adaptation using CycleGANs combined with CIConv

Abstract

CycleGANs [1] and CIConv [2] are both relatively new approaches to their respective applications. For CycleGANs this application is unpaired image-to-image domain adaptation and for CIConv this application is making images more
robust to illumination changes. We investigate whether CycleGANs in combination with CIConv can be used to improve the day-night domain adaptation. The resulting images could then be used during the training of CNNs that can be found in self-driving cars. Attempts were made to get the CycleGANs in combination with CIConv to train in a stable manner. These attempts included a variety of hyperparameter combinations, a number of architecture alterations and training procedure adjustments, and most significantly two different loss functions. Both these loss functions apply a CycleConsistency Loss, one applies an additional Adversarial Loss [1] and the other an additional Wasserstein Distance and Gradient Penalty [3]. In this paper we show that CycleGANs with CIConv as the first layer in either the Discriminators or the Generators resulted in unstable training. We conclude that the root of the instability issues lies in the CIConv layer causing exploding gradients resulting in unsuccessful training of the model. Finally, we propose an adjustment to the CIConv layer which shows promise in resolving these issues for the architecture with CIConv in the Generators. However, no extensive testing has been done.