Convolutional Neural Networks (CNNs) benefit from fine-grained details in high-resolution images, but these images are not always easily available as data collection can be expensive or time-consuming. Transfer learning pre-trains models on data from a related domain before fine-tuning on the main domain, and is a common strategy to deal with limited data. However, transfer learning requires a similar domain with enough available data to exist, and transferability varies from task to task. To deal with limited high-resolution data we propose resolution transfer: using low-resolution data to improve high-resolution accuracy. For resolution transfer, we use Continuous kernel CNNs (CKCNNs) that can adapt their kernel size to changes in resolution and perform well on unseen resolutions. Training CKCNNs on high-resolution images is currently significantly slower than CNNs. We lower the inference costs of CKCNNs to enable training on high-resolution data. We introduce a CKCNN parameterization that constrains the frequencies of kernels to avoid distortions when the kernel size is changed, improving resolution transfer accuracy. We improve fine-tuning with a High-Frequency Adaptation module that complements our constrained kernels. We demonstrate that CKCNNs with kernel resolution adaptation outperform CNNs for resolution transfer tasks with no fine-tuning or with limited fine-tuning data. We compare to transfer learning, and achieve competitive classification accuracy with an ImageNet pre-trained ResNet-18. Our method provides an alternative to transfer learning that uses low-resolution data to improve classification accuracy when high-resolution data is limited.

Location information is essential for the ViT model. Image data has three types of location information: absolute location, relative direction, and relative distance. Various position embeddings methods have been used to introduce location information to the ViT model. Some existing methods are absolute position embeddings, relative position embeddings, fixed sinusoidal position embeddings, and learnable Fourier position embeddings. However, it is unclear what type of location information can be encoded by different position embeddings methods. This paper investigates this question by conducting fully-controlled experiments and feature-level analysis on synthetic datasets. The results suggest that the relative position embeddings cannot encode absolute location information, which leads to inferior performance. All the position embeddings approaches that we test can encode relative location information. However, they have different levels of relative location bias. The learnable absolute position embeddings do not contain any relative location bias and therefore need more data to learn. The fixed sinusoidal and learnable Fourier position embeddings are relatively better, but they also have minor drawbacks. The fixed sinusoidal position embeddings are not trainable, while the Fourier method does not have much bias on relative location information. We propose to make the fixed sinusoidal position embeddings learnable and use pretraining tasks to improve the Fourier method. Our two new approaches show promising results on the testing datasets, and they are competitive compared with a similar approach.

Training Convolutional Neural Network (CNN) models is difficult when there is a lack of labeled training data and no unlabeled data is available. A popular method for this is domain adaptation where the weights of a pre-trained CNN model are transferred to the problem setup. The model is pre-trained on the same task but in a different domain that has plenty of labeled data samples available. In a CNN model, we can rearrange the weights of a convolutional layer by permuting them along the input channel dimension. This work shows that certain weights that are learned in the pre-trained model work well in the problem setup when the weights are rearranged in this manner. Computing the set of all possible rearrangements of the weights is computationally intractable. This work proposes two algorithms to find a good rearrangement of the weights in reasonable computation time. The solutions from the algorithms perform equally well or better than fine-tuning in the domain adaptation between SVHN and MNIST data.