Video Object Detectors (VID) are used in various applications such as surveillance, inspection, etc. Often in these applications there exists a spatial area of interest and a static background. The static backgrounds remain constant throughout the video sequence in the training data establishing an undesirable correlation with the moving object during training. To hide static backgrounds in the video, masking is an option. We create multiple synthetic datasets and reveal that
(i) VIDs detect moving objects better if the static background in the train and test set are similar or from the same distribution.
(ii) VIDs drop in performance if the static background in the train and test are different.
(iii) Adding more static backgrounds during training does not make VID robust to static background changes at test time.
(iv) Masking or removing static backgrounds cannot prevent VIDs from learning correlations with static backgrounds.
The experiments shed light on the usage of static backgrounds for detecting dynamic objects.