Skip to main content Skip to main navigation

Publikation

Self-supervised Test-time Adaptation on Video Data

Fatemeh Azimi; Sebastian Palacio; Federico Raue; Jörn Hees; Luca Bertinetto; Andreas Dengel
In: WACV. IEEE Winter Conference on Applications of Computer Vision (WACV-2022), January 4-8, IEEE, 1/2022.

Zusammenfassung

In typical computer vision problems revolving around video data, pre-trained models are simply evaluated at test time without further adaptation. This general approach inevitably fails to capture potential distribution shifts between training and test data. Adapting a pre-trained model to a new video encountered at test time could be essential to improve performance or to avoid the potentially catastrophic effects of such a shift. However, the lack of available annotations prevents practitioners from using vanilla fine-tuning techniques. This paper explores whether the recent progress in self-supervised learning and test-time domain adaptation (TTA) methods in the image domain can be leveraged to efficiently adapt a model to a previously unseen and unlabelled video. We analyze the effectiveness of several recent self-supervised TTA techniques under the effect of both mild (but arbitrary) and severe domain shifts. From our extensive benchmark on multiple self-supervised dense tracking methods under various domain shifts, we find out that self-supervised TTA methods consistently improve the performance compared to the baselines without adaptation, especially in the presence of severe covariate shift.