Publication
Self-supervised Test-time Adaptation on Video Data
Fatemeh Azimi; Sebastian Palacio; Federico Raue; Jörn Hees; Luca Bertinetto; Andreas Dengel
In: WACV. IEEE Winter Conference on Applications of Computer Vision (WACV-2022), January 4-8, IEEE, 1/2022.
Abstract
In typical computer vision problems revolving around video data, pre-trained models are simply evaluated at test time without further adaptation.
This general approach inevitably fails to capture potential distribution shifts between training and test data.
Adapting a pre-trained model to a new video encountered at test time could be essential to improve performance or to avoid the potentially catastrophic effects of such a shift.
However, the lack of available annotations prevents practitioners from using vanilla fine-tuning techniques.
This paper explores whether the recent progress in self-supervised learning and test-time domain adaptation (TTA) methods in the image domain can be leveraged to efficiently adapt a model to a previously unseen and unlabelled video.
We analyze the effectiveness of several recent self-supervised TTA techniques under the effect of both mild (but arbitrary) and severe domain shifts.
From our extensive benchmark on multiple self-supervised dense tracking methods under various domain shifts, we find out that self-supervised TTA methods consistently improve the performance compared to the baselines without adaptation, especially in the presence of severe covariate shift.