Publication
Improving Neural Saliency Prediction with a Cognitive Model of Human Visual Attention
Ekta Sood; Lei Shi; Matteo Bortoletto; Yao Wang; Philipp Müller; Andreas Bulling
In: Proceedings of the 45th Annual Meeting of the Cognitive Science Society (CogSci). Annual Conference of the Cognitive Science Society (CogSci-2023), July 26-29, Sydney, NSW, Australia, Cognitive Science Society, 2023.
Abstract
We present a novel method for saliency prediction that leverages a cognitive model of visual attention as an inductive bias. This approach is in stark contrast to recent purely data-driven saliency models that achieve performance improvements mainly by increased capacity, resulting in high computational costs and the need for large-scale training datasets. We demonstrate that by using a cognitive model, our method achieves competitive performance to the state of the art across several natural image datasets while only requiring a fraction of the parameters. Furthermore, we set the new state of the art for saliency prediction on information visualizations, demonstrating the effectiveness of our approach for cross-domain generalization. We further provide augmented versions of the full MSCOCO dataset with synthetic gaze data using the cognitive model, which we used to pre-train our method. Our results are highly promising and underline the significant potential of bridging between cognitive and data-driven models, potentially also beyond attention.