Publication
Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations
Wolfgang Stammer; Marius Memmel; Patrick Schramowski; Kristian Kersting
In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. International Conference on Computer Vision and Pattern Recognition (CVPR-2022), June 18-24, New Orleans, LA, USA, Pages 10307-10318, IEEE, 2022.
Abstract
Learning visual concepts from raw images without strong supervision is a challenging task. In this work, we show the advantages of prototype representations for understanding and revising the latent space of neural concept learners. For this purpose, we introduce interactive Concept Swapping Networks (iCSNs), a novel framework for learning concept-grounded representations via weak supervision and implicit prototype representations. iCSNs learn to bind conceptual information to specific prototype slots by swapping the latent representations of paired images. This semantically grounded and discrete latent space facilitates human understanding and human-machine interaction. We support this claim by conducting experiments on our novel data set "Elementary Concept Reasoning" (ECR), focusing on visual concepts shared by geometric objects.