Publikation
Object Centric Concept Bottlenecks
David Steinmann; Wolfgang Stammer; Antonia Wüst; Kristian Kersting
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2505.24492, Pages 1-26, Computing Research Repository, 2025.
Zusammenfassung
Developing high-performing, yet interpretable models remains a critical challenge
in modern AI. Concept-based models (CBMs) attempt to address this by extracting
human-understandable concepts from a global encoding (e.g., image encoding)
and then applying a linear classifier on the resulting concept activations, enabling
transparent decision-making. However, their reliance on holistic image encodings
limits their expressiveness in object-centric real-world settings and thus hinders
their ability to solve complex vision tasks beyond single-label classification. To
tackle these challenges, we introduce Object-Centric Concept Bottlenecks (OCB),
a framework that combines the strengths of CBMs and pre-trained object-centric
foundation models, boosting performance and interpretability. We evaluate OCB on
complex image datasets and conduct a comprehensive ablation study to analyze key
components of the framework, such as strategies for aggregating object-concept
encodings. The results show that OCB outperforms traditional CBMs and allows
one to make interpretable decisions for complex visual tasks