Publication
MultiMediate'24: Multi-Domain Engagement Estimation
Philipp Müller; Michal Balazia; Tobias Baur; Michael Dietz; Alexander Heimerl; Anna Penzkofer; Dominik Schiller; François Brémond; Jan Alexandersson; Elisabeth André; Andreas Bulling
In: Proceedings of the 32nd ACM International Conference on Multimedia. ACM International Conference on Multimedia (ACM MM-2024), New York, NY, USA, MM '24, ISBN 9798400706868, Association for Computing Machinery, 2024.
Abstract
Estimating the momentary level of participant's engagement is an important prerequisite for assistive systems that support human interactions. Previous work has addressed this task in within-domain evaluation scenarios, i.e. training and testing on the same dataset. This is in contrast to real-life scenarios where domain shifts between training and testing data frequently occur. With MultiMediate'24, we present the first challenge addressing multi-domain engagement estimation. As training data, we utilise the NOXI database of dyadic novice-expert interactions. In addition to within-domain test data, we add two new test domains. First, we introduce recordings following the NOXI protocol but covering languages that are not present in the NOXI training data. Second, we collected novel engagement annotations on the MPIIGroupInteraction dataset which consists of group discussions between three to four people. In this way, MultiMediate'24 evaluates the ability of approaches to generalise across factors such as language and cultural background, group size, task, and screen-mediated vs. face-to-face interaction. This paper describes the MultiMediate'24 challenge and presents baseline results. In addition, we discuss selected challenge solutions.