Publication
All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streams
Marco Pegoraro; Jonas Seng; Dustin Heller; Wil M. P. van der Aalst; Kristian Kersting
In: Fabrizio Maria Maggi; Henrik Leopold; Flávia Maria Santoro; Avigdor Gal (Hrsg.). Business Process Management - 24th International Conference, BPM 2026, Toronto, ON, Canada, September 27-October 2, 2026, Proceedings. Business Process Management (BPM), Pages 521-538, Lecture Notes in Computer Science, Vol. 16936, Springer, 2026.
Abstract
Disciplines such as business process management and pro-
cess mining aid organizations by discovering insights about processes
on the basis of recorded event data. However, an obstacle to process
analysis is data multi-modality: for instance, data in video form are not
directly interpretable as events. Existing approaches rely on a dictionary
of activity label as input, cannot provide frame-by-frame labeling expla-
nations, or rely on superseded computer vision techniques. In this work,
we present SnapLog, an approach to extract event data from videos by
converting frames to feature vectors using image embeddings and per-
forming temporal segmentation through frame-wise similarity matrices.
A generalized few-shot classification is then used to assign labels to the
video segments, yielding labeled, timestamped sub-sequences of frames
that are interpretable as events. Conventional process mining techniques
can be used to analyze the resulting data. We show that our approach
produces logs that accurately reflect the process in the videos.