Skip to main content Skip to main navigation

Publication

Costream: Learned Cost Model for Operator Placement in Edge-Cloud Environments

Roman Heinrich; Carsten Binnig; Harald Kornmayer; Manisha Luthra
In: 40th IEEE International Conference on Data Engineering (ICDE 2024) (recently accepted). IEEE International Conference on Data Engineering (ICDE-2024), May 13-17, Netherlands, Pages 1-14, IEEE, 2024.

Abstract

In this work, we present COSTREAM, a novel learned cost model for Distributed Stream Processing Systems that provides accurate predictions of the execution costs of a streaming query in an edge-cloud environment. The cost model can be used to find an initial placement of operators across heterogeneous hardware, which is especially extremely important in these environments. In our evaluation, we demonstrate that COSTREAM can produce highly accurate cost estimates for the initial operator placement and even generalize to unseen placements, queries, and hardware. When using COSTREAM to optimize the placements of streaming operators, a median speed-up of around 21× can be achieved compared to existing placement strategies.