Skip to main content Skip to main navigation

Publication

Grizzly: Efficient Stream Processing Through Adaptive Query Compilation

Philipp M Grulich; Sebastian Breß; Steffen Zeuch; Jonas Traub; Janis von Bleichert; Zongxiong Chen; Tilmann Rabl; Volker Markl
In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. ACM SIGMOD International Conference on Management of Data (SIGMOD-2020), Pages 2487-2503, ACM, 2020.

Abstract

Stream Processing Engines (SPEs) execute long-running queries on unbounded data streams. They follow an interpretation-based processing model and do not perform runtime optimizations. This limits the utilization of modern hardware and neglects changing data characteristics at runtime. In this paper, we present Grizzly, a novel adaptive query compilation-based SPE, to enable highly efficient query execution. We extend query compilation and task-based parallelization for the unique requirements of stream processing and apply adaptive compilation to enable runtime re-optimizations. The combination of light-weight statistic gathering with just-in-time compilation enables Grizzly to adjust to changing data-characteristics dynamically at runtime. Our experiments show that Grizzly outperforms state-of-the-art SPEs by up to an order of magnitude in throughput.

Weitere Links