PANDA: Performance prediction for parallel and dynamic stream processing

Pratyush Agnihotri*, Boris Koldehofe, Carsten Binnig, Manisha Luthra

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Distributed Stream Processing (DSP) systems highly rely on parallelism mechanisms to deliver high performance in terms of latency and throughput. Yet the development of such parallel systems altogether comes with numerous challenges. In this paper, we focus on how to select appropriate resources for parallel stream processing under the presence of highly dynamic and unseen workloads. We present PANDA that provides a novel learned approach for highly efficient and parallel DSP systems. The main idea is to provide accurate resource estimates and hence optimal parallelism degree using zero-shot cost models to ensure the performance demands.
Original languageEnglish
Title of host publicationProceedings of the 16th ACM International Conference on Distributed and Event-Based Systems (DEBS'22)
PublisherACM Press
Pages 180–181
Number of pages2
ISBN (Electronic)9781450393089
DOIs
Publication statusPublished - 15-Jul-2022
EventDEBS'22: 16TH ACM INTERNATIONAL CONFERENCE ON DISTRIBUTED AND EVENT-BASED SYSTEMS - Copenhagen, Denmark
Duration: 27-Jun-202230-Jun-2022

Conference

ConferenceDEBS'22
Country/TerritoryDenmark
CityCopenhagen
Period27/06/202230/06/2022

Keywords

  • Distributed Stream Processing
  • parallel operator execution
  • zero shot learning
  • high performance

Cite this