Pattern-based serverless data processing pipelines for Function-as-a-Service orchestration systems

Anil Mathew, Vasilios Andrikopoulos*, Frank J. Blaauw, Dimka Karastoyanova

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

13 Downloads (Pure)

Abstract

The serverless cloud computing execution model presents an opportunity for implementing data processing pipelines in a cost efficient manner with minimum or no consideration about managing their operational aspects. In doing so, however, a major concern that emerges is the adoption of provider-specific languages required for orchestrating serverless functions and supporting services into the higher order logic of pipelines. As a result, the vendor lock-in problem inherent to cloud computing becomes further intensified. In this work we propose an approach to mitigate this issue through the adoption and adaptation of well-known patterns from the literature as the means for modeling the pipeline logic. By providing mappings from these patterns to vendor-specific orchestration language constructs we allow for efficient realization of provider-agnostic pipeline models as provider-specific executable workflows. An industrial case study provides evidence towards the suitability of our proposal for practical purposes.

Original languageEnglish
Pages (from-to)87-100
Number of pages14
JournalFuture generation computer systems
Volume154
Early online date6-Jan-2024
DOIs
Publication statusE-pub ahead of print - 6-Jan-2024

Keywords

  • Enterprise integration patterns
  • Function orchestration
  • Function-as-a-Service (FaaS)
  • Serverless
  • Workflow patterns

Cite this