Exploring the cost and performance benefits of AWS Step Functions using a data processing pipeline

Anil Mathew, Vasilios Andrikopoulos, Frank J. Blaauw

OnderzoeksoutputAcademicpeer review

165 Downloads (Pure)

Samenvatting

In traditional cloud computing, dedicated hardware is substituted by dynamically allocated, utility-oriented resources such as virtualized servers. While cloud services are following the pay-as-you-go pricing model, resources are billed based on instance allocation and not on the actual usage, leading the customers to be charged needlessly. In serverless computing, as exemplified by the Function-as-a-Service (FaaS) model where functions are the basic resources,
functions are typically not allocated or charged until invoked or triggered. Functions are not applications, however, and to build compelling serverless applications they frequently need to be orchestrated with some kind of application logic. A major issue emerging by the use of orchestration is that it complicates further the already complex billing model used by FaaS providers, which in combination with the lack of granular billing and execution details
offered by the providers makes the development and evaluation of serverless applications challenging. Towards shedding some light into this matter, in this work we extensively evaluate the state-of-the-art function orchestrator AWS
Step Functions (ASF) with respect to its performance and cost. For this purpose we conduct a series of experiments using a serverless data processing pipeline application developed as both ASF Standard and Express workflows. Our results show that Step Functions using Express workflows are economical when running short-lived tasks with many state transitions. In contrast, Standard workflows
are better suited for long-running tasks, offering in addition detailed debugging and logging information. However, even if the behavior of the orchestrated AWS Lambda functions influences both types of workflows, Step Functions realized as Express workflows get impacted the most by the phenomena affecting Lambda functions.
Originele taal-2English
Titel2021 IEEE/ACM 14th International Conference on Utility and Cloud Computing (UCC’21), December 6–9, 2021, Leicester, United Kingdom
Plaats van productieNew York
UitgeverijAssociation for Computing Machinery
Aantal pagina's10
DOI's
StatusPublished - 2021

Citeer dit