Carbon emission-aware job scheduling for Kubernetes deployments

Tobias Piontek, Kawsar Haghshenas*, Marco Aiello

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

14 Downloads (Pure)

Abstract

Decreasing carbon emissions of data centers while guaranteeing Quality of Service (QoS) is one of the major challenges for efficient resource management of large-scale cloud infrastructures and societal sustainability. Previous works in the area of carbon reduction mostly focus on decreasing overall energy consumption, replacing energy sources with renewable ones, and migrating workloads to locations where lower emissions are expected. These measures do not consider the energy mix of the power used for the data center. In other words, all KWh of energy are considered the same from the point of view of emissions, which is rarely the case in practice. In this paper, we overcome this deficit by proposing a novel practical CO2-aware workload scheduling algorithm implemented in the Kubernetes orchestrator to shift non-critical jobs in time. The proposed algorithm predicts future CO2 emissions by using historical data of energy generation, selects time-shiftable jobs, and creates job schedules utilizing greedy sub-optimal CO2 decisions. The proposed algorithm is implemented using Kubernetes’ scheduler extender solution due to its ease of deployment with little overheads. The algorithm is evaluated with real-world workload traces and compared to the default Kubernetes scheduling implementation on several actual scenarios.

Original languageEnglish
Pages (from-to)549–569
Number of pages21
JournalJournal of supercomputing
Volume80
Early online date27-Jun-2023
DOIs
Publication statusPublished - 2023

Keywords

  • Carbon-aware load shifting
  • CO signal following
  • CO signal prediction
  • Kubernetes
  • Workload scheduling

Fingerprint

Dive into the research topics of 'Carbon emission-aware job scheduling for Kubernetes deployments'. Together they form a unique fingerprint.

Cite this