TY - JOUR
T1 - Carbon emission-aware job scheduling for Kubernetes deployments
AU - Piontek, Tobias
AU - Haghshenas, Kawsar
AU - Aiello, Marco
N1 - Publisher Copyright:
© 2023, The Author(s).
PY - 2023
Y1 - 2023
N2 - Decreasing carbon emissions of data centers while guaranteeing Quality of Service (QoS) is one of the major challenges for efficient resource management of large-scale cloud infrastructures and societal sustainability. Previous works in the area of carbon reduction mostly focus on decreasing overall energy consumption, replacing energy sources with renewable ones, and migrating workloads to locations where lower emissions are expected. These measures do not consider the energy mix of the power used for the data center. In other words, all KWh of energy are considered the same from the point of view of emissions, which is rarely the case in practice. In this paper, we overcome this deficit by proposing a novel practical CO2-aware workload scheduling algorithm implemented in the Kubernetes orchestrator to shift non-critical jobs in time. The proposed algorithm predicts future CO2 emissions by using historical data of energy generation, selects time-shiftable jobs, and creates job schedules utilizing greedy sub-optimal CO2 decisions. The proposed algorithm is implemented using Kubernetes’ scheduler extender solution due to its ease of deployment with little overheads. The algorithm is evaluated with real-world workload traces and compared to the default Kubernetes scheduling implementation on several actual scenarios.
AB - Decreasing carbon emissions of data centers while guaranteeing Quality of Service (QoS) is one of the major challenges for efficient resource management of large-scale cloud infrastructures and societal sustainability. Previous works in the area of carbon reduction mostly focus on decreasing overall energy consumption, replacing energy sources with renewable ones, and migrating workloads to locations where lower emissions are expected. These measures do not consider the energy mix of the power used for the data center. In other words, all KWh of energy are considered the same from the point of view of emissions, which is rarely the case in practice. In this paper, we overcome this deficit by proposing a novel practical CO2-aware workload scheduling algorithm implemented in the Kubernetes orchestrator to shift non-critical jobs in time. The proposed algorithm predicts future CO2 emissions by using historical data of energy generation, selects time-shiftable jobs, and creates job schedules utilizing greedy sub-optimal CO2 decisions. The proposed algorithm is implemented using Kubernetes’ scheduler extender solution due to its ease of deployment with little overheads. The algorithm is evaluated with real-world workload traces and compared to the default Kubernetes scheduling implementation on several actual scenarios.
KW - Carbon-aware load shifting
KW - CO signal following
KW - CO signal prediction
KW - Kubernetes
KW - Workload scheduling
UR - http://www.scopus.com/inward/record.url?scp=85163390301&partnerID=8YFLogxK
U2 - 10.1007/s11227-023-05506-7
DO - 10.1007/s11227-023-05506-7
M3 - Article
AN - SCOPUS:85163390301
SN - 0920-8542
VL - 80
SP - 549
EP - 569
JO - Journal of supercomputing
JF - Journal of supercomputing
ER -