Carbon Optimized Cronjobs in Score
When you have cronjobs that do not need to run at a specific time of the day, you can as well run them at a time when power production comes mainly from renewable energy and reduce the carbon footprint of your application.
In this small example, we utilize:
The result will be a Score specification of cronjob with carbon optimized schedule.
Finding the Schedule
Energy production, at least in Germany, is quite easy to forecast, at least 24 hours ahead. If you use solar panels or electricity provider with dynamic prices, you probably know the charts from Fraunhofer ISE. Based on the open data from Fraunhofer, the company Bluehands in Karlsruhe, Germany, created a carbon aware computing SDK and an API.
First thing we have to do is creating an API token, which we receive when we register at Carbon Aware Computing. Use the /register endpoint at the Swagger-UI. The token will be delivered to your email address.
Once received, we can use the token to get authorization using the second endpoint /emissions/forecast/current . If you want to know exactly for which locations you can request forecasts, use the /locations endpoint.
The result will look something like this:
For a cronjob that requires 10 minutes (windowSize) the optimal execution time will be 10:43 UTC the next day.
Configure Score Specification
At the time of writing this article, cronjob specification in Score is an extension on the Score specification. Meaning, you need a Score specification defining your workload and an override specification defining the cronjob. As a developer who just wants things to work, I use Humanitec to orchestrate my workloads. That’s why the cronjob extension is tailored for a Humanitec cronjob.
The actual workload will be a hello world container:
The interesting part is the cronjob extension with placeholders for hours and minutes:
Since we will use a Gitlab CI pipeline for configuring and deploying the cronjob, we need to get the carbon optimized timestamps from command line. It will look a bit chaotic, that’s why each line is documented:
In this example, we state that we want to run the workload in germanywestcentral for 10 minutes. We will use some python scripting, therefore we use python:3 base image. For mustache templating we require the python modules chevron and pyyaml.
At first we call the API with our desired region and cronjob duration we would like to have the optimized schedule for. We use wget for that and do not forget to provide our API_KEY in the header.
Next we echo the JSON result from the API call and pipe it as argument in to a small python script. In the script we parse the optimal timestamp and store it in a local console variable carbon_date .
Since a cronjob has a recurring schedule, we are not interested in the date of the timestamp, but only in hours and minutes. We use string slicing to pares minutes and hours out of the carbon_date in to CARBON_OPTIMIZED_HOURS and CARBON_OPTIMIZED_MINUTES .
We then create a mustache template json containing CARBON_OPTIMIZED_HOURS and CARBON_OPTIMIZED_MINUTES. This mustache template is then used by chevron to replace values inside the humanitec cronjob yaml.
Deploy Cronjob
Finally we can deploy our cronjob to Humanitec:
The result in Humanitec looks like this:
The cronjob schedule has the name carbon-efficient as we had defined in the Score extension file.
Lookout
There are a few things to consider.
First, you need to be aware of timezones. On command line we parse hours and minutes from a UTC timestamp, but your cluster may run in a different timezone. In that case, you would need to correct the hours of the carbon optimized schedule before writing the mustache template.
Then there is the carbon efficiency of this approach itself. Yes, the cronjob is running carbon efficient, but when is the CI job defining the next carbon optimized schedule being triggered? If I run a CI job every night for setting the new cronjob schedule which results in a new deployment, I use more energy then just running the cronjob. Making this exercise every day for a short running cronjob is probably not efficient, neither power consumption wise nor carbon aware.
Still, I do use this for my cronjobs. Every weekend around noon, when statistically the carbon emissions are the lowest, I determine the next carbon efficient schedule and redeploy my workloads. I hope that in average the optimal timestamp is approximately the same throughout the week. Statistically, I should be more often on the right side than on the wrong side.
It would probably be more efficient if the cronjob could find its own best next schedule after each execution. Maybe it is possible to implement a carbon aware Humanitec driver for cronjobs that resets the schedule for the next optimized timestamp.
Summary
In this post, I demonstrated, that it is basically possible to run workloads carbon optimized. What I like about this exercise is the new challenge for more efficiency. When do I actually need to run my workloads? Do I always need to trigger pipelines on each commit? Can I optimize the execution time?
Running a CI pipeline every day, to redeploy a cronjob that runs for only 10 minutes, is probably not carbon efficient. However, the approach opens the door and is a first step to iterate on.
