Carbon-Aware Scheduler for Deferrable Batch and CI Jobs

The problem: flexible work run at arbitrary times

A lot of compute does not need to run right now. Nightly test suites, database backups, data exports, image processing queues and model training jobs run at whatever time a developer typed into a cron expression years ago. The carbon intensity of electricity, the grams of CO2 emitted per kilowatt-hour, changes a lot across the day and between regions as solar, wind and fossil plants come on and off. Moving the same job a few hours, or to another cloud region, can change its emissions noticeably without any change to the code.

Google has written publicly about its carbon-intelligent computing platform, which shifts flexible tasks within its data centres to times and places with cleaner power. Small teams have the same kind of flexible work but no scheduler that uses this signal, and growing sustainability reporting requirements mean customers increasingly ask suppliers about emissions.

What already exists and the gap

• Carbon Aware SDK from the Green Software Foundation provides a web API and CLI that answer "when and where is cleanest", but leaves scheduling and execution to you.
• Carbon-aware KEDA operator from Microsoft scales Kubernetes workloads using carbon intensity data, which only helps teams on Kubernetes.
• Electricity Maps, WattTime and the UK Carbon Intensity API publish intensity data and forecasts for different regions.

The gap is the practical layer for small teams: a scheduler that takes deadlines seriously, runs jobs where they already live (a VPS, a CI system), and reports savings honestly, including uncertainty.

What you build

• Job submission: an HTTP API and a small CLI. Each job declares a window, estimated duration and allowed regions, plus a command, image or workflow to run.
• Signal service: fetches forecasts and actuals from the chosen providers on a schedule, stores them and exposes a simple query.
• Scheduler: decides slot and region for each job and re-plans when forecasts update, never breaking a deadline.
• Executors: Docker on registered servers, and GitHub Actions workflow dispatch for CI jobs.
• Reporting: per-job and monthly views comparing the chosen plan with the naive plan.

The technically interesting part

The scheduling problem is a small constrained optimisation under uncertainty. Jobs have windows and durations, servers have limited capacity, forecasts are wrong by varying amounts, and the "right" answer changes as new forecasts arrive. You must choose between simple greedy placement, a sliding-window search, or an integer programme, and justify the choice. Just as important is honest accounting: estimated energy per job, the intensity used (forecast at decision time versus actual afterwards), and a clear statement of what the numbers do and do not mean. Evaluators will respect a careful method over a large claimed saving.

How to evaluate it

• Backtest: replay four or more weeks of historical forecast and actual data for several regions with a realistic job mix, and compare run-now, fixed-night, greedy and your scheduler on estimated emissions and deadline misses.
• Forecast sensitivity: how savings change when you use forecasts rather than perfect hindsight.
• Deadline safety: zero missed deadlines under capacity pressure, or a clear report of when and why it happened.
• Live pilot: run a real CI or backup workload through the scheduler for two weeks.
• Overhead: added latency to job start and the scheduler's own resource use.

Suggested plan

1. Semester 1, weeks 1 to 4: study carbon accounting basics and the data sources; build the signal service.
2. Weeks 5 to 9: job model, first greedy scheduler and the Docker executor.
3. Weeks 10 to 14: backtesting harness with historical data; first results.
4. Semester 2, weeks 1 to 5: improved scheduler, capacity limits and re-planning.
5. Weeks 6 to 10: GitHub Actions executor, dashboard, live pilot.
6. Weeks 11 to 14: final evaluation and write-up.

Path to a product

The first customer is a software company that reports emissions to clients or wants to, and runs heavy CI or data jobs. Offer it as open source with a hosted version that includes reporting, priced per scheduled job volume. A second path is a plug-in for existing CI systems. Regions with cleaner grids at predictable hours give the most convincing savings stories.

Scope warning

Do not build your own carbon intensity forecasts. Do not attempt full lifecycle emissions of hardware. Do not move latency-sensitive user traffic; this is only for deferrable work. Avoid claiming precise savings that your data cannot support.

Learn what you need on Sefism

• DevOps and cloud roadmap : CI/CD, containers and scheduling basics.
• System design cheat sheet : queues, schedulers and trade-offs for the job system.
• Google Cloud Blog : Google writes about carbon-free energy and region choice for workloads.
• Apache Airflow Documentation : how a mature scheduler models jobs, windows and retries.
• Which final year project helps you get a job? : why a measured, honest evaluation makes a project stand out.

Get help with this idea

The accounting method is where most students overreach, so agree it with a mentor early. A mentor can help you set the scope, pick a stack you can finish with, and design an evaluation your examiners will respect. The build is yours; the guidance is on scope, stack and evaluation. New here? Create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session