Home Posts Carbon-Aware Workloads with Real-Time Grid APIs [2026]
Cloud Infrastructure

Carbon-Aware Workloads with Real-Time Grid APIs [2026]

Carbon-Aware Workloads with Real-Time Grid APIs [2026]
Dillip Chowdary
Dillip Chowdary
Tech Entrepreneur & Innovator · May 11, 2026 · 9 min read

Bottom Line

The winning pattern is simple: treat carbon intensity as a scheduling input, not a reporting metric. Pull a short-horizon forecast, score candidate time slots against SLA rules, and always keep a deterministic fallback window.

Key Takeaways

  • Use 30-minute forecast windows, because the NESO API publishes regional intensity in half-hour intervals.
  • Model carbon as one scheduler input alongside deadline, queue age, cost, and retry budget.
  • Always define a fallback run time so low-carbon deferral never breaks your SLA.
  • Start with one flexible workload class such as batch ETL, CI, rendering, or retraining.

Carbon-aware scheduling is one of the few sustainability techniques that can reduce emissions without rewriting application logic. If your batch jobs, CI pipelines, video rendering, analytics, or model retraining tasks already have slack in their start time, you can shift them into cleaner grid windows instead of running immediately. This tutorial uses NESO's Carbon Intensity API for Great Britain and turns that forecast feed into an SLA-aware scheduler pattern you can ship behind a feature flag.

  • Use 30-minute forecast windows, because the NESO API publishes regional intensity in half-hour intervals.
  • Model carbon as one scheduler input alongside deadline, queue age, cost, and retry budget.
  • Always define a fallback run time so low-carbon deferral never breaks your SLA.
  • Start with one flexible workload class such as batch ETL, CI, rendering, or retraining.

Prerequisites

What you need before you start

  • A workload that can tolerate delayed start, with a clear mustRunBy deadline.
  • A region mapping for where the job actually runs. For this tutorial, that is a GB regionid.
  • A runtime with the standard Fetch API, or any HTTP client you already trust in production.
  • A trigger point such as a queue worker, cron job, CI orchestrator, or internal scheduler.
  • Basic observability so you can log why a job ran now, later, or at fallback time.

Bottom Line

Treat carbon intensity like a placement signal. The scheduler should choose the cleanest eligible slot before the deadline, then fall back deterministically if the forecast is stale, missing, or too dirty.

1. Define the scheduling contract

Do not begin with a general platform rewrite. Begin with one workload class and one policy object. The contract should answer four questions:

  • How long can this job wait before it becomes late?
  • Which region's electricity mix should represent the job's execution location?
  • What carbon threshold is considered clean enough for opportunistic execution?
  • What is the hard fallback time if no clean slot appears?

A minimal contract looks like this:

const policy = {
  jobType: 'nightly-etl',
  regionId: 13, // London
  maxDelayMinutes: 180,
  thresholdGco2PerKwh: 120,
  fallbackMode: 'run-at-deadline'
};

const request = {
  queuedAt: '2026-05-11T09:00:00Z',
  mustRunBy: '2026-05-11T12:00:00Z',
  estimatedRuntimeMinutes: 35
};

This is the architectural pivot: carbon-aware systems are not magical optimizers. They are policy-driven schedulers. If the deadline is vague, or every workload is marked urgent, the design collapses immediately.

Watch out: NESO timestamps are UTC and the forecast windows are half-hour settlement periods. Keep your scheduler in UTC internally, then localize only at the UI edge.

2. Pull the right intensity feed

NESO's regional forecast endpoint is the cleanest place to start because it is unauthenticated, regional, and returns both forecast intensity and generation mix. The documented path for the next 24 hours is /regional/intensity/{from}/fw24h/regionid/{regionid}.

For a quick smoke test, use -s with curl so the output is machine-friendly:

curl -s \
  'https://api.carbonintensity.org.uk/regional/intensity/2026-05-11T09:00Z/fw24h/regionid/13'

If you want to normalize example payloads while testing selectors, TechBytes' Code Formatter is useful for cleaning the returned JSON before you wire it into code.

Now wrap the call in a tiny fetch layer. Keep it dumb and observable:

async function fetchForecast(regionId, fromIso) {
  const url = `https://api.carbonintensity.org.uk/regional/intensity/${fromIso}/fw24h/regionid/${regionId}`;
  const res = await fetch(url, {
    headers: { Accept: 'application/json' }
  });

  if (!res.ok) {
    throw new Error(`forecast_request_failed:${res.status}`);
  }

  const body = await res.json();
  const region = body.data?.[0];

  if (!region?.data?.length) {
    throw new Error('forecast_empty');
  }

  return region.data.map(slot => ({
    from: slot.from,
    to: slot.to,
    forecast: slot.intensity?.forecast,
    index: slot.intensity?.index,
    generationmix: slot.generationmix || []
  }));
}

At this layer, do not make scheduling decisions yet. Just validate shape, record latency, and fail loudly on malformed data. A lot of carbon-aware systems become unreliable because the ingestion layer quietly coerces missing values into zero.

3. Score and choose the window

Once you have forecast slots, the scheduler's job is to find the lowest-emission slot that still satisfies the SLA. A good first implementation uses three gates:

  • The slot must start after the job was queued.
  • The slot plus estimated runtime must finish before mustRunBy.
  • The selected slot should prefer lower forecast intensity, but still honor a deterministic fallback.

Here is a pragmatic scoring function:

function chooseWindow(slots, request, policy) {
  const queuedAtMs = Date.parse(request.queuedAt);
  const mustRunByMs = Date.parse(request.mustRunBy);
  const runtimeMs = request.estimatedRuntimeMinutes * 60 * 1000;

  const eligible = slots.filter(slot => {
    const startMs = Date.parse(slot.from);
    const endMs = startMs + runtimeMs;
    return startMs >= queuedAtMs && endMs <= mustRunByMs;
  });

  if (!eligible.length) {
    return { decision: 'run_now', reason: 'no_eligible_slot' };
  }

  const underThreshold = eligible
    .filter(slot => Number.isFinite(slot.forecast) && slot.forecast <= policy.thresholdGco2PerKwh)
    .sort((a, b) => a.forecast - b.forecast);

  if (underThreshold.length) {
    return { decision: 'defer', reason: 'cleaner_slot_found', slot: underThreshold[0] };
  }

  const lowest = eligible
    .filter(slot => Number.isFinite(slot.forecast))
    .sort((a, b) => a.forecast - b.forecast)[0];

  return lowest
    ? { decision: 'defer', reason: 'best_available_slot', slot: lowest }
    : { decision: 'run_now', reason: 'no_numeric_forecast' };
}

This is intentionally conservative. It does not try to estimate savings from queue contention, spot interruptions, or downstream data freshness. That comes later. Early on, the objective is to prove that the scheduler can defer safely and explain every decision.

4. Execute with guardrails

The orchestration layer should make one decision, emit one audit record, and then either run or requeue. That keeps operations sane when a forecast changes 30 minutes later.

async function scheduleOrRun(request, policy) {
  const forecast = await fetchForecast(policy.regionId, request.queuedAt);
  const choice = chooseWindow(forecast, request, policy);

  if (choice.decision === 'run_now') {
    return {
      action: 'run_now',
      reason: choice.reason,
      scheduledFor: request.queuedAt
    };
  }

  return {
    action: 'schedule',
    reason: choice.reason,
    scheduledFor: choice.slot.from,
    forecastGco2PerKwh: choice.slot.forecast,
    index: choice.slot.index
  };
}

In production, add these guardrails before rollout:

  • Staleness checks so old forecasts never defer fresh work.
  • Idempotency keys so retries do not enqueue duplicate runs.
  • Reason codes in logs and metrics so every defer decision is explainable.
  • A kill switch so operators can disable deferral during incidents or backlog spikes.
  • A per-job opt-out for customer-facing or latency-sensitive paths.

If your platform spans multiple countries, this is where a global provider becomes useful. Electricity Maps documents real-time, historical, and forecast carbon intensity endpoints, plus geolocation and data-center region lookups, using an auth-token request header.

Verification and expected output

What success looks like

Your first verification target is not emissions savings. It is decision correctness. The scheduler should produce a stable, auditable decision from a valid forecast payload.

{
  "action": "schedule",
  "reason": "cleaner_slot_found",
  "scheduledFor": "2026-05-11T10:30:00Z",
  "forecastGco2PerKwh": 98,
  "index": "low"
}

The exact numbers will vary by region and time, but the shape should be consistent. NESO's documented examples expose from, to, intensity.forecast, intensity.index, and generationmix for regional forecast data.

Top 3 troubleshooting issues

  1. No eligible slot is found. Your job deadline is probably tighter than your runtime estimate or deferral window. Reduce maxDelayMinutes, lower runtime assumptions, or treat that workload class as non-deferrable.
  2. The scheduler defers forever. You forgot a hard deadline path. Every policy needs a final run condition such as run-at-deadline or run-when-queue-age-exceeds.
  3. Carbon savings look random. You may be using the wrong geography. Carbon-aware scheduling only works when the signal matches the workload's real execution region, not the user's region or your company headquarters.

What's next

Once the basic scheduler is working, the next layer is not bigger code. It is better policy design.

  • Add a cost dimension so the scheduler can weigh carbon against day-ahead price or internal spend targets.
  • Separate workloads into gold, silver, and bronze urgency classes instead of using one global threshold.
  • Track avoided emissions per deferral decision so the platform can justify the extra orchestration complexity.
  • Expand from one region to multi-region placement, where the scheduler can choose both when and where to run.
  • Graduate from a single-country public API to a global commercial feed when your fleet outgrows GB-only coverage.

The important architectural lesson is that carbon-aware execution should behave like any other production scheduler: bounded, observable, and boring under failure. If you achieve that, the emissions improvements become operationally repeatable instead of aspirational.

Frequently Asked Questions

How do I choose a carbon threshold for deferrable jobs? +
Start with a policy threshold that is easy to reason about, then tune from production data. A common first cut is to schedule only when forecast <= thresholdGco2PerKwh, but you should compare that against queue delay, missed windows, and actual job completion times before tightening it.
Should I schedule on forecast intensity or actual intensity? +
For deferred execution, use forecast because the scheduler is deciding about a future slot. Actual intensity is still useful for post-run reporting and for validating how close the forecast was, but it cannot drive a future placement decision on its own.
What happens if the grid API is down or returns bad data? +
Fail closed, not open-ended. If the API is stale, empty, or malformed, the scheduler should fall back to the default run path or a hard deadline rule such as run-at-deadline, and it should emit a reason code so operators can see that carbon optimization was bypassed.
Can I use this pattern outside Great Britain? +
Yes, but you need a provider with the right regional coverage and operational terms. NESO's public API is excellent for GB, while providers such as Electricity Maps document broader real-time and forecast coverage and use authenticated endpoints like /v3/carbon-intensity/latest with an auth-token header.

Get Engineering Deep-Dives in Your Inbox

Weekly breakdowns of architecture, security, and developer tooling — no fluff.

Found this useful? Share it.