Scheduling Weekly NDVI Refreshes with EventBridge

Create an EventBridge Scheduler schedule with cron(0 3 ? * MON *) and ScheduleExpressionTimezone: "Europe/Amsterdam", a FlexibleTimeWindow of 15 minutes, a retry policy, and an SQS dead-letter queue, targeting a discovery Lambda through sfn:StartExecution. The discovery step searches STAC from a watermark stored in DynamoDB rather than a fixed now - 7 days window, records every scene it accepts with a conditional write so a scene is never tiled twice, and dispatches at most MAX_SCENES_PER_RUN scenes — because eleven scenes backlogged behind a three-week outage would otherwise fan out into 396 simultaneous tile workers.

Context

The serverless NDVI tiling recipe starts when a new scene lands in a bucket you own. That is the right trigger when you are mirroring imagery. It is the wrong trigger — because it does not exist — when your source is a public archive you read directly, which is the usual case for Sentinel-2: the data is already sitting in object storage, catalogued in STAC, and nobody is going to send you an event when a new acquisition covers your fields. The pipeline needs a clock instead.

A clock introduces two problems an event trigger does not have. The first is duplication: a poll that runs weekly and looks back seven days will re-examine scenes it already processed every time, and any overlap in those windows means processing the same scene twice unless something remembers. The second is catch-up. Schedules fire into a void when their target is broken, and the occurrences that were missed do not come back.

Both problems have the same shape of answer: state. A DynamoDB ledger remembers which scenes have been dispatched, which makes re-examining a scene harmless. A watermark remembers how far through time the pipeline has actually got, which makes a missed occurrence irrelevant — the next successful run simply searches a longer interval. What is left is making sure that longer interval does not arrive all at once.

Prerequisites

• A STAC endpoint and an area of interest. The examples query Earth Search over four MGRS tiles covering a study area; any STAC API with collections, bbox and datetime works unchanged.
• The NDVI state machine from the parent recipe, taking {scene_id, red, nir, tile_grid} and fanning out one execution of the per-tile NDVI kernel per window.
• DynamoDB table ndvi_scene_ledger, partition key pk (string). Scene records use SCENE#<id>; the watermark lives in the single item pk = WATERMARK#<aoi>.
• IAM: the schedule’s role needs lambda:InvokeFunction on the discovery function; the discovery function needs dynamodb:PutItem, dynamodb:GetItem and states:StartExecution.
• Reserved concurrency on the tile worker. 200 is the figure used below; see reserved concurrency for geospatial Lambda fan-out for choosing it against the 1,000 default regional quota.
• Environment variables:
  code Copy

  STAC_URL=https://earth-search.aws.element84.com/v1/search
  STAC_COLLECTION=sentinel-2-l2a
  AOI_ID=fields-nl-01
  AOI_BBOX=4.02,51.72,5.44,52.31
  MAX_CLOUD_COVER=40
  MAX_SCENES_PER_RUN=4
  LEDGER_TABLE=ndvi_scene_ledger
  STATE_MACHINE_ARN=arn:aws:states:eu-west-1:...:stateMachine:ndvi-tiling
  

The schedule

EventBridge Scheduler, rather than a classic EventBridge rule, because a schedule is a first-class resource with its own timezone, its own retry policy and its own dead-letter queue, and because FlexibleTimeWindow lets the platform spread invocations rather than firing thousands of schedules on the same second.

resource "aws_scheduler_schedule" "weekly_ndvi" {
  name        = "ndvi-refresh-${var.aoi_id}"
  group_name  = "geo-pipelines"
  description = "Weekly Sentinel-2 NDVI discovery for ${var.aoi_id}"

  # Monday 03:00 local. Only cron honours the timezone, so this tracks DST;
  # a rate() expression would drift an hour twice a year.
  schedule_expression          = "cron(0 3 ? * MON *)"
  schedule_expression_timezone = "Europe/Amsterdam"

  # Let Scheduler pick any minute in a 15-minute window. Nothing downstream
  # cares about the exact second, and the spread avoids a thundering herd
  # when several AOIs share the schedule group.
  flexible_time_window {
    mode                      = "FLEXIBLE"
    maximum_window_in_minutes = 15
  }

  target {
    arn      = aws_lambda_function.ndvi_discovery.arn
    role_arn = aws_iam_role.scheduler.arn
    input    = jsonencode({ aoi_id = var.aoi_id })

    # Retries the INVOCATION of this one occurrence — not the schedule, and
    # not the discovery logic. After 6 hours it goes to the DLQ instead.
    retry_policy {
      maximum_retry_attempts       = 5
      maximum_event_age_in_seconds = 21600
    }

    dead_letter_config {
      arn = aws_sqs_queue.schedule_dlq.arn
    }
  }
}

The dead-letter queue is the only signal that a weekly schedule failed to deliver at all. Alarm on its ApproximateNumberOfMessagesVisible — a silent weekly job that stopped firing is otherwise indistinguishable from a week with no new scenes.

Implementation

The discovery function does four things in order: read the watermark, search STAC forward from it, filter, and dispatch under a cap.

# discovery.py — weekly STAC search, ledger filter, capped fan-out.
import json
import os
from datetime import datetime, timedelta, timezone

import boto3
import urllib3

ddb = boto3.client("dynamodb")
sfn = boto3.client("stepfunctions")
http = urllib3.PoolManager()

TABLE = os.environ["LEDGER_TABLE"]
AOI = os.environ["AOI_ID"]
BBOX = [float(v) for v in os.environ["AOI_BBOX"].split(",")]
MAX_CLOUD = float(os.environ["MAX_CLOUD_COVER"])
MAX_SCENES = int(os.environ["MAX_SCENES_PER_RUN"])
TILE_PX = 2048           # window edge; a 10,980 px band gives a 6 x 6 grid
BAND_PX = 10980          # Sentinel-2 L2A 10 m band edge


def _watermark() -> str:
    """Where the last successful run got to — NOT 'now minus seven days'."""
    item = ddb.get_item(TableName=TABLE,
                        Key={"pk": {"S": f"WATERMARK#{AOI}"}}).get("Item")
    if item:
        return item["datetime"]["S"]
    # First ever run: seed one revisit cycle back so we do not scan the archive.
    return (datetime.now(timezone.utc) - timedelta(days=10)).isoformat()


def _search(since: str) -> list[dict]:
    """Every scene published since the watermark, oldest first."""
    body = {
        "collections": [os.environ["STAC_COLLECTION"]],
        "bbox": BBOX,
        # Open-ended range: after a three-week outage this returns three
        # weeks of scenes, which is exactly the catch-up behaviour we want.
        "datetime": f"{since}/..",
        "sortby": [{"field": "properties.datetime", "direction": "asc"}],
        "limit": 100,
    }
    resp = http.request("POST", os.environ["STAC_URL"],
                        body=json.dumps(body).encode(),
                        headers={"Content-Type": "application/json"})
    return json.loads(resp.data)["features"]


def _claim(scene_id: str, reason: str = "dispatched") -> bool:
    """Atomically claim a scene. False means someone already has it."""
    try:
        ddb.put_item(
            TableName=TABLE,
            Item={"pk": {"S": f"SCENE#{scene_id}"},
                  "aoi": {"S": AOI},
                  "reason": {"S": reason},
                  "claimed_at": {"S": datetime.now(timezone.utc).isoformat()}},
            ConditionExpression="attribute_not_exists(pk)",
        )
        return True
    except ddb.exceptions.ConditionalCheckFailedException:
        return False


def handler(event, context):
    since = _watermark()
    scenes = _search(since)

    dispatched, skipped, clouded, newest = [], 0, 0, since
    for item in scenes:
        scene_id = item["id"]
        cloud = item["properties"].get("eo:cloud_cover", 0.0)

        if cloud > MAX_CLOUD:
            # Record it so a later run never re-evaluates the same scene.
            if _claim(scene_id, reason=f"cloud={cloud:.0f}"):
                clouded += 1
            newest = max(newest, item["properties"]["datetime"])
            continue

        if not _claim(scene_id):
            skipped += 1                     # already tiled by an earlier run
            newest = max(newest, item["properties"]["datetime"])
            continue

        if len(dispatched) >= MAX_SCENES:
            # Cap reached: release the claim so the NEXT run picks it up, and
            # stop advancing the watermark past this scene.
            ddb.delete_item(TableName=TABLE,
                            Key={"pk": {"S": f"SCENE#{scene_id}"}})
            break

        grid = -(-BAND_PX // TILE_PX)        # 6 windows per axis, 36 per scene
        sfn.start_execution(
            stateMachineArn=os.environ["STATE_MACHINE_ARN"],
            # Deterministic name: a retried dispatch collides instead of
            # starting a second identical run over the same scene.
            name=f"ndvi-{scene_id}"[:80],
            input=json.dumps({
                "scene_id": scene_id,
                "red": item["assets"]["red"]["href"],
                "nir": item["assets"]["nir"]["href"],
                "tile_grid": {"cols": grid, "rows": grid, "tile_px": TILE_PX},
            }),
        )
        dispatched.append(scene_id)
        newest = max(newest, item["properties"]["datetime"])

    # Advance only as far as we actually got. A capped run leaves the rest
    # of the backlog in front of the watermark for the following week.
    ddb.put_item(TableName=TABLE,
                 Item={"pk": {"S": f"WATERMARK#{AOI}"},
                       "datetime": {"S": newest}})

    return {"found": len(scenes), "dispatched": dispatched,
            "already_processed": skipped, "too_cloudy": clouded,
            "watermark": newest, "backlog": max(0, len(scenes) - MAX_SCENES)}

Two details carry most of the weight. The datetime range is open-ended ("{since}/..") rather than a seven-day box, so the search interval is defined by how far behind the pipeline is, not by how often the schedule fires. And the execution name is derived from the scene id, so Step Functions itself becomes the second line of defence: a retried dispatch of an already-running scene fails with ExecutionAlreadyExists rather than duplicating work — the same deterministic-naming discipline described in deterministic job IDs from object URI and ETag.

The catch-up problem

Nothing in a weekly schedule is prepared for the week it does not run. If the discovery function is broken for three weeks — a bad deploy, an expired STAC credential, an IAM change — Scheduler fires three times into failure, exhausts its retries, drops three messages in the dead-letter queue, and moves on. The occurrences are gone. What survives is the watermark, still sitting three weeks in the past, and the moment discovery works again the next search returns every scene since.

That is the desired behaviour and the dangerous one. Eleven scenes over four MGRS tiles is an ordinary three-week haul; at 36 windows each that is 396 tile workers wanting to start at the same moment. Against a reserved concurrency of 200 the second half of them throttle, Step Functions retries them, and the run either takes far longer than it should or trips the failure tolerance discussed in the 10 GB GeoTIFF recipe’s Map state. Capping dispatch at four scenes per run holds the burst to 144 concurrent workers and lets three subsequent runs drain the rest.

If waiting three more weeks for the backlog to clear is unacceptable, do not raise the cap — add a temporary one-time schedule (at(2026-08-08T09:00:00)) or simply invoke the discovery function again by hand. Each invocation dispatches another four scenes and advances the watermark; the ledger makes repeated invocation safe.

Verification

Backdate the watermark to simulate a three-week outage and confirm the run caps itself rather than dispatching everything.

# Pretend the last successful run was three weeks ago
aws dynamodb put-item --table-name ndvi_scene_ledger \
  --item '{"pk":{"S":"WATERMARK#fields-nl-01"},"datetime":{"S":"2026-07-13T00:00:00Z"}}'

aws lambda invoke --function-name ndvi-discovery \
  --payload '{"aoi_id":"fields-nl-01"}' --cli-binary-format raw-in-base64-out \
  /dev/stdout | head -c 600

# And confirm the schedule itself is configured as intended
aws scheduler get-schedule --name ndvi-refresh-fields-nl-01 --group-name geo-pipelines \
  --query '{expr:ScheduleExpression, tz:ScheduleExpressionTimezone, win:FlexibleTimeWindow}'

Expected output:

{"found": 11, "dispatched": ["S2B_31UFU_20260714_0_L2A", "S2A_31UFU_20260719_0_L2A",
 "S2B_31UFU_20260724_0_L2A", "S2A_31UFU_20260729_0_L2A"],
 "already_processed": 0, "too_cloudy": 3, "watermark": "2026-07-29T10:41:19Z",
 "backlog": 7}
{"expr": "cron(0 3 ? * MON *)", "tz": "Europe/Amsterdam",
 "win": {"MaximumWindowInMinutes": 15, "Mode": "FLEXIBLE"}}

Four dispatched, seven still in front of the watermark, and three cloudy scenes recorded so they are never re-evaluated. Invoke the function a second time immediately: dispatched should hold the next four scene ids and already_processed should be zero, because the first four are behind the watermark rather than being re-searched. If already_processed climbs instead, the watermark is not advancing and the ledger is doing all the work — correct, but it means every run re-searches the whole backlog.

Gotchas and Edge Cases

• A flexible time window is not a delay you can rely on. MaximumWindowInMinutes: 15 means the schedule fires somewhere inside that window, so two schedules that must run in order cannot both be flexible. Chain them from the first target instead, or set Mode: "OFF" on the one that has to be first.
• rate() ignores the timezone field. Only cron expressions accept ScheduleExpressionTimezone, and only they track daylight saving. A rate(7 days) schedule counts from creation and slowly walks away from local Monday morning, which matters if a report consumes the output.
• The retry policy retries delivery, not your logic. If the discovery function returns successfully having dispatched nothing because STAC returned an error it swallowed, Scheduler sees a successful invocation and nothing retries. Let genuine failures raise so the invocation fails; a function that catches everything and returns {"ok": false} has disabled the retry policy and the dead-letter queue in one line.
• Cloudy scenes must be recorded, not ignored. Skipping a scene without writing a ledger entry means every subsequent run re-fetches and re-evaluates it for as long as it sits behind the watermark. Record it with its reason; the ledger doubles as an audit of why a date has no NDVI product.
• The watermark must advance to the newest scene examined, not the newest dispatched — except at the cap. The code above advances past cloudy and already-claimed scenes but stops at the cap. Advancing past a scene that was never dispatched and never recorded loses it permanently, which is the one failure mode this design cannot recover from.

Frequently Asked Questions

Does EventBridge Scheduler run a missed schedule after an outage?

No. Scheduler does not backfill occurrences missed while a target was unavailable. Its retry policy retries the invocation of a single occurrence — up to 185 attempts, within a MaximumEventAgeInSeconds of at most 86,400 — and then routes it to the dead-letter queue. Catch-up therefore has to be a property of the work being triggered, which is why discovery searches forward from a stored watermark rather than a fixed seven-day lookback.

How do I stop a scene being tiled twice?

Claim it before working on it. A conditional PutItem with attribute_not_exists(pk) is atomic, so if two overlapping runs both see the same STAC item exactly one write succeeds and the loser gets a ConditionalCheckFailedException and skips. That single property makes the discovery function safe to invoke as often as you like, which in turn makes manual catch-up runs trivial.

Why cap the number of scenes dispatched per run?

Because a backlog is a multiplier, not a queue of one. A Sentinel-2 L2A 10 m band is 10,980 × 10,980 pixels, which tiles into 36 windows of 2,048 pixels, so eleven backlogged scenes want 396 tile workers at once. Capping at four scenes holds the burst to 144 workers, inside a reserved concurrency of 200, and lets the next three runs drain the rest without throttling anything else in the account.

Should the schedule use rate or cron?

Use cron whenever the wall-clock time matters — to a human, to a report, or to a provider’s publication cycle — because only cron accepts ScheduleExpressionTimezone and therefore follows daylight saving. rate() counts from the schedule’s creation time and drifts against local time, which is perfectly fine for an internal refresh and wrong for anything expected to have landed before Monday morning.

Related

• Serverless NDVI Tiling from Sentinel-2 Imagery — the pipeline this schedule triggers, and where the 36-window grid comes from
• Computing NDVI per Tile with Rasterio and NumPy — the per-window kernel each dispatched scene runs 36 times
• Idempotency and Exactly-Once Spatial Processing — the ledger and deterministic-name patterns behind the scene claim
• Reserved Concurrency for Geospatial Lambda Fan-Out — sizing the 200-worker pool the catch-up cap is measured against
• Batch vs Stream Geospatial Processing — why a satellite refresh is a scheduled batch and a vessel position is not

Back to Serverless NDVI Tiling from Sentinel-2 Imagery