Timeout Ceiling Comparison for Long-Running Geospatial Jobs

Serverless functions terminate at a hard wall: AWS Lambda at 15 minutes, GCP Cloud Functions 2nd gen at 60 minutes, and Azure Functions at 10 minutes on Consumption (60 on Premium) — and the platform kills the invocation at that instant regardless of how much of your DEM mosaic or reprojection is complete. For geospatial workloads, where a single global reprojection or ML inference pass routinely runs longer than 15 minutes, that ceiling is the deciding factor between running a job in one invocation, decomposing it into tiles, or offloading it to a container runtime entirely. This page, part of Serverless Geospatial Architecture & Platform Limits, compares all three ceilings against real raster workloads and maps each to the right execution strategy.

Why the Timeout Ceiling Matters for Geospatial Workloads

Geospatial operations are among the few serverless workloads that genuinely bump the timeout wall. Reprojecting a continental-scale raster, mosaicking hundreds of DEM tiles into a seamless surface, or running a segmentation model across a full Sentinel-2 scene are all minutes-to-tens-of-minutes operations even at full memory allocation. Because CPU scales with memory on both AWS and GCP, throwing more memory and CPU at the raster workload buys speed — but only up to the point where the operation is single-threaded or I/O bound on a remote COG, after which no allocation saves you from the wall.

The ceiling therefore forces an architectural decision at design time, not a tuning decision at runtime. A job whose worst-case input finishes in four minutes fits comfortably in a single Lambda invocation. A job that takes 40 minutes on its largest input cannot run on Lambda or Azure Consumption at all — it must be decomposed into tile-sized units orchestrated by a state machine, or handed to a container. This is the same partitioning logic behind the 10 GB GeoTIFF processing recipe with Step Functions, where a scene too large for one invocation is split into a map of tiles. It also interacts with the ephemeral-storage ceiling: a job that writes large intermediate VRTs can hit the /tmp storage limit before it hits the timeout, and both must be respected together.

Three workload archetypes illustrate why the ceiling bites differently. DEM mosaicking stitches many adjacent tiles into a seamless surface; the total is large but the work is embarrassingly parallel, so it tiles cleanly and is a poor fit for a single long invocation regardless of ceiling. Global reprojection transforms every pixel of a continental raster into a new CRS; it is parallel per tile but each tile carries the full PROJ pipeline cost, so tile sizing has to account for transform density, not just pixel count. ML inference over satellite imagery is the awkward case — a model that must see a wide spatial context per prediction resists naive tiling, because splitting the raster splits the receptive field. That archetype is the one most likely to exceed even GCP’s 60-minute window intact and land on a container runtime. Knowing which archetype you are dealing with tells you in advance whether the answer is a smaller tile, a longer ceiling, or an offload — before you write a line of orchestration.

The cold-start dimension interacts here too: a chunked job pays the GDAL initialization cost once per concurrent worker, so a fan-out of 40 tiles is 40 potential cold starts. That is usually acceptable against a multi-minute job, but it is a real tax that a single long invocation avoids — one more reason the 60-minute window is sometimes the pragmatic choice even when tiling is technically possible.

Platform-by-Platform Limits

The table pairs each provider’s execution ceiling with the geospatial operations it can and cannot host in a single invocation, plus the native offload target when the ceiling is exceeded.

The practical reading: AWS Lambda’s 15-minute wall makes chunking mandatory for any continental raster, which is why the per-tile pattern below is the default AWS architecture. GCP’s 60-minute window absorbs many single-pass jobs but offers no partial-failure isolation. Azure Consumption’s 10-minute ceiling is the tightest of the three — most heavy raster work there either moves to Premium or offloads to Container Apps. When a burst of per-tile invocations fans out, route them through SQS and Pub/Sub queues so a spike does not saturate concurrency and stall the very tiles racing the clock.

Step-by-Step Implementation

Step 1: Profile the Worst-Case Single Invocation

Never size against the average input. Measure the runtime of your largest realistic scene at production memory, and record where the time actually goes.

# profile_runtime.py — run against your worst-case input, not a sample tile
import time, rasterio
from rasterio.warp import calculate_default_transform, reproject, Resampling

def reproject_scene(src_path: str, dst_crs: str = "EPSG:3857") -> float:
    t0 = time.perf_counter()
    with rasterio.open(src_path) as src:
        transform, w, h = calculate_default_transform(
            src.crs, dst_crs, src.width, src.height, *src.bounds
        )
        # Reproject every band — this is the CPU-bound stretch that eats the clock
        for b in range(1, src.count + 1):
            reproject(
                source=rasterio.band(src, b),
                destination=f"/tmp/out_b{b}.tif",
                dst_crs=dst_crs, dst_transform=transform,
                resampling=Resampling.bilinear,
            )
    return round(time.perf_counter() - t0, 1)

# If this returns > 630 s (70% of the 900 s Lambda ceiling), you must chunk.

Step 2: Decide Chunk vs Single Invocation

Apply a 70% headroom rule against the target ceiling. On Lambda, if the worst case exceeds ~630 s of the 900 s limit, decompose. The chunking raster jobs to fit the 15-minute Lambda ceiling guide covers the tile-sizing and checkpoint pattern that keeps each unit of work comfortably under 900 seconds.

Step 3: Orchestrate the Fan-Out with a State Machine

When chunking, a Step Functions Map state issues one invocation per tile and retries only failures — the partial-failure isolation a single 60-minute invocation cannot give you.

{
  "Comment": "Tiled reprojection under the 15-minute ceiling",
  "StartAt": "GenerateTiles",
  "States": {
    "GenerateTiles": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:tile-planner",
      "Next": "ProcessTiles"
    },
    "ProcessTiles": {
      "Type": "Map",
      "ItemsPath": "$.tiles",
      "MaxConcurrency": 40,
      "Iterator": {
        "StartAt": "ReprojectTile",
        "States": {
          "ReprojectTile": {
            "Type": "Task",
            "Resource": "arn:aws:lambda:us-east-1:123456789012:function:reproject-tile",
            "Retry": [{"ErrorEquals": ["States.TaskFailed"], "MaxAttempts": 3, "BackoffRate": 2.0}],
            "End": true
          }
        }
      },
      "End": true
    }
  }
}

Step 4: Offload When Chunking Cannot Help

Some jobs resist tiling — a whole-basin hydrological flow-accumulation needs the entire raster in memory at once. Route those to a container runtime with no timeout wall. On AWS, launch a Fargate task; the equivalent targets are Cloud Run jobs (GCP) and Container Apps jobs (Azure).

# offload_to_fargate.py — hand a monolithic raster job to a container
import boto3
ecs = boto3.client("ecs")

def run_batch_job(input_uri: str) -> str:
    resp = ecs.run_task(
        cluster="geo-batch",
        launchType="FARGATE",
        taskDefinition="dem-mosaic:12",       # 8 vCPU / 32 GB task, no 15-min wall
        overrides={"containerOverrides": [
            {"name": "mosaic", "environment": [{"name": "INPUT_URI", "value": input_uri}]}
        ]},
    )
    return resp["tasks"][0]["taskArn"]

Measurement and Verification

Confirm no invocation is silently brushing the ceiling. On AWS, a job that hits the wall is terminated with Task timed out after 900.00 seconds in CloudWatch — search for it explicitly, because a timeout can masquerade as a generic failure in the orchestrator.

# Flag any invocation that ran past 80% of the 900s ceiling
aws logs filter-log-events \
  --log-group-name /aws/lambda/reproject-tile \
  --filter-pattern '"Task timed out"' \
  --query 'events[].message'

Track Duration as a fraction of the configured timeout per stage; a p99 above 80% of the ceiling means the next larger input will time out. For GCP, watch the Cloud Run request latency distribution against the 3,600 s cap; for Azure, alert on the FunctionExecutionTimeMs metric approaching the plan limit. The healthy signal is a p99 duration comfortably below the wall with the retry count at zero.

Failure Modes and Debugging

Task timed out after 900.00 seconds (AWS): The single invocation exceeded the Lambda ceiling. Either the input grew past your chunk sizing or a per-tile operation is doing more work than planned. Re-profile the worst case and shrink the tile, or offload.

Silent partial output with no error: A 60-minute GCP invocation was killed mid-write at the ceiling, leaving a truncated COG. Single long invocations have no checkpoint — add per-band flushing or move to chunked execution so partial progress is durable.

Retry storm on chunk failures: A Map state retried failed tiles without backoff and saturated concurrency, so healthy tiles then timed out waiting for capacity. Add exponential backoff and cap MaxConcurrency below the regional quota.

Azure Consumption job never completes: The operation needs more than 10 minutes and Consumption cannot grant it. Move to the Premium plan for the 60-minute window or to a Container Apps job for unbounded runtime.

Offloaded container OOM-kills: The monolithic job that would not tile also does not fit the task’s memory. Right-size the Fargate/Cloud Run task memory to the full-raster footprint plus GDAL overhead before assuming the timeout was the problem.

Cost and Scaling Considerations

The timeout choice is also a cost choice. A chunked Lambda job with 40 concurrent tiles finishes in wall-clock minutes and bills only for actual per-tile GB-seconds, but carries orchestration overhead and per-invocation cold starts. A single 60-minute GCP invocation is operationally simpler and has one cold start, but bills for the full hour at the function’s memory tier even during I/O waits, and loses everything on a crash. Container offload trades per-second serverless granularity for a longer-lived task that is cheaper per unit of sustained compute but slower to start and always-on for the job’s duration.

The scaling rule of thumb: decompose when the workload is embarrassingly parallel across tiles, because fan-out turns a 40-minute serial job into a few minutes of wall-clock time. Offload when the operation is irreducibly monolithic. Reserve the 60-minute GCP or Azure Premium window for jobs that are almost short enough — where paying for a single longer invocation is cheaper than building and operating a state machine. For the largest recurring pipelines, the 10 GB GeoTIFF recipe shows the fan-out pattern end to end.

Frequently Asked Questions

What are the maximum execution times for serverless functions across the three clouds?

AWS Lambda caps a single invocation at 15 minutes. GCP Cloud Functions 2nd gen allows up to 60 minutes for HTTP functions. Azure Functions on the Consumption plan caps at 10 minutes, rising to 60 minutes on the Premium plan. These are hard ceilings — the platform terminates the invocation at the limit regardless of progress.

When must I chunk a geospatial job instead of running it in one invocation?

Chunk when the worst-case runtime of your largest expected input exceeds roughly 70% of the platform ceiling, because you need headroom for cold starts, network variance, and retries. Global DEM mosaicking and full-scene reprojection routinely exceed 15 minutes, so they must be tiled on AWS and Azure Consumption; a 60-minute GCP or Azure Premium window fits more single-pass jobs.

Where should I offload a geospatial job that cannot fit any function timeout?

Offload to a container or batch runtime: AWS Fargate or AWS Batch, GCP Cloud Run jobs, or Azure Container Apps jobs. These have no 15- or 60-minute cap and are the right home for monolithic operations like whole-country hydrological modelling that resist tiling.

Does GCP’s 60-minute ceiling make it always the better choice for heavy raster work?

Not automatically. A longer ceiling reduces orchestration complexity, but a single 60-minute invocation has no partial-failure isolation — one crash loses an hour of work. Chunked AWS jobs retry only the failed tile. The longer window is a convenience, not a substitute for decomposition on very large inputs.

Related

• Chunking Raster Jobs to Fit the 15-Minute AWS Lambda Ceiling — tile sizing and checkpoint/resume for partial completion
• Process a 10 GB GeoTIFF with Step Functions and Lambda — the full fan-out recipe for scenes too big for one invocation
• Memory and CPU Allocation for Raster Workloads — buying speed with memory before you hit the wall
• SQS and Pub/Sub Queue Routing Strategies — keeping per-tile fan-out within concurrency quotas
• Ephemeral Storage Limits in AWS Lambda — the /tmp ceiling that long jobs hit alongside the timeout

Back to Serverless Geospatial Architecture & Platform Limits