Cost Modelling for Serverless Raster Pipelines

On a realistic 6,400-tile run over a single 10 GB GeoTIFF, AWS Lambda GB-seconds account for 35% of the bill — $1.69 of $4.83 — while NAT Gateway data processing, CloudWatch Logs ingestion and Step Functions state transitions account for 63% between them. Tune those three and the same scene, the same 6,400 invocations and the same handler code cost $1.78, with compute now at 95% of the total. Every price quoted on this page is a us-east-1 on-demand list price at the time of writing; wherever a ratio can carry the argument instead of a dollar figure, it does, because the ratios survive a price change and the dollars do not.

Cost modelling for raster work is not a finance exercise bolted on after the pipeline runs. It is a design input, because the two decisions that dominate the bill — the memory tier and the concurrency model — are the same two decisions that dominate throughput and reliability. This page builds the model from the per-tile equation upwards, shows where the real money hides, compares the billing structures on AWS, GCP and Azure, and finishes with an end-to-end costing you can reproduce.

Why Cost Structure Differs for Geospatial Workloads

A generic serverless API handler runs for 40 ms, reads nothing large, and writes a few hundred bytes. Its cost is dominated by the per-request charge and nothing else matters. A raster tile worker inverts every one of those properties, and each inversion moves a different line item to the top of the bill.

Invocations are long, so GB-seconds dominate the request charge. At 5.4 seconds and 3,008 MB, one tile costs $0.00026438 in compute and $0.00000020 in request charges — a ratio of 1,322 to 1. Advice tuned for short handlers (“batch your invocations to save on requests”) is worthless here.

Invocations are I/O heavy, so the per-request S3 charge becomes visible. A windowed read of a Cloud Optimized GeoTIFF is not one GET. It is a header read, an IFD read, an overview probe and one range request per band block — and more again if GDAL_DISABLE_READDIR_ON_OPEN is unset and GDAL lists the prefix and probes for sidecar files first. The gap between a tuned six-GET read and an untuned twenty-two-GET read is a 3.7× multiplier on a line item most teams never look at, and tuning HTTP range requests for COG reads on S3 is where it gets removed.

The data volume is large enough that the network path has a price. Twenty-five gigabytes moving between a function and object storage costs nothing over an S3 gateway endpoint and $1.13 over a NAT Gateway. Neither number appears in any Lambda metric.

Fan-out is wide, so anything charged per unit of orchestration multiplies by the tile count. A Step Functions Standard workflow running a Map iteration per tile bills about four state transitions per tile. At 6,400 tiles that is 25,600 transitions and $0.64 — 38% of the compute cost, spent entirely on coordination.

Logs are proportional to invocations, not to traffic. A GDAL handler with CPL_DEBUG=ON emits a few hundred kilobytes per invocation, and a wide fan-out turns CloudWatch Logs ingestion at $0.50 per GB into the second-largest line on the bill.

A raster cost model therefore needs at least six terms. A model with only the GB-second term understates a real pipeline by roughly 2.7×.

The Cost-per-Tile Equation

Start with the term everyone knows and then add the ones that actually change the answer. For a single AWS Lambda invocation:

cost_per_tile
  = (memory_mb / 1024) * duration_s * PRICE_GB_SECOND      # compute
  + PRICE_PER_REQUEST                                       # invocation
  + max(0, (tmp_mb - 512) / 1024) * duration_s * PRICE_TMP_GB_SECOND
  + get_requests * PRICE_S3_GET
  + put_requests * PRICE_S3_PUT
  + bytes_through_nat_gb * PRICE_NAT_GB
  + log_bytes_gb * PRICE_LOG_INGEST_GB

With the us-east-1 x86 on-demand list prices at the time of writing:

Two ratios in that table set the shape of every raster cost model and survive any price change. The ephemeral-storage rate is roughly 1/540th of the memory rate, so provisioning /tmp is nearly free relative to the RAM you are already buying — even the full 10,240 MB for a five-second invocation adds about $0.0000015. Teams that leave /tmp at the default and stage GeoTIFF intermediates in memory instead are raising the memory tier to avoid a charge that does not exist; the guidance in Ephemeral Storage Limits in AWS Lambda is free to follow. And a PUT costs 12.5× a GET, so for a job that reads six ranges and writes one object per tile the single PUT is a third of the S3 request cost — writing per-band tiles instead of one interleaved tile triples that line, which is an argument for the merge strategy in merging tiled Lambda outputs into a COG.

The full arithmetic for one tile, with every intermediate value shown and a runnable script, is worked through in cost-per-tile math for Lambda raster jobs.

Why a Bigger Memory Setting Is Often Cheaper

Lambda couples vCPU to memory at one full vCPU per 1,769 MB, and bills GB-seconds. If duration scaled perfectly with vCPU, GB-seconds would be constant and cost would be flat across the whole memory range. It does not scale perfectly in either direction, and the two deviations pull in opposite directions.

Below about 3 GB the deviation is severe and specific to GDAL. GDAL_CACHEMAX defaults to 5% of available memory, so a 1,024 MB function gets a 51 MB block cache. A 512-pixel window across four bands with overview access does not fit, the cache evicts blocks that are about to be needed again, and the same byte ranges are fetched from S3 two or three times. Duration rises faster than linearly as memory falls, and the extra GETs arrive as a second, smaller penalty. Above about 5 GB the deviation reverses: the serial fraction of the tile — a single-threaded deflate encode, the PUT, the Python interpreter’s own bookkeeping — does not shrink with more cores, so duration flattens while the per-second price keeps climbing.

The measured minimum for this workload shape sits at 3,008 MB. Moving from 1,024 MB to 3,008 MB cuts the bill by 26% and makes each tile four times faster — the rare change that is unambiguously correct. Moving from 3,008 MB to 10,240 MB raises the bill by 51% and makes each tile 2.25× faster, which is a genuine trade you might take when a backfill has a deadline. The 10 GB memory configuration guide covers the mechanics; the point here is that you should never pick the top tier by default and never pick the bottom one either.

Sweep rather than guess: a three-point run at 1,769 / 3,008 / 5,308 MB over fifty real tiles costs a few cents and settles the argument. Set GDAL_CACHEMAX explicitly first, or the sweep measures the 5% default rather than the memory tier — the sizing rationale is in Memory and CPU Allocation for Raster Workloads. And re-sweep when the input changes, because a switch from uint16 to float32, or from LZW to ZSTD, moves the minimum by a tier or more.

The Costs That Actually Dominate

Compute is the line item you control from the handler, which is why it gets all the attention and why it is rarely the problem. The four charges below are all set outside the function code and all of them can exceed it.

NAT Gateway data processing

A Lambda function attached to a VPC — usually because it needs to reach an RDS instance or a private PostGIS database — routes its S3 traffic through a NAT Gateway unless a gateway endpoint exists for S3. At $0.045 per GB processed, a pipeline reading 15 GB and writing 10 GB pays $1.13 per scene for network that would otherwise be free. An S3 gateway VPC endpoint costs nothing per gigabyte and takes one Terraform resource. It is the largest avoidable line item in serverless raster processing, and it is invisible in every Lambda-scoped dashboard because the charge lands under EC2.

CloudWatch Logs ingestion

Ingestion is billed at $0.50 per GB. A handler that logs one line per window at INFO produces around 6 KB per invocation and $0.019 across 6,400 tiles — irrelevant. Turn on CPL_DEBUG=ON to diagnose a range-read problem, forget to turn it off, and the same run produces 400 KB per invocation, 2.56 GB in total and $1.28 — three-quarters of the compute cost. Structured logging helps only if you also cap its volume: one record per invocation with the fields you actually query on, and GDAL’s debug channel behind a flag that defaults to off.

Step Functions state transitions

Standard workflows bill $0.025 per 1,000 state transitions, and a Map state with a retry policy and a catch branch spends about four per iteration — $0.64 of pure coordination across 6,400 tiles. A Distributed Map with itemBatcher grouping fifty tiles per child execution, and Express children instead of Standard, collapses that to roughly $0.02 with identical semantics. The item-reader and batching mechanics are in partitioning a GeoTIFF into Step Functions Map tiles, and process a 10 GB GeoTIFF with Step Functions and Lambda assumes the batched form throughout. Where the fan-out is a queue instead, SQS and Pub/Sub queue routing strategies removes the per-transition charge entirely at the price of owning your own progress tracking.

Per-request S3 charges on range reads

Twenty-two GETs per tile instead of six is a 3.7× multiplier on a $0.056 line — small in isolation, and it matters because the same misconfiguration that causes it also causes a duration penalty. GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR, GDAL_PAM_ENABLED=NO and CPL_VSIL_CURL_ALLOWED_EXTENSIONS=.tif remove the prefix listing, the sidecar probes and the speculative reads. The S3 saving is a rounding error; the 0.9 seconds of latency they take off every tile is not.

Platform-by-Platform Billing Models

The three providers charge for genuinely different things, and the difference is structural rather than a matter of rate.

Three observations that outlast any price change:

The effective price of a vCPU-second is nearly identical on all three. On Lambda you buy 1,769 MB with each vCPU, so a vCPU-second costs $0.0000166667 × 1.728 = $0.0000288. On Cloud Run at 2 GiB per vCPU it is $0.000024 + 2 × $0.0000025 = $0.0000290. On Azure Consumption at the 1,536 MB ceiling it works out to about $0.0000276. Within 5%. Any cost argument that turns on the rate card is measuring noise.

Only GCP clears both raster ceilings without chunking. Cloud Functions 2nd gen offers 60 minutes and 32,768 MB, and Cloud Run 60 minutes and 32 GiB, against AWS’s 15 minutes and 10,240 MB. Chunking is usually the better design anyway — see chunking raster jobs to fit the 15-minute Lambda ceiling and the boundary drawn in Timeout Ceiling Comparison for Long-Running Geospatial Jobs.

Azure Functions on the Consumption plan cannot reach the cheap part of the curve. At 1,536 MB it sits below the memory tier where a four-band raster job stops thrashing, and there is no higher Consumption tier to move to. Its rate is the lowest of the three and its bill is not, because you pay that rate for twice as many seconds. The side-by-side numbers are in comparing the cost of Lambda, Cloud Run and Azure Functions for tiling.

When a Container or a Batch Job Wins

Per-tile Lambda charges you for wall clock, and a COG tile spends most of its wall clock waiting on the network rather than computing. That wait is 57% of a warm 512-pixel tile in the assumption set used throughout this page, and during all of it the function holds 3,008 MB of idle RAM that you are paying for.

A container runtime with in-process concurrency reclaims exactly that fraction. Cloud Run serving eight concurrent requests per instance overlaps eight tiles’ range reads against each other, so the same 6,400 tiles need roughly 19,200 vCPU-seconds instead of Lambda’s 58,700 — and the bill falls from $1.69 to $0.56, a factor of three. Set --concurrency 1 and the advantage vanishes; the container becomes 24% more expensive than Lambda because you now pay for container startup and the scale-down tail as well. The same pattern applies to ECS and to generating MVT tiles with Tippecanoe in Cloud Run, where the per-instance concurrency is the whole reason the vector pipeline is affordable.

AWS Batch on Fargate sits further along the same axis. Fargate on-demand costs about $0.0000112 per vCPU-second against Lambda’s effective $0.0000288 — 2.6× cheaper — and Fargate Spot is roughly 70% below on-demand again, putting it near 17× cheaper for the same work. Against that you take on queue depth, scaling policy, image pull time and Spot interruption handling. Batch is the right answer for a scheduled backfill of ten thousand scenes and the wrong answer for four scenes a day, where the scheduler costs more in engineering time than the entire compute bill.

The decision rule has nothing to do with which provider you are on. Measure the fraction of tile wall clock spent in object-storage I/O; above 50%, in-process concurrency is worth more than any rate difference. Check whether one unit of work fits in 15 minutes and 10,240 MB; if not, you chunk or you leave Lambda. Check whether arrivals are bursty, because Lambda charges nothing between scenes and a warm container pool does. And check the retry cost: a failed 5.4-second tile costs $0.00027 to redo, a failed 40-minute container job about 450× that.

Implementation: A Cost Model You Can Run

The model below is the one used for every number on this page. It takes the platform rates as explicit constants so a price change is a one-line edit, and it separates per-tile terms from per-scene terms so the fixed orchestration costs do not get smeared across tiles.

# raster_cost_model.py — deterministic cost model for a serverless tiling run.
# All rates are us-east-1 on-demand list prices at the time of writing.
from dataclasses import dataclass, field

# --- Rate card -------------------------------------------------------------
PRICE_GB_SECOND      = 0.0000166667   # Lambda x86 on-demand, per GB-second
PRICE_REQUEST        = 0.20 / 1e6     # Lambda, per invocation
PRICE_TMP_GB_SECOND  = 0.0000000309   # ephemeral storage above the 512 MB default
PRICE_S3_GET         = 0.0004 / 1000  # per GET / range request
PRICE_S3_PUT         = 0.005 / 1000   # per PUT
PRICE_NAT_GB         = 0.045          # NAT Gateway data processing, per GB
PRICE_LOG_GB         = 0.50           # CloudWatch Logs ingestion, per GB
PRICE_SFN_TRANSITION = 0.025 / 1000   # Step Functions Standard, per state transition

TMP_FREE_MB = 512                     # /tmp included with every function


@dataclass
class TileProfile:
    """One invocation's measured behaviour. Every field comes from a real run."""
    memory_mb: int
    duration_s: float
    tmp_mb: int = TMP_FREE_MB
    s3_gets: int = 6
    s3_puts: int = 1
    log_kb: float = 6.0
    bytes_over_nat_mb: float = 0.0     # 0.0 with an S3 gateway VPC endpoint

    def breakdown(self) -> dict:
        billable_tmp_gb = max(0, self.tmp_mb - TMP_FREE_MB) / 1024
        parts = {
            "compute": (self.memory_mb / 1024) * self.duration_s * PRICE_GB_SECOND,
            "request": PRICE_REQUEST,
            "ephemeral": billable_tmp_gb * self.duration_s * PRICE_TMP_GB_SECOND,
            "s3_requests": self.s3_gets * PRICE_S3_GET + self.s3_puts * PRICE_S3_PUT,
            "nat": (self.bytes_over_nat_mb / 1024) * PRICE_NAT_GB,
            "logs": (self.log_kb / 1_048_576) * PRICE_LOG_GB,
        }
        parts["total"] = sum(parts.values())
        return parts


@dataclass
class SceneProfile:
    """A whole scene: N tiles plus the orchestration that fans them out."""
    tile: TileProfile
    tile_count: int
    sfn_transitions_per_tile: float = 0.0   # 0.0 for SQS or a batched Distributed Map
    fixed_costs: dict = field(default_factory=dict)

    def breakdown(self) -> dict:
        per_tile = self.tile.breakdown()
        totals = {k: v * self.tile_count for k, v in per_tile.items() if k != "total"}
        totals["step_functions"] = (
            self.tile_count * self.sfn_transitions_per_tile * PRICE_SFN_TRANSITION
        )
        totals.update(self.fixed_costs)
        totals["total"] = sum(totals.values())
        return totals


def report(name: str, scene: SceneProfile) -> None:
    b = scene.breakdown()
    total = b.pop("total")
    print(f"\n{name}: ${total:.4f} over {scene.tile_count:,} tiles")
    for k, v in sorted(b.items(), key=lambda kv: -kv[1]):
        if v > 0:
            print(f"  {k:<16} ${v:>8.4f}  {v / total:>6.1%}")


if __name__ == "__main__":  # doctest-free, deterministic, no cloud calls
    # Untuned: VPC without an S3 endpoint, CPL_DEBUG on, per-tile Standard Map.
    untuned = SceneProfile(
        tile=TileProfile(
            memory_mb=3008, duration_s=5.4, tmp_mb=2048,
            s3_gets=22, s3_puts=1, log_kb=400.0, bytes_over_nat_mb=4.0,
        ),
        tile_count=6400,
        sfn_transitions_per_tile=4.0,
    )
    # Tuned: gateway endpoint, INFO logging, batched Distributed Map.
    tuned = SceneProfile(
        tile=TileProfile(
            memory_mb=3008, duration_s=5.4, tmp_mb=2048,
            s3_gets=6, s3_puts=1, log_kb=6.0, bytes_over_nat_mb=0.0,
        ),
        tile_count=6400,
        fixed_costs={"step_functions": 0.02},
    )
    report("Untuned", untuned)
    report("Tuned", tuned)

Running it prints the two ends of the worked example:

Untuned: $4.8303 over 6,400 tiles
  compute          $  1.6921  35.0%
  logs             $  1.2800  26.5%
  nat              $  1.1250  23.3%
  step_functions   $  0.6400  13.3%
  s3_requests      $  0.0883   1.8%
  ephemeral        $  0.0016   0.0%
  request          $  0.0013   0.0%

Tuned: $1.7835 over 6,400 tiles
  compute          $  1.6921  94.9%
  s3_requests      $  0.0474   2.7%
  step_functions   $  0.0200   1.1%
  logs             $  0.0192   1.1%
  ephemeral        $  0.0016   0.1%
  request          $  0.0013   0.1%

The infrastructure changes behind the tuned column

Three of the four savings are infrastructure, not code. In Terraform:

# 1. S3 gateway endpoint — removes the entire NAT data-processing charge.
resource "aws_vpc_endpoint" "s3" {
  vpc_id            = aws_vpc.pipeline.id
  service_name      = "com.amazonaws.us-east-1.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [aws_route_table.private.id]
}

# 2. Log retention — ingestion is charged once, storage every month.
resource "aws_cloudwatch_log_group" "tiler" {
  name              = "/aws/lambda/raster-tiler"
  retention_in_days = 14
}

# 3. GDAL flags that cut both the GET count and the duration.
resource "aws_lambda_function" "tiler" {
  function_name = "raster-tiler"
  runtime       = "python3.12"
  handler       = "handler.process_tile"
  memory_size   = 3008          # the measured minimum of the cost curve
  timeout       = 120
  architectures = ["x86_64"]    # switch to arm64 for ~20% off if the layer supports it

  ephemeral_storage { size = 2048 }

  environment {
    variables = {
      GDAL_CACHEMAX                     = "750"   # 25% of 3,008 MB
      GDAL_NUM_THREADS                  = "2"
      GDAL_DISABLE_READDIR_ON_OPEN      = "EMPTY_DIR"
      GDAL_PAM_ENABLED                  = "NO"
      CPL_VSIL_CURL_ALLOWED_EXTENSIONS  = ".tif"
      VSI_CACHE                         = "TRUE"
      VSI_CACHE_SIZE                    = "50000000"
      CPL_DEBUG                         = "OFF"   # never ship this ON
    }
  }
}

The fourth is the workflow shape: a Distributed Map with ItemBatcher and an Express child workflow, rather than a Standard Map iterating once per tile.

Measurement and Verification

A cost model is a hypothesis. These are the exact metrics that confirm or refute it.

Lambda. AWS/Lambda publishes Duration, Invocations and ConcurrentExecutions; billed duration, memory size and max memory used appear only in the REPORT log line, so parse them out or read them from the context object. Feed the model the p99 of Duration, not the average — a long tail on 5% of tiles moves the bill by more than most tuning does.

import boto3
from datetime import datetime, timedelta, timezone

cw = boto3.client("cloudwatch", region_name="us-east-1")

def billed_gb_seconds(function_name: str, memory_mb: int, hours: int = 24) -> float:
    """Reconstruct GB-seconds from Duration and Invocations."""
    end = datetime.now(timezone.utc)
    start = end - timedelta(hours=hours)
    stats = {}
    for metric, stat in (("Duration", "Sum"), ("Invocations", "Sum")):
        resp = cw.get_metric_statistics(
            Namespace="AWS/Lambda", MetricName=metric,
            Dimensions=[{"Name": "FunctionName", "Value": function_name}],
            StartTime=start, EndTime=end, Period=3600, Statistics=[stat],
        )
        stats[metric] = sum(p[stat] for p in resp["Datapoints"])
    gb_seconds = (memory_mb / 1024) * (stats["Duration"] / 1000)
    print(f"{stats['Invocations']:,.0f} invocations, {gb_seconds:,.1f} GB-s, "
          f"${gb_seconds * 0.0000166667:.4f} compute")
    return gb_seconds

Per-scene attribution. Cost allocation tags do not reach invocation granularity, so emit it yourself with the CloudWatch embedded metric format and the scene identifier as a dimension:

import json, time

def emit_cost_metric(scene_id: str, memory_mb: int, duration_ms: float) -> None:
    gb_seconds = (memory_mb / 1024) * (duration_ms / 1000)
    print(json.dumps({
        "_aws": {
            "Timestamp": int(time.time() * 1000),
            "CloudWatchMetrics": [{
                "Namespace": "RasterPipeline",
                "Dimensions": [["SceneId"]],
                "Metrics": [{"Name": "GbSeconds"}, {"Name": "TileCostUsd"}],
            }],
        },
        "SceneId": scene_id,
        "GbSeconds": gb_seconds,
        "TileCostUsd": gb_seconds * 0.0000166667,
    }))

The three charges outside Lambda’s namespace. NAT Gateway data processing is AWS/NATGateway → BytesOutToDestination and BytesInFromDestination; a non-zero value while a tiling run is in flight means no gateway endpoint is in the route table. CloudWatch Logs volume is AWS/Logs → IncomingBytes, filtered by log group. Step Functions transitions have no direct metric — divide the Cost Explorer StateTransition usage-type quantity by the number of executions and compare against your expected transitions per tile.

GCP and Azure. On Cloud Run, model from run.googleapis.com/container/billable_instance_time alongside container/cpu/utilizations and request_count; billable instance time counts wall time per instance, so it already reflects your concurrency setting. On Azure, FunctionExecutionUnits is reported in MB-milliseconds — divide by 1,024,000 for GB-seconds — and FunctionExecutionCount gives the execution charge.

Reconcile monthly against Cost Explorer grouped by usage type, not by service. Service-level grouping hides the NAT and Logs charges inside “EC2 — Other” and “CloudWatch”, which is exactly why they stay unnoticed.

Failure Modes and Debugging

The model says $1.78 and the bill says $4.83. Almost always NAT Gateway. Check AWS/NATGateway BytesOutToDestination during a run; if it tracks the scene size, the S3 gateway endpoint is missing from the private subnet’s route table. The second candidate is log volume from a debug flag left on after an incident.

Cost per tile rises without any code change. Duration drifted. The usual causes are a changed compression codec on the input, a missing overview level that forces GDAL to read full-resolution blocks, or a source bucket that moved to another region. Compare Duration p99 week over week and check GETs per tile before touching the memory tier.

Cost falls but the pipeline gets slower and starts failing. Someone lowered the memory tier to save money and crossed into the cache-thrashing region, where the bill goes up per tile and the silent OOM signature follows shortly. Verify against the sweep before accepting a memory reduction.

A retry storm doubles the bill overnight. Failed tiles that retry three times cost four times as much and produce no output. Cap retries, route permanent failures to a dead-letter queue as in implementing dead-letter queues for failed vector jobs, and alarm on DLQ depth rather than cost — cost alarms fire a day late, queue depth fires in minutes.

Cold starts inflate the cost of small tiles. Init duration is billed on Lambda, so a 2.1-second GDAL initialisation added to a 1.9-second 256-pixel tile more than doubles its cost — a large part of why small windows look so expensive in the tile-size sweep. Trimming the layer, as in stripping unnecessary Python packages from AWS Lambda layers, is a cost optimisation as much as a latency one. Provisioned concurrency is the opposite trade: billed per hour of provisioned capacity whether invoked or not, it is a latency purchase, and reducing Python GDAL cold starts with provisioned concurrency works through the break-even.

Scaling the Model from One Scene to a Backfill

The per-scene model composes linearly for compute and non-linearly for everything else, and the non-linear parts are where budgets get missed.

Concurrency quota, not cost, is the first ceiling. A 6,400-tile fan-out against a 1,000-concurrency regional quota queues rather than fails, so the bill is unchanged and the wall clock is six times longer than expected. Reserve concurrency for the tiler so a neighbouring function cannot starve it, and size the queue’s visibility timeout to the p99 duration rather than the average — sizing SQS visibility timeout for long-running raster jobs explains why the average is the wrong statistic.

The free grants matter only at the smallest scale. 400,000 GB-seconds per month covers about 25,000 tiles at 3,008 MB and 5.4 seconds — four scenes. Any real backfill leaves the free tier in its first hour, so never model with it included.

Codec and architecture are flat multipliers on the compute term. ZSTD level 1 decodes roughly twice as fast as deflate at similar ratios on multi-band imagery, cutting 30–40% from the decode segment of every tile; re-encoding a source archive once is often cheaper over a year than any function-level tuning. Graviton is about 20% below x86 per GB-second, and the only obstacle is building GDAL and PROJ for aarch64 — see building rasterio Lambda layers on Amazon Linux 2023.

Backfills belong on a different runtime from steady ingest. The same handler can run per-tile on Lambda for the four scenes a day that arrive by event and inside a container on Batch for the fifty-thousand-scene historical reprocess. Forcing one runtime to serve both is how a pipeline ends up either 17× over budget on the backfill or needlessly complex for the daily trickle — the trade framed in batch vs stream geospatial processing, viewed through the bill instead of through latency.

Frequently Asked Questions

Is a higher Lambda memory setting always more expensive?

No. Lambda bills GB-seconds, so the price per second rises with memory, but vCPU rises with it too. Below roughly 3 GB a GDAL raster job is cache-starved, duration rises faster than the memory price falls, and raising memory reduces cost. Above that the vCPU gains are sub-linear and cost climbs again. The result is a U-shaped curve whose minimum for a 512-pixel four-band window sits near 3,008 MB in the assumption set used here.

What usually costs more than Lambda compute in a raster pipeline?

Three line items regularly beat compute. NAT Gateway data processing at $0.045 per GB when the function sits in a private subnet with no S3 gateway endpoint. CloudWatch Logs ingestion at $0.50 per GB when CPL_DEBUG or verbose GDAL logging is left on. Step Functions Standard state transitions at $0.025 per 1,000 when a Map state runs one iteration per tile. All three are us-east-1 list prices at the time of writing, and all three are configuration rather than code.

How much does Lambda ephemeral storage above 512 MB actually cost?

Very little. Ephemeral storage above the 512 MB default is $0.0000000309 per GB-second in us-east-1 at the time of writing — about 1/540th of the memory rate. Provisioning the full 10,240 MB of /tmp for a five-second invocation adds roughly $0.0000015, under 1% of the compute charge. Treating /tmp as an expensive resource and forcing intermediates into RAM is a false economy.

When does a Cloud Run container beat per-tile Lambda on cost?

When more than half of each tile’s wall clock is object-storage wait. Lambda serves one request per execution environment, so you pay the full memory allocation throughout every range read. Cloud Run serves up to 1,000 concurrent requests per instance, so eight tiles share one instance-second and the wait is paid once instead of eight times. On the 6,400-tile scene modelled here that is a 3× saving; at --concurrency 1 the same container is 24% more expensive than Lambda.

How do I attribute serverless cost to a single raster scene?

Emit it from the handler. Cost allocation tags do not reach invocation granularity, so read billed duration and memory size from the REPORT line or the context object, compute GB-seconds, and publish a CloudWatch embedded-metric-format record with the scene identifier as a dimension. Reconcile the total against Cost Explorer weekly, grouped by usage type rather than by service, so the NAT and Logs charges do not stay hidden inside other services.

Guides in this topic

• Comparing the Cost of Lambda, Cloud Run and Azure Functions for Tiling — Price the same 6,400-tile raster job on AWS Lambda, GCP Cloud Run and Azure Functions
• Cost-per-Tile Math for Lambda Raster Jobs — Work the exact arithmetic for one AWS Lambda raster tile: GB-seconds, request charge, ephemeral storage above…

Related

• Cost-per-Tile Math for Lambda Raster Jobs — the full arithmetic for one invocation, with a runnable script and a tile-size sweep
• Comparing the Cost of Lambda, Cloud Run and Azure Functions for Tiling — the same 6,400-tile scene priced on all three runtimes
• Memory and CPU Allocation for Raster Workloads — the sizing method behind the memory tier the cost curve selects
• Ephemeral Storage Limits in AWS Lambda — what the GB-second charge above 512 MB buys and why it is almost free
• Tuning HTTP Range Requests for COG Reads on S3 — removing the per-request S3 charges and the latency that comes with them

Back to Serverless Geospatial Architecture & Platform Limits