Ephemeral Storage Limits in AWS Lambda for Geospatial Processing

AWS Lambda mounts an NVMe-backed /tmp volume with a configurable ceiling of 10,240 MB (10 GB), up from the historical 512 MB default. For geospatial workloads, sizing this correctly is non-negotiable: a single 2 GB compressed GeoTIFF routinely expands to 6–8 GB of working files once GDAL decompresses blocks, writes VRT intermediates, and fills its block cache. Undershoot the allocation and the function terminates mid-transform with a DiskSpaceExhausted error; overshoot and you pay for storage you never touch.

Why This Constraint Matters for Geospatial Workloads

Geospatial formats are inherently I/O-heavy. A LZW-compressed GeoTIFF holding a single Sentinel-2 granule (110 × 110 km at 10 m resolution) weighs roughly 700 MB on disk but expands to 1.8–2.5 GB when GDAL decompresses it for processing. Add a VRT wrapper, an output draft file, and GDAL’s block cache and the total /tmp footprint easily crosses 5 GB for a single invocation. Common scenarios and their scratch footprints:

For the orthomosaic case, even 10 GB /tmp is insufficient in a single Lambda invocation — that workload belongs in a batch vs. streaming split or a step-function workflow that shards the mosaic before dispatching tiles to individual functions.

Beyond raw capacity, the cold start sequence for a GDAL-heavy function already contends for /tmp during shared-library extraction. Insufficient space here aborts the function before a single pixel is processed.

Platform-by-Platform Limits Table

GCP Cloud Functions stores temp data in a tmpfs that draws from instance RAM — allocating 4 GB of RAM gives you at most 2 GB of usable /tmp before your function begins swapping. For large raster jobs on GCP, Cloud Run with a mounted Cloud Filestore volume is the more natural fit than Cloud Functions.

Azure Functions on the Consumption plan shares ephemeral storage across function instances on the same host; large geospatial jobs must use a Premium or Dedicated plan with managed disks.

Step-by-Step Implementation

1. Baseline your peak scratch requirement

Run this profiler locally against your largest expected input before touching IaC:

import shutil
import os
import rasterio
import tempfile

def profile_tmp_usage(input_path: str) -> dict:
    """
    Measure /tmp consumption during a typical rasterio read-transform cycle.
    Run locally with /tmp mounted; translate results to Lambda /tmp sizing.
    """
    os.environ["CPL_TMPDIR"] = "/tmp/gdal_profile"
    os.environ["GDAL_CACHEMAX"] = "512"       # MB — mirror your Lambda setting
    os.environ["GDAL_DATA"] = "/usr/share/gdal"
    os.environ["PROJ_LIB"] = "/usr/share/proj"
    os.makedirs("/tmp/gdal_profile", exist_ok=True)

    before = shutil.disk_usage("/tmp").used

    with rasterio.open(input_path) as src:
        with tempfile.NamedTemporaryFile(suffix=".tif", dir="/tmp", delete=False) as tmp:
            profile = src.profile.copy()
            profile.update(compress="lzw", tiled=True, blockxsize=512, blockysize=512)
            with rasterio.open(tmp.name, "w", **profile) as dst:
                for ji, window in src.block_windows(1):
                    dst.write(src.read(window=window), window=window)
            peak = shutil.disk_usage("/tmp").used
            os.unlink(tmp.name)

    return {
        "before_bytes": before,
        "peak_bytes": peak,
        "delta_mb": (peak - before) / 1_048_576,
    }

Add 20 % headroom to delta_mb and round up to the nearest 512 MB when setting EphemeralStorage.Size.

2. Set EphemeralStorage in your IaC

AWS SAM (template.yaml):

Resources:
  GeoTransformFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.12
      Handler: handler.process
      MemorySize: 3008          # memory controls CPU share and network bandwidth
      EphemeralStorage:
        Size: 5120              # MB — sized to 1.5x peak scratch from profiler
      Timeout: 600
      Environment:
        Variables:
          CPL_TMPDIR: /tmp/gdal
          GDAL_CACHEMAX: "512"
          GDAL_DISABLE_READDIR_ON_OPEN: EMPTY_DIR
          TMPDIR: /tmp
          GDAL_DATA: /opt/share/gdal
          PROJ_LIB: /opt/share/proj

Terraform (aws_lambda_function):

resource "aws_lambda_function" "spatial_processor" {
  function_name = "spatial-processor"
  runtime       = "python3.12"
  handler       = "handler.process"
  memory_size   = 3008
  timeout       = 600

  ephemeral_storage {
    size = 5120   # MB
  }

  environment {
    variables = {
      CPL_TMPDIR                    = "/tmp/gdal"
      GDAL_CACHEMAX                 = "512"
      GDAL_DISABLE_READDIR_ON_OPEN  = "EMPTY_DIR"
      TMPDIR                        = "/tmp"
      GDAL_DATA                     = "/opt/share/gdal"
      PROJ_LIB                      = "/opt/share/proj"
    }
  }
}

Note: MemorySize and EphemeralStorage.Size are billed independently. Setting both to 10,240 MB doubles your per-invocation cost — right-size each axis separately.

3. Initialise scratch directories at handler entry

import os
import shutil
import tempfile
import logging

logger = logging.getLogger(__name__)

# Module-level initialisation (runs once per cold start)
_TMP_ROOT = "/tmp"
_GDAL_CACHE = "/tmp/gdal"
os.makedirs(_GDAL_CACHE, exist_ok=True)
tempfile.tempdir = _TMP_ROOT

# Explicitly export every GDAL/PROJ path — never rely on defaults
os.environ.setdefault("CPL_TMPDIR", _GDAL_CACHE)
os.environ.setdefault("GDAL_CACHEMAX", "512")
os.environ.setdefault("GDAL_DISABLE_READDIR_ON_OPEN", "EMPTY_DIR")
os.environ.setdefault("TMPDIR", _TMP_ROOT)


def _check_warm_start_stale_files() -> None:
    """Remove any files left by a previous warm-start invocation."""
    for entry in os.scandir(_GDAL_CACHE):
        try:
            os.unlink(entry.path) if entry.is_file() else shutil.rmtree(entry.path)
        except OSError as exc:
            logger.warning("Could not remove stale /tmp file %s: %s", entry.path, exc)


def handler(event: dict, context) -> dict:
    _check_warm_start_stale_files()

    # Emit /tmp baseline usage at invocation start
    usage_start = shutil.disk_usage(_TMP_ROOT).used
    logger.info("tmp_usage_start_mb=%.1f", usage_start / 1_048_576)

    try:
        result = _process(event)
    finally:
        usage_end = shutil.disk_usage(_TMP_ROOT).used
        logger.info("tmp_usage_end_mb=%.1f", usage_end / 1_048_576)

    return result

4. Use block-windowed I/O to cap peak usage

Processing an entire GeoTIFF in memory at once is the single most common cause of /tmp exhaustion. The block_windows() pattern keeps the in-memory footprint bounded regardless of dataset size:

import rasterio
import numpy as np
import os

def reproject_raster(src_uri: str, dst_path: str, target_crs: str = "EPSG:3857") -> None:
    """
    Reproject a raster block-by-block, writing directly to /tmp.
    src_uri may be an s3:// path — rasterio streams via VSIFILE.
    """
    from rasterio.warp import calculate_default_transform, reproject, Resampling

    with rasterio.open(src_uri) as src:
        transform, width, height = calculate_default_transform(
            src.crs, target_crs, src.width, src.height, *src.bounds
        )
        profile = src.profile.copy()
        profile.update(
            crs=target_crs,
            transform=transform,
            width=width,
            height=height,
            compress="lzw",
            tiled=True,
            blockxsize=512,
            blockysize=512,
        )

        with rasterio.open(dst_path, "w", **profile) as dst:
            for band_idx in src.indexes:
                reproject(
                    source=rasterio.band(src, band_idx),
                    destination=rasterio.band(dst, band_idx),
                    src_transform=src.transform,
                    src_crs=src.crs,
                    dst_transform=transform,
                    dst_crs=target_crs,
                    resampling=Resampling.lanczos,
                )

For more targeted techniques around GeoTIFF extraction specifically, see Managing /tmp Storage Limits for GeoTIFF Extraction.

Measurement and Verification

Lambda does not emit /tmp utilisation metrics by default. Instrument your functions with CloudWatch custom metrics using the Embedded Metric Format (EMF):

import json
import shutil
import time

def emit_tmp_metric(value_mb: float, function_name: str) -> None:
    """
    Write an EMF metric to stdout. CloudWatch Logs ingests this automatically
    and creates a metric in the 'GeoSpatial/Lambda' namespace.
    """
    metric = {
        "_aws": {
            "Timestamp": int(time.time() * 1000),
            "CloudWatchMetrics": [
                {
                    "Namespace": "GeoSpatial/Lambda",
                    "Dimensions": [["FunctionName"]],
                    "Metrics": [{"Name": "TmpUsageMB", "Unit": "Megabytes"}],
                }
            ],
        },
        "FunctionName": function_name,
        "TmpUsageMB": value_mb,
    }
    print(json.dumps(metric))


def snapshot_tmp(label: str, function_name: str) -> float:
    used_mb = shutil.disk_usage("/tmp").used / 1_048_576
    emit_tmp_metric(used_mb, function_name)
    return used_mb

Expected output ranges after tuning:

Use the CloudWatch metric math expression MAX(TmpUsageMB) / EphemeralStorageSizeMB * 100 over a 7-day window to right-size your allocation. Target the 95th-percentile peak, not the average.

The Cold Start Mapping for Python GDAL page explains how to correlate InitDuration spikes with /tmp write contention during shared-library extraction.

Failure Modes and Debugging

1. DiskSpaceExhausted mid-transform

Signature: OSError: [Errno 28] No space left on device or the Lambda runtime’s DiskSpaceExhausted error, usually mid-write inside rasterio.open(..., "w") or during GDAL’s internal block flush.

Root cause: EphemeralStorage.Size is smaller than the decompressed working set plus GDAL intermediates.

Fix: Increase EphemeralStorage.Size by 2× and re-run. Also set GDAL_CACHEMAX explicitly — an unset value defaults to 5 % of system RAM, which on a 3 GB Lambda is ~150 MB in memory but can generate large disk spills.

2. Silent cache overflow fills /tmp without error

Signature: TmpUsageMB grows monotonically across invocations (visible in your EMF metric), eventually triggering DiskSpaceExhausted on a warm invocation long after the offending file was written.

Root cause: CPL_TMPDIR is unset, so GDAL writes cache files to /tmp outside your managed directories. Without explicit cleanup, warm-start reuse accumulates them.

Fix:

# Always set before importing gdal or rasterio
import os
os.environ["CPL_TMPDIR"] = "/tmp/gdal"
os.makedirs("/tmp/gdal", exist_ok=True)

Then add a glob.glob("/tmp/gdal/*") purge at handler entry.

3. OSError: [Errno 30] Read-only file system

Signature: Immediate crash on the first file write, usually inside a library that defaults to /var/task or /var/runtime.

Root cause: Everything outside /tmp in Lambda is read-only. Some GDAL plugins or Python packages write to tempfile.gettempdir() which can resolve to /var if TMPDIR is not set.

Fix: Export TMPDIR=/tmp in your Lambda environment variables and confirm tempfile.gettempdir() returns /tmp from a test invocation.

4. Warm-start stale file collision

Signature: Transient FileExistsError or corrupted output on a fraction of invocations, correlating with high concurrency.

Root cause: A previous warm-start invocation left a partial output file at the same deterministic path (e.g., /tmp/output.tif). The next invocation tries to create it anew.

Fix: Use tempfile.NamedTemporaryFile(dir="/tmp", suffix=".tif", delete=False) to generate unique paths, or add a UUID prefix: f"/tmp/{uuid.uuid4().hex}_output.tif".

5. GeoPackage / SQLite journal exhausts /tmp

Signature: sqlite3.OperationalError: disk I/O error during fiona or GDAL GeoPackage writes.

Root cause: SQLite’s WAL journal can double the effective file size during large transactions. A 1 GB GeoPackage write briefly needs 2 GB of /tmp.

Fix: Set OGR_GPKG_JOURNAL_MODE=MEMORY as a GDAL config option, or reduce transaction batch size so the WAL journal stays small.

Cost and Scaling Considerations

Per-invocation cost math

Storage above 512 MB is billed at $0.0000000367 per GB-second (us-east-1, June 2026).

For a 5,120 MB allocation running a 60-second GeoTIFF reproject:

Billable storage = (5,120 − 512) MB = 4,608 MB = 4.5 GB
Cost = 4.5 GB × 60 s × $0.0000000367 = ~$0.0000099 per invocation

At 1 million invocations per month: ~$9.90/month in storage alone. Compare this with memory billing (which also covers CPU) before optimising the wrong axis.

When to prefer alternatives

EFS mounts eliminate the /tmp ceiling entirely but add ~10 ms of cold-start latency per mount point. For workloads where 10 ms matters, profile against the memory and CPU allocation patterns before adding EFS.

At scale, the EphemeralStorage.Size setting becomes a cost-optimisation variable: use Lambda Power Tuning (AWS open-source tool) to sweep storage sizes alongside memory across a representative test payload and find the cheapest configuration that avoids DiskSpaceExhausted.

The IAM Security Boundaries for Cloud GIS page covers the complementary topic of scoping S3 write permissions to exactly the output prefix your Lambda targets — important when /tmp intermediates get uploaded during error recovery.

Frequently Asked Questions

What is the maximum /tmp storage in AWS Lambda? 512 MB to 10,240 MB (10 GB). The default is 512 MB if you do not set EphemeralStorage.Size. You can increase it in the console, SAM, CDK, or Terraform independently of memory allocation.

Does /tmp persist between Lambda invocations? Yes, during warm starts the execution environment is reused and /tmp contents survive. Data is only wiped when AWS reclaims the sandbox. Always clean up or check for stale files at the start of each handler.

How much /tmp does a 2 GB GeoTIFF need? Plan for 3–4 GB: the compressed source (~2 GB), an uncompressed working copy (~2–3× compressed size depending on band count), plus GDAL VRT and block-cache intermediates. A 4 GB /tmp allocation with GDAL_CACHEMAX capped at 512 MB is a safe starting point.

Is ephemeral storage billed separately from memory? Yes. Storage above 512 MB is billed at $0.0000000367 per GB-second. For a 4 GB allocation running 30 seconds, that is roughly (4 − 0.5) × 30 × $0.0000000367 ≈ $0.0000039 per invocation — negligible per call but material at millions of invocations per month.

Related

• Managing /tmp Storage Limits for GeoTIFF Extraction
• Cold Start Mapping for Python GDAL
• Memory and CPU Allocation for Raster Workloads
• IAM Security Boundaries for Cloud GIS
• Batch vs. Stream Geospatial Processing

Back to Serverless Geospatial Architecture & Platform Limits