Ephemeral Storage Comparison Across Serverless Platforms

Serverless geospatial functions live and die by their scratch disk: GDAL unpacks GeoTIFF overviews, writes .vrt mosaics, and caches PROJ grids to /tmp before a single pixel is returned, and the three major platforms model that scratch space in three fundamentally different ways. AWS Lambda gives 512 MB of disk-backed /tmp by default that you can raise to 10,240 MB; GCP Cloud Functions 2nd gen gives an in-memory tmpfs that draws down your 32 GB memory ceiling; and Azure Functions Consumption gives a shared pool of roughly 500 MB to 1.5 GB. Getting this wrong produces two different failure signatures — a disk-full ENOSPC on Lambda versus an out-of-memory kill on Cloud Functions for the exact same workload — so the model matters as much as the number. This comparison sits within serverless geospatial architecture and platform limits and drills into a constraint that silently caps raster throughput.

Why Ephemeral Storage Matters for Geospatial Workloads

Most serverless workloads touch /tmp lightly — a log buffer, a downloaded config. Geospatial workloads hammer it. Opening a Cloud Optimized GeoTIFF over /vsicurl/ still spills decoded blocks and overview pyramids to disk; building a mosaic writes a .vrt plus per-tile intermediates; reprojecting through PROJ downloads and caches multi-megabyte transformation grids; and any ogr2ogr conversion of a zipped shapefile unpacks the .shp, .dbf, .shx, and .prj before it reads a feature. A single moderate GeoTIFF extraction can stage several gigabytes of temporary files that never appear in your code but decide whether the invocation succeeds.

The trap is that the number of megabytes is only half the story — the backing medium is the other half. On AWS Lambda /tmp is a real, separate disk allocation: writing 3 GB there consumes none of your function memory, and overflowing it raises OSError: [Errno 28] No space left on device. On GCP Cloud Functions 2nd gen /tmp is a tmpfs mounted in RAM: writing 3 GB there consumes 3 GB of your memory allocation, and overflowing it does not raise a disk error at all — the kernel OOM-kills the process and you get a bare Memory limit exceeded with no stack trace. Two platforms, one workload, two completely different debugging paths. This distinction is why the memory and CPU allocation for raster workloads math has to include temp-file footprint on GCP but can treat it separately on AWS.

The AWS-specific mechanics — how the 512 MB default is billed, how to raise it, and how GDAL config options redirect intermediates — are covered in depth under ephemeral storage limits in AWS Lambda and the hands-on managing /tmp storage limits for GeoTIFF extraction guide. This page is the cross-platform view: how the three providers compare, and how to pick.

Platform-by-Platform Limits

The exact quotas and, more importantly, the backing model for each platform’s scratch space. Read the “Backing medium” and “Overflow signature” rows together — they are what separate a five-minute fix from an afternoon of confused debugging.

Three practical takeaways fall out of the table. First, Lambda is the only platform where scratch is genuinely free of memory pressure — you can run a 512 MB-memory function that writes 10 GB to /tmp, which is impossible anywhere else. Second, Cloud Functions has by far the largest potential scratch (up to 32 GB) but you pay for it as memory, and it competes with GDAL’s own buffers. Third, Azure Consumption is the tightest on every axis and the only one where you cannot simply turn a dial to get more — you graduate to a Premium plan or a container.

Step-by-Step Implementation

Step 1: Inventory the temp-file footprint of one invocation

Before choosing a platform sizing, measure what a single job actually stages on disk. Run the workload locally with a probe that reports the high-water mark, not the end-state, because GDAL deletes most intermediates before returning.

import os
import threading
import time

def watch_tmp_high_water(path="/tmp", interval=0.25):
    """Background sampler that records peak /tmp usage during a run."""
    peak = {"bytes": 0}

    def _sample():
        while not stop.is_set():
            st = os.statvfs(path)
            used = (st.f_blocks - st.f_bfree) * st.f_frsize
            peak["bytes"] = max(peak["bytes"], used)
            time.sleep(interval)

    stop = threading.Event()
    t = threading.Thread(target=_sample, daemon=True)
    t.start()
    return stop, peak

# Wrap the actual geospatial job
stop, peak = watch_tmp_high_water()
run_geotiff_extraction("s3://bucket/scene.tif")   # your workload
stop.set()
print(f"peak /tmp high-water: {peak['bytes'] / 1e6:.0f} MB")

A typical single-scene Sentinel-2 extraction peaks around 1.5–3 GB of intermediates; a national mosaic build can peak well past 6 GB. That peak, plus headroom, is the number you size against.

Step 2: Redirect GDAL and PROJ scratch explicitly

By default GDAL scatters temporaries based on compile-time paths that may not be writable in a serverless sandbox. Pin every scratch location to /tmp and cap GDAL’s in-process cache so it flushes to disk predictably rather than ballooning memory.

import os

# All of these must point at writable scratch — /tmp on every platform
os.environ["CPL_TMPDIR"] = "/tmp"          # GDAL temporary files (VRTs, tiles)
os.environ["TMPDIR"] = "/tmp"              # generic C library temp
os.environ["GDAL_DATA"] = "/opt/share/gdal"
os.environ["PROJ_LIB"] = "/opt/share/proj"
os.environ["PROJ_DATA"] = "/opt/share/proj"

# PROJ downloads transformation grids on demand — cache them in /tmp,
# not the read-only package dir, or reprojection fails with a write error
os.environ["PROJ_NETWORK"] = "ON"
os.environ["XDG_DATA_HOME"] = "/tmp/proj_cache"

# Cap GDAL's block cache so decoded raster blocks flush instead of
# growing memory unbounded (critical on tmpfs-backed platforms)
os.environ["GDAL_CACHEMAX"] = "256"        # megabytes
os.environ["VSI_CACHE"] = "TRUE"
os.environ["VSI_CACHE_SIZE"] = "26214400"  # 25 MB per-file /vsicurl cache

On Cloud Functions this configuration is doubly important: because /tmp is tmpfs, an uncapped GDAL_CACHEMAX plus large .vrt intermediates can exhaust memory from two directions at once. Setting GDAL_CACHEMAX=256 bounds the in-process side while the tmpfs bound is your total memory minus the process working set.

Step 3: Provision scratch per platform

# AWS Lambda — raise disk-backed /tmp to 4 GB (separate from memory)
aws lambda update-function-configuration \
  --function-name geo-extract \
  --ephemeral-storage '{"Size": 4096}' \
  --memory-size 2048

# GCP Cloud Functions 2nd gen — /tmp is memory, so size memory to cover
# process working set + peak tmpfs footprint (e.g. 3 GB temp + 2 GB work)
gcloud functions deploy geo-extract \
  --gen2 --runtime python312 --region us-central1 \
  --memory 5Gi --timeout 540s

# Azure Functions — Consumption caps scratch; for large temp files deploy
# the same code as a container on a Premium plan or Container Apps instead
az functionapp create \
  --name geo-extract --consumption-plan-location eastus \
  --runtime python --functions-version 4 \
  --storage-account geoextractsa --resource-group geo-rg

Note the asymmetry: on Lambda --ephemeral-storage 4096 and --memory-size 2048 are independent dials. On Cloud Functions there is no separate ephemeral dial at all — --memory 5Gi is the only lever, and it must simultaneously cover the process and the tmpfs.

Step 4: Wipe /tmp at the top of every invocation

Warm instances reuse /tmp, so leftovers from a previous job count against the current one — a slow leak that surfaces as an intermittent ENOSPC or OOM only under sustained traffic.

import glob
import os
import shutil

def clean_tmp(prefix="/tmp"):
    """Remove per-job scratch left by a previous warm invocation."""
    for entry in glob.glob(os.path.join(prefix, "*")):
        # Preserve caches you intend to keep warm (e.g. PROJ grids)
        if entry.endswith("proj_cache"):
            continue
        try:
            if os.path.isdir(entry) and not os.path.islink(entry):
                shutil.rmtree(entry, ignore_errors=True)
            else:
                os.remove(entry)
        except OSError:
            pass

def handler(event, context):
    clean_tmp()          # first line — reclaim scratch from prior runs
    return run_geotiff_extraction(event["uri"])

Measurement and Verification

Instrument every invocation to emit its peak /tmp usage as a structured metric, then alert when it crosses 80% of the provisioned ceiling. This is the same os.statvfs reading used in Step 1, promoted to a per-invocation log line.

import json
import logging
import os

logger = logging.getLogger()
logger.setLevel(logging.INFO)

def emit_tmp_metric(ephemeral_limit_mb: int):
    st = os.statvfs("/tmp")
    used_mb = (st.f_blocks - st.f_bfree) * st.f_frsize / 1e6
    total_mb = st.f_blocks * st.f_frsize / 1e6
    logger.info(json.dumps({
        "event": "tmp_usage",
        "tmp_used_mb": round(used_mb, 1),
        "tmp_total_mb": round(total_mb, 1),
        "pct_of_limit": round(used_mb / ephemeral_limit_mb * 100, 1),
    }))

Expected log output from a healthy Lambda with 4 GB /tmp after a Sentinel-2 extraction:

{"event": "tmp_usage", "tmp_used_mb": 2140.5, "tmp_total_mb": 4096.0, "pct_of_limit": 52.3}

On Cloud Functions, cross-check the same run against Memory limit exceeded events in Cloud Logging — because tmpfs and process memory share a pool, a pct_of_limit of 60% on /tmp can still OOM if the process working set is large. The verification that matters on GCP is the combined figure, not the /tmp figure alone.

Failure Modes and Debugging

OSError: [Errno 28] No space left on device on Lambda: /tmp overflowed. Either raise EphemeralStorage or, better, find the un-deleted intermediate — usually a .vrt or an unpacked shapefile the code forgot to remove. The managing /tmp storage limits for GeoTIFF extraction guide walks the windowed-read pattern that avoids staging the whole raster.

Memory limit exceeded on Cloud Functions with modest process memory: the tmpfs is the culprit, not your arrays. A 3 GB GeoTIFF written to /tmp on a 4 GB function leaves 1 GB for everything else. Raise --memory, or stream the raster through /vsicurl/ windowed reads so it never fully lands on disk.

Reprojection fails with a write error to the package directory: PROJ tried to cache a transformation grid under its read-only install path. Set XDG_DATA_HOME=/tmp/proj_cache and PROJ_NETWORK=ON so grids download to writable scratch, as in Step 2.

Intermittent ENOSPC that only appears under load: warm-instance /tmp leakage. A previous invocation left scratch behind and the next one topped it off. Add the Step 4 clean_tmp() call as the first line of the handler.

Azure function silently truncates a large output file: the shared Consumption scratch pool filled and the host reclaimed it mid-write. There is no dial to raise it — move the workload to a Premium plan with a larger /tmp or containerize it, as flagged in Step 3.

Cost and Scaling Considerations

The cost calculus differs by platform precisely because the backing medium differs. On Lambda, ephemeral storage beyond the free 512 MB is billed per GB-second at a rate roughly two orders of magnitude cheaper than memory GB-seconds — so provisioning 4 GB of /tmp on a 2 GB-memory function is far cheaper than provisioning 6 GB of memory to fake the same scratch on a tmpfs platform. When your bottleneck is temp-file space rather than compute, Lambda’s separate disk allocation is the most cost-efficient of the three.

On Cloud Functions the scratch has no separate line item — it is memory — so the trade is between a larger memory tier (which also buys you more CPU, since CPU scales with memory) and restructuring the job to touch disk less. Because GCP grants up to 32 GB and 60 minutes, the pragmatic move for a genuinely large mosaic is to buy the memory, tile the raster to bound peak footprint, and finish in one invocation, rather than fanning out. That fan-out-versus-single-invocation decision is the same one weighed in the timeout ceiling comparison for geospatial jobs, and it interacts directly with scratch: more tiles means less peak /tmp per invocation but more invocations to pay for. Azure Consumption’s fixed, small scratch pushes any storage-heavy geospatial job off the plan entirely — budget for Premium or a container from the start rather than discovering the ceiling in production. Whichever platform you land on, keep the /tmp high-water metric from the verification section on a dashboard next to memory and duration, so a creeping temp-file footprint is visible before it becomes an outage.

Frequently Asked Questions

How much ephemeral /tmp storage does each serverless platform give?

AWS Lambda provides 512 MB of disk-backed /tmp by default, configurable up to 10,240 MB and billed per GB-second beyond the first 512 MB. GCP Cloud Functions 2nd gen provides an in-memory tmpfs that shares the function’s memory allocation, so it scales up to the 32 GB memory ceiling. Azure Functions on the Consumption plan gives roughly 500 MB to 1.5 GB of shared host scratch, depending on the underlying host, with no dial to raise it.

Does writing to /tmp on Cloud Functions use memory?

Yes. Cloud Functions 2nd gen backs /tmp with an in-memory tmpfs, so every byte written to /tmp counts against the function’s memory allocation. A 4 GB-memory function that writes a 3 GB GeoTIFF to /tmp has only 1 GB left for the process, and exceeding the combined total triggers an out-of-memory kill — a Memory limit exceeded event, not a disk-full error. This is the single biggest difference from Lambda’s disk-backed model.

How do I measure /tmp usage in a serverless function?

Call os.statvfs('/tmp') and compute used bytes as (f_blocks - f_bfree) * f_frsize. Emit it as a structured log metric at the end of each invocation, and sample it on a background thread during the run to capture the high-water mark — GDAL deletes most intermediate VRTs before the handler returns, so a single point-in-time reading understates the true peak.

Which platform should I pick for temp-heavy geospatial jobs?

If scratch space is your bottleneck and compute is modest, AWS Lambda wins because its /tmp is a cheap, separate disk allocation up to 10 GB. If you need both large scratch and long runtime — a national mosaic, say — GCP Cloud Functions 2nd gen’s 32 GB and 60 minutes handle it in one invocation, at the cost of paying for scratch as memory. Azure Functions Consumption is the wrong tool for any storage-heavy raster job; move to Premium or a container.

Related

• Ephemeral Storage Limits in AWS Lambda — the AWS-specific mechanics: billing, the 10 GB ceiling, and GDAL config for /tmp
• Managing /tmp Storage Limits for GeoTIFF Extraction — windowed reads that keep a raster from fully landing on disk
• Memory and CPU Allocation for Raster Workloads — why tmpfs-backed scratch must be folded into the memory budget on GCP
• Timeout Ceiling Comparison for Geospatial Jobs — the fan-out-versus-single-invocation trade that governs peak /tmp footprint
• Cold Start Mapping for Python GDAL — how scratch extraction during initialization adds to cold-start latency

Back to Serverless Geospatial Architecture & Platform Limits