Managing /tmp Storage Limits for GeoTIFF Extraction

Avoid writing GeoTIFF data to /tmp entirely: stream windowed reads from cloud storage via GDAL VSI paths, buffer extracted tiles in rasterio.io.MemoryFile, and guard every invocation with a pre-flight /tmp usage check. AWS Lambda provisions 512 MB of /tmp by default (expandable to 10,240 MB), but even with the maximum allocation a naive full-file download of a multi-band Sentinel-2 scene will exhaust it. The only reliable strategy is zero-disk materialization combined with explicit GDAL cache limits.

Context: What Drives /tmp Exhaustion During GeoTIFF Work

Ephemeral storage limits in AWS Lambda are the primary constraint for raster extraction jobs. Three failure modes produce OSError: [Errno 28] No space left on device in the middle of a rasterio or GDAL operation:

  1. Full-file downloads. boto3.download_file() or a raw requests.get() writes every byte of the source GeoTIFF to /tmp before the first pixel is decoded. A single 1 GB Cloud-Optimized GeoTIFF (COG) wipes out the default 512 MB quota immediately.
  2. GDAL block-cache spills. GDAL maintains an in-memory block cache controlled by GDAL_CACHEMAX. When that limit is hit, GDAL silently evicts blocks to a temp directory — which resolves to /tmp inside Lambda. Without an explicit cap this can grow unbounded.
  3. Intermediate file artefacts. Reprojection, format conversion, and in-place compression all produce .aux.xml metadata files, .tmp swap files, and partially-written COG staging files. These accumulate across warm container reuse.

A secondary concern is that cold start timing for Python GDAL is sensitive to /tmp I/O load: if GDAL driver registration coincides with an active /tmp spill from a prior warm-container extraction, latency spikes compound.

The diagram below shows the two execution paths — the disk-heavy path that exhausts /tmp and the zero-disk path that this page implements.

GeoTIFF extraction: disk-heavy path vs zero-disk path Two parallel swimlanes showing how a naive full-file download writes to /tmp and fails, while windowed VSI reads with MemoryFile bypass /tmp entirely and upload directly to S3. DISK-HEAVY PATH (fails) ZERO-DISK PATH (this page) S3 source GeoTIFF boto3.download_file() → /tmp OSError: No space left on device Invocation fails — /tmp exhausted S3 source GeoTIFF (COG) /vsis3/ range requests (windowed read only) rasterio.io.MemoryFile (RAM buffer, no /tmp write) s3.put_object() → destination

Prerequisites

Before implementing the pattern below, confirm each of the following:

  • Python 3.11+ and rasterio >= 1.3.9 (earlier versions have a MemoryFile seek regression that corrupts multi-band writes)
  • GDAL >= 3.6 (bundled inside the rasterio wheel; verify with python -c "from osgeo import gdal; print(gdal.__version__)")
  • boto3 >= 1.28 for put_object with streaming body support
  • Lambda ephemeral storage set to at least 512 MB (the default) — increase to 2,048 MB if you anticipate GDAL cache spills from concurrent warm invocations; see Ephemeral Storage Limits in AWS Lambda for the SAM/CDK config syntax
  • IAM permissions on the Lambda execution role:
    • s3:GetObject on the source bucket (read)
    • s3:PutObject on the destination bucket (write)
    • No s3:GetBucketLocation needed — GDAL VSI handles region resolution automatically when AWS_DEFAULT_REGION is set
  • Environment variables set in the function configuration (not in code):
    code 
    GDAL_CACHEMAX=256M
GDAL_DISABLE_READDIR_ON_OPEN=TRUE
VSI_CURL_CACHE_SIZE=100M
GDAL_DATA=/opt/share/gdal
PROJ_LIB=/opt/share/proj
AWS_DEFAULT_REGION=eu-west-1

Implementation

The function below extracts a geographic bounding box from a cloud-hosted COG without writing anything to /tmp. All intermediate data lives in process RAM via rasterio.io.MemoryFile. A pre-flight /tmp check guards against accumulated state from warm container reuse.

python 
import os
import shutil
from urllib.parse import urlparse

import boto3
import rasterio
from rasterio.enums import Resampling
from rasterio.io import MemoryFile
from rasterio.windows import from_bounds, transform as window_transform


def _tmp_used_mb() -> float:
    """Return current /tmp disk usage in MB using shutil.disk_usage."""
    usage = shutil.disk_usage("/tmp")
    return (usage.total - usage.free) / (1024 ** 2)


def _to_vsi(uri: str) -> str:
    """
    Convert an S3 or HTTPS URI to a GDAL Virtual File System path.

    /vsis3/ enables range-request streaming with no intermediate download.
    /vsicurl/ handles HTTPS COGs hosted on any server (e.g. Planetary Computer).
    """
    parsed = urlparse(uri)
    if parsed.scheme == "s3":
        return f"/vsis3/{parsed.netloc}{parsed.path}"
    if parsed.scheme in ("http", "https"):
        return f"/vsicurl/{uri}"
    # Assume already a VSI path or local path — pass through unchanged
    return uri


def extract_geotiff_window(
    src_uri: str,
    dst_uri: str,
    bounds: tuple[float, float, float, float],
    *,
    max_tmp_mb: int = 50,
    resampling: Resampling = Resampling.bilinear,
    compress: str = "deflate",
    tile_size: int = 256,
    profile_overrides: dict | None = None,
) -> str:
    """
    Extract a spatial window from a cloud-hosted GeoTIFF and upload to S3.

    No data is written to /tmp at any point. The output tile is buffered
    entirely in RAM via rasterio.io.MemoryFile.

    Args:
        src_uri:           S3 (s3://bucket/key) or HTTPS URI to a COG.
        dst_uri:           S3 URI (s3://bucket/key) for the output tile.
        bounds:            (left, bottom, right, top) in the source dataset CRS.
        max_tmp_mb:        Abort before opening the raster if /tmp already
                           exceeds this threshold (guards warm container drift).
        resampling:        Resampling algorithm applied during windowed read.
        compress:          GDAL compression codec for the output tile.
        tile_size:         Internal tile block size for the output COG.
        profile_overrides: Any additional rasterio profile keys to apply last.

    Returns:
        The dst_uri string, confirming successful upload.

    Raises:
        RuntimeError: If /tmp usage exceeds max_tmp_mb before processing.
    """
    # --- 1. Pre-flight /tmp guard -------------------------------------------
    # Warm containers can carry leftover /tmp state from previous invocations.
    # Fail fast here rather than letting a partial write exhaust the partition
    # midway through raster I/O.
    current_tmp = _tmp_used_mb()
    if current_tmp > max_tmp_mb:
        raise RuntimeError(
            f"/tmp already at {current_tmp:.1f} MB (threshold: {max_tmp_mb} MB). "
            "Cannot safely proceed — prior invocation may have left residual files."
        )

    # --- 2. Resolve VSI path -------------------------------------------------
    # rasterio.open() on a /vsis3/ path issues HTTP range requests directly
    # against S3 — no intermediate download, no /tmp write.
    vsi_path = _to_vsi(src_uri)

    # --- 3. Windowed read ----------------------------------------------------
    # from_bounds() converts geographic coordinates to the pixel window that
    # covers the requested bounding box. Only those pixels are fetched.
    with rasterio.open(vsi_path) as src:
        window = from_bounds(*bounds, transform=src.transform)

        # Read returns a NumPy array (bands × rows × cols) for this window only.
        data = src.read(window=window, resampling=resampling)

        # Build the output profile: preserve CRS, dtype, and band count, then
        # override spatial metadata and encoding settings for the extracted tile.
        profile = src.meta.copy()
        profile.update(
            width=round(window.width),
            height=round(window.height),
            transform=window_transform(window, src.transform),
            compress=compress,
            tiled=True,
            blockxsize=tile_size,
            blockysize=tile_size,
            **(profile_overrides or {}),
        )

    # --- 4. Write to MemoryFile and upload -----------------------------------
    # MemoryFile is a RAM-backed file object that satisfies rasterio's file
    # protocol. Writing here never touches /tmp.
    with MemoryFile() as buf:
        with buf.open(**profile) as tile:
            tile.write(data)

        # Rewind the MemoryFile buffer before reading its bytes.
        buf.seek(0)
        payload = buf.read()

    parsed_dst = urlparse(dst_uri)
    boto3.client("s3").put_object(
        Bucket=parsed_dst.netloc,
        Key=parsed_dst.path.lstrip("/"),
        Body=payload,
        ContentType="image/tiff",
    )

    return dst_uri

Why each design decision matters

  • /vsis3/ VSI path: GDAL’s virtual filesystem layer intercepts open() and translates it into HTTP range requests. Combined with a Cloud-Optimized GeoTIFF’s internal tile index, only the pixel blocks that overlap bounds are ever transferred.
  • rasterio.windows.from_bounds(): This is the canonical API for converting geographic coordinates to a pixel Window. Computing the window manually via affine arithmetic is error-prone with rotated or south-up rasters.
  • MemoryFile instead of a tempfile: MemoryFile allocates its buffer from the process heap. For tile sizes under ~200 MB it is faster than NVMe I/O because it avoids a kernel round-trip; it also works correctly when Lambda memory is configured for raster workloads at 3–6 GB.
  • Pre-flight threshold check: A Lambda container that processed a large job in its previous invocation may have lingering /tmp files from GDAL’s .aux.xml writer or a partial cache spill. Checking before opening the raster surface this before any data transfer begins.

Verification

Run the following against a known COG to confirm no /tmp growth occurs during extraction:

python 
import shutil
import json

def _tmp_snapshot() -> dict:
    u = shutil.disk_usage("/tmp")
    return {"total_mb": u.total / 1e6, "used_mb": (u.total - u.free) / 1e6, "free_mb": u.free / 1e6}

before = _tmp_snapshot()

extract_geotiff_window(
    src_uri="s3://your-bucket/sentinel2/B04.tif",
    dst_uri="s3://your-output-bucket/tiles/B04_crop.tif",
    bounds=(13.0, 52.4, 13.2, 52.6),   # lon/lat bbox over Berlin
)

after = _tmp_snapshot()

growth_mb = after["used_mb"] - before["used_mb"]
print(json.dumps({"before": before, "after": after, "growth_mb": growth_mb}))
assert growth_mb < 1.0, f"/tmp grew by {growth_mb:.2f} MB — investigate GDAL cache config"

Expected output (successful run):

json 
{"before": {"total_mb": 536.9, "used_mb": 0.0, "free_mb": 536.9},
 "after":  {"total_mb": 536.9, "used_mb": 0.1, "free_mb": 536.8},
 "growth_mb": 0.1}

The 0.1 MB growth is from GDAL’s internal driver state, not a data write. The assertion will pass. If growth_mb exceeds 1 MB, GDAL has spilled a cache block — recheck that GDAL_CACHEMAX=256M is present in the Lambda environment variables.

Gotchas and Edge Cases

  • Warm container /tmp accumulation. Lambda may reuse a container across dozens of invocations. GDAL writes .aux.xml sidecar metadata files to the same directory as the source path. When the source is a VSI path, GDAL resolves this relative to /tmp. Over many invocations these files accumulate. Add an os.scandir("/tmp") cleanup step at the end of each handler, or set GDAL_DISABLE_READDIR_ON_OPEN=TRUE to suppress .aux.xml creation entirely.

  • MemoryFile + very large windows. MemoryFile allocates from heap. Extracting a window that covers most of a large mosaic (e.g. a 5 000 × 5 000 pixel crop of a 10-band dataset) will require several hundred MB of RAM. If your tile size approaches the function’s memory allocation, split the bounding box into a grid of smaller sub-windows using a quadtree or fixed-stride approach, and queue each sub-request separately via SQS queue routing strategies.

  • rasterio < 1.3.9 MemoryFile seek bug. In versions before 1.3.9 a missing seek(0) in MemoryFile.__exit__ causes the buffer to be uploaded at the wrong offset, producing a corrupt TIFF that passes put_object without error but fails to open on the receiving end. Pin rasterio>=1.3.9 in your Lambda layer or Docker image.

  • from_bounds() CRS mismatch. bounds must be in the same coordinate reference system as src.crs. If your caller provides WGS-84 (EPSG:4326) coordinates but the source COG is in a projected CRS (e.g. UTM), the computed window will be wrong or empty. Reproject the bounds first using pyproj.Transformer.from_crs() before calling from_bounds(), or expose bounds_crs as a parameter and handle the reprojection inside the function.

Frequently Asked Questions

What is the default /tmp size on AWS Lambda and can it be increased?

AWS Lambda provisions 512 MB of /tmp by default. You can increase this independently of memory, up to 10,240 MB, via the EphemeralStorage configuration in SAM, CDK, or the AWS Console. The additional storage is billed at roughly $0.0000000309 per GB-second, independently of the memory billing dimension.

Does rasterio.io.MemoryFile write anything to /tmp?

No. MemoryFile holds its buffer entirely in process RAM. As long as your extracted tile fits in the available memory allocation, /tmp is never touched during the write phase. The only /tmp activity you may observe is GDAL’s .aux.xml metadata writer, which you can suppress with GDAL_DISABLE_READDIR_ON_OPEN=TRUE.

Why does GDAL_CACHEMAX matter for /tmp usage?

When GDAL’s in-memory block cache is exhausted, GDAL evicts overflow blocks to a temporary directory — which defaults to /tmp inside Lambda. Setting GDAL_CACHEMAX to a value well below your available RAM (256 MB is safe for most workloads) prevents this silent spill. For more on GDAL cache configuration see the GDAL Virtual File Systems documentation.

Can /tmp data persist between Lambda invocations?

Yes. On warm container reuse, /tmp content survives between invocations in the same execution environment. Accumulated partial writes from a prior invocation can exhaust the partition for a new one. Always run the pre-invocation threshold check (_tmp_used_mb()) rather than assuming /tmp is empty.

Back to Ephemeral Storage Limits in AWS Lambda