Docker Container Optimization for GIS

A stock GDAL + rasterio container built on ubuntu:22.04 lands at 1.8 GB before you add a single line of application code. Multi-stage builds, binary stripping, and dependency isolation consistently reduce that to under 500 MB — the practical ceiling for serverless deployments on AWS Lambda container images, GCP Cloud Run, and Azure Container Apps. This page covers the exact steps, platform limits, and CI gates to get there reliably.

Optimizing container size sits at the heart of Packaging & Dependency Management for Serverless GIS, where every megabyte saved translates directly into reduced cold-start latency and faster layer pulls across serverless runtimes.

Why Container Size Matters for Geospatial Workloads

Geospatial libraries carry an unusually large dependency footprint. GDAL alone links against libproj, libgeos, libsqlite3, libcurl, and a suite of optional format drivers (HDF5, NetCDF, ECW). Python wrappers like rasterio, Fiona, and geopandas add their own C extensions on top. The result is a deep native library graph that no amount of pip tricks can eliminate — only disciplined image architecture can.

The size problem compounds under serverless constraints in three ways:

1. Layer pull latency. Every cold start must pull uncached image layers from the container registry before your Lambda or Cloud Run instance can serve the first request. A 1.8 GB image takes 15-40 seconds to hydrate on a cold worker; a 400 MB image takes 3-8 seconds. The Cold Start Mapping for Python GDAL analysis shows that layer hydration, not Python import time, dominates cold-start duration for geospatial containers.

2. Deployment package limits. AWS Lambda container images support up to 10 GB, but practical limits are tighter: images above 500 MB noticeably degrade the cold-start P99 even with provisioned concurrency. GCP Cloud Run has no hard image-size limit but bills on CPU time including startup, making bloated images expensive at scale.

3. Ephemeral storage contention. When a container decompresses raster tiles or caches PROJ grid shift files to /tmp, it competes with the OS’s own layer mount. Ephemeral Storage Limits in AWS Lambda details how Lambda’s default 512 MB /tmp can exhaust before GDAL finishes registering drivers on first import, depending on which auxiliary datasets the build copies in.

The diagram below maps the optimization workflow from a fat builder stage to a lean runtime image.

Platform-by-Platform Limits Table

Container deployments on all three major providers share the multi-stage pattern but differ on hard quotas, image registry behaviour, and how image size affects billing.

For Lambda, images stored in the same region as your function are pulled over the AWS internal network — there is no egress charge, but layer hydration still adds measurable latency above 500 MB. Cloud Run pre-warms containers on demand; images above 1 GB start to breach the 60-second default request timeout during cold starts under bursty traffic.

Step-by-Step Implementation

1. Architect Multi-Stage Builds

Single-stage Dockerfiles accumulate build tools, headers, and intermediate artifacts in the final image. A multi-stage build discards the compiler toolchain entirely: the builder stage installs gcc, cmake, and geospatial development headers, compiles Python wheels from source, and generates .so binaries; the runtime stage copies only the resulting site-packages, required shared libraries, and geospatial data directories.

# syntax=docker/dockerfile:1.6
FROM ubuntu:22.04 AS builder
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential cmake python3-dev python3-pip \
    libgdal-dev libproj-dev libgeos-dev \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /build
COPY requirements.txt .
# Build wheels into a staging directory; do not install here
RUN pip wheel --no-cache-dir --wheel-dir=/wheels -r requirements.txt

FROM ubuntu:22.04 AS runtime
ENV DEBIAN_FRONTEND=noninteractive
# Runtime shared libraries only — no -dev headers, no compilers
RUN apt-get update && apt-get install -y --no-install-recommends \
    libgdal30 libproj22 libgeos3.10.2 python3 python3-pip \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /wheels /wheels
RUN pip install --no-cache-dir /wheels/*.whl && rm -rf /wheels

# Mandatory environment variables for GDAL and PROJ data resolution
ENV GDAL_DATA=/usr/share/gdal
ENV PROJ_LIB=/usr/share/proj
ENV PYTHONUNBUFFERED=1

This pattern guarantees that only runtime-essential binaries survive into the final layer. For projects that also ship native libraries compiled outside pip, consult Native Library Compilation for Serverless for ABI compatibility across Alpine, Debian, and Amazon Linux runtimes.

2. Pin Dependencies and Isolate Layers

Geospatial Python packages pull in transitive system dependencies. Unpinned versions cause unpredictable layer cache invalidation and silent ABI mismatches between the builder and runtime stages. Use exact version pinning (package==1.2.3) in requirements.txt or pyproject.toml.

Separate system packages from Python packages into distinct RUN instructions. Docker caches each layer independently; isolating heavy system installs ensures that updating a single Python dependency does not force a full recompilation of GDAL or PROJ. When managing wheels alongside system libraries at scale, Python Layer Management and Size Reduction covers strategies for decoupling dependency resolution from image assembly across multiple Lambda functions that share the same GDAL version.

Concrete layer-isolation rules:

• Group apt-get update and apt-get install in a single RUN statement. Splitting them lets Docker cache a stale package index, causing reproducibility failures days or weeks later.
• Always pass --no-install-recommends to exclude documentation, man pages, and GUI utilities (libgdal-dev alone recommends 200+ MB of extras).
• Install Python packages via pre-compiled wheels (pip install --only-binary :all:) when available to skip local compilation in the runtime stage.

3. Strip Binaries and Purge Caches

Compiled .so files embed debug symbols, relocation tables, and DWARF sections that inflate binary size by 30–60% without providing any runtime benefit. The strip utility removes these sections safely for production deployments.

# In the builder stage, after compiling wheels
RUN find /wheels -name "*.so" -exec strip --strip-unneeded {} + \
    && find /usr/lib -name "*.so*" -exec strip --strip-unneeded {} +

# In the runtime stage, after installing wheels
RUN find /usr/local/lib/python3.*/site-packages -name "*.so" \
        -exec strip --strip-unneeded {} + \
    && rm -rf \
        /var/cache/apt /var/lib/apt/lists \
        /usr/share/doc /usr/share/man /usr/share/locale \
        /root/.cache

Critical: use --strip-unneeded rather than --strip-all. The --strip-all flag removes dynamic symbol tables that dlopen requires when GDAL registers format drivers at startup. Over-stripping produces a runtime error such as:

ImportError: /usr/local/lib/python3.10/site-packages/osgeo/_gdal.cpython-310-x86_64-linux-gnu.so:
  undefined symbol: GDALAllRegister

Validate shared library dependencies after stripping with ldd:

docker run --rm <image> ldd /usr/local/lib/python3.10/site-packages/osgeo/_gdal.cpython-310-x86_64-linux-gnu.so
# All entries must resolve — "not found" means an over-stripped or missing library

4. Align with Serverless Runtime Constraints

Serverless platforms enforce non-root execution, read-only filesystems outside /tmp, and strict environment variable scoping. Configure the container before shipping:

# Drop root privileges — required by Lambda and Cloud Run security policies
RUN groupadd -r gisuser && useradd -r -g gisuser gisuser
USER gisuser

# GDAL and PROJ must resolve their data directories at import time.
# Missing these causes silent fallback to default WGS84 or complete driver failure.
ENV GDAL_DATA=/usr/share/gdal
ENV PROJ_LIB=/usr/share/proj
ENV PROJ_NETWORK=OFF
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1

# Startup health check — fails fast if data paths are misconfigured
CMD ["python", "-c", "from osgeo import gdal, osr; assert gdal.GetConfigOption('GDAL_DATA'); srs = osr.SpatialReference(); srs.ImportFromEPSG(4326); print('GDAL OK')"]

PROJ_NETWORK=OFF prevents PROJ 9+ from attempting network downloads of CDN-hosted grid shift files during Lambda cold starts, where outbound network access may be blocked by VPC configuration. For workloads that actually need online grid shift corrections, set this to ON and ensure your Lambda VPC has an internet-facing NAT gateway.

For raster workloads that cache tile matrices or spatial indexes to /tmp, request additional ephemeral storage. AWS Lambda supports up to 10 GB when configured:

# AWS SAM template
Resources:
  GeoProcessor:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      EphemeralStorage:
        Size: 4096  # MB — increase for GeoTIFF extraction workloads
      MemorySize: 3008
      Timeout: 900

5. Validate, Scan, and Gate in CI/CD

Optimization is a continuous process. Without CI gates, a new transitive dependency reintroduces 300 MB and nobody notices until the next cold-start regression. Automate size checks, vulnerability scans, and layer inspection:

# .github/workflows/geo-image.yml
jobs:
  build-and-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build image
        run: docker build --target runtime -t geoprocessor:${{ github.sha }} .

      - name: Scan for vulnerabilities
        run: |
          docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
            aquasec/trivy image --exit-code 1 --severity HIGH,CRITICAL \
            geoprocessor:${{ github.sha }}

      - name: Enforce size gate
        run: |
          SIZE=$(docker image inspect geoprocessor:${{ github.sha }} \
            --format='{{.Size}}')
          echo "Image size: ${SIZE} bytes"
          if [ "${SIZE}" -gt 524288000 ]; then
            echo "ERROR: Image exceeds 500 MB limit (${SIZE} bytes). Aborting."
            exit 1
          fi

      - name: Inspect layers
        run: |
          docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
            wagoodman/dive geoprocessor:${{ github.sha }} --ci

For teams targeting ultra-lightweight deployments, consider musl-based distributions. Alpine Linux reduces base image overhead by roughly 70%, but requires either source compilation or pre-built Alpine-compatible GDAL packages. Building Minimal Docker Images with Alpine and GDAL covers proven patterns for musl compatibility, static linking, and Alpine-specific gdal package repositories.

Measurement and Verification

Run the following benchmark script inside the built container to confirm the optimization delivered measurable improvements. It measures both image pull time (simulated) and library initialization time:

import time
import subprocess
import sys

def benchmark_gdal_init():
    """
    Measures cold import time for the geospatial stack.
    Expected: < 2.0 s on a 512 MB Lambda with a warm layer cache.
    """
    start = time.perf_counter()
    from osgeo import gdal, ogr, osr
    import rasterio
    import fiona
    elapsed = time.perf_counter() - start

    print(f"Import time: {elapsed:.3f}s")
    assert elapsed < 5.0, f"Import too slow: {elapsed:.1f}s — image may still be bloated"

    # Verify GDAL data paths resolved correctly
    gdal_data = gdal.GetConfigOption("GDAL_DATA")
    assert gdal_data, "GDAL_DATA not set — coordinate transformations will fail"
    print(f"GDAL_DATA: {gdal_data}")

    # Confirm driver count — low counts indicate missing format drivers
    driver_count = gdal.GetDriverCount()
    assert driver_count > 50, f"Only {driver_count} GDAL drivers — expected 50+"
    print(f"GDAL drivers available: {driver_count}")

    # Validate a CRS round-trip
    srs = osr.SpatialReference()
    srs.ImportFromEPSG(32632)  # UTM zone 32N
    assert "32632" in srs.GetAuthorityCode(None) or srs.IsProjected()
    print("CRS round-trip: OK")

if __name__ == "__main__":
    benchmark_gdal_init()
    print("All checks passed.")

Measure image size before and after each optimization pass:

# Compressed size (what gets pulled from the registry)
docker save geoprocessor:latest | gzip | wc -c | awk '{printf "%.1f MB\n", $1/1048576}'

# Uncompressed layer breakdown
docker history geoprocessor:latest --format "{{.Size}}\t{{.CreatedBy}}"

Expected output ranges after a full optimization pass:

Failure Modes and Debugging

1. ImportError: undefined symbol after stripping

Signature: ImportError: /path/to/_gdal.cpython-310.so: undefined symbol: GDALAllRegister

Cause: strip --strip-all removed the dynamic symbol table. GDAL plugins use dlopen to load format drivers at runtime; --strip-all breaks dynamic linking.

Fix: Replace --strip-all with --strip-unneeded in all strip invocations. Re-run ldd on the .so file — all symbols must resolve.

2. PROJ: proj_create_from_database: Cannot open at runtime

Signature: proj_create_from_database: /usr/share/proj/proj.db not found or a silent fallback producing incorrect coordinate output.

Cause: PROJ_LIB is unset or points to a directory that did not survive the multi-stage copy. This is common when the runtime base image installs proj-bin but not proj-data, or when the data directory path differs between builder and runtime.

Fix: Explicitly set ENV PROJ_LIB=/usr/share/proj in the runtime stage Dockerfile and verify with:

docker run --rm geoprocessor:latest python -c "import pyproj; print(pyproj.datadir.get_data_dir())"

3. Container exits immediately with Permission denied on Lambda

Signature: Lambda returns Runtime.ExitError with exit code 126 immediately after image pull.

Cause: The container entrypoint binary is not executable by the non-root gisuser, or the /tmp mount is not writable. Lambda runs containers as UID 993 by default.

Fix: Ensure the CMD or ENTRYPOINT binary has 755 permissions and that your app writes scratch files to /tmp, not to the application directory:

RUN chmod 755 /app/handler.py
ENV TMPDIR=/tmp

4. Image size regression after dependency update

Signature: CI size gate fails after updating rasterio or shapely to a new minor version.

Cause: A transitive dependency (often numpy, pyarrow, or scipy) upgraded and pulled in a larger wheel that includes optional bundled libraries (e.g., numpy shipping its own OpenBLAS).

Fix: Audit the wheel contents with unzip -l <wheel>.whl | sort -k3 -rn | head -20 to identify the largest files. Use pip install --no-binary numpy to compile numpy without bundled BLAS, or pin the offending package at the previous version until you can audit the change.

5. GDAL driver count drops after switching base image

Signature: gdal.GetDriverCount() returns 30–40 instead of 80–120. Formats like HDF5, NetCDF, or JPEG2000 stop working.

Cause: The runtime base image installs a minimal GDAL package (libgdal30) that omits optional format plugins. Format drivers are loaded from /usr/lib/gdalplugins/ at startup.

Fix: Install the optional driver packages:

RUN apt-get install -y --no-install-recommends \
    libgdal30 \
    gdal-plugins \    
    libhdf5-103 \
    libnetcdf19

Cost and Scaling Considerations

Smaller images produce compound cost savings across three dimensions:

ECR/Artifact Registry storage. AWS ECR charges $0.10 per GB-month. A team deploying 50 image versions per month at 1.8 GB pays ~$9/month per repository; at 420 MB it pays ~$2.10. Multiply by the number of microservices.

Cold-start frequency. Under AWS Lambda’s scale-to-zero model, every function invocation after an idle period triggers a full container pull. At 1.8 GB compressed (~700 MB), a cold start on a 3 GB memory Lambda takes 20–35 seconds. At 420 MB compressed (~175 MB), it takes 4–8 seconds. For user-facing APIs this directly impacts P99 latency.

Provisioned concurrency cost. If your cold starts are too slow to tolerate under the unoptimized image, you pay for provisioned concurrency to pre-warm instances ($0.015 per GB-hour on Lambda). A 420 MB image may eliminate the need for provisioned concurrency entirely for non-latency-sensitive batch workloads, saving $50–$300/month depending on the function’s memory allocation.

When Alpine is worth the complexity. Switching from Ubuntu 22.04 to Alpine 3.19 reduces the base layer from ~80 MB to ~8 MB, with the final GIS image landing around 200–280 MB. The tradeoff is that all C-extension wheels must be compiled against musl, which adds 15–30 minutes of CI build time and complicates dependency auditing. Prefer Alpine when cold-start P99 is a hard product requirement and you have CI infrastructure to support musl compilation. For general-purpose batch processing pipelines, Debian-slim with multi-stage builds hits the 500 MB target with significantly less operational overhead.

Provisioned concurrency vs. image optimization. The two are not mutually exclusive but target different failure modes. Image optimization reduces cold-start duration; provisioned concurrency eliminates cold starts entirely for a steady base load. Optimize the image first to reduce the cost of provisioned concurrency; then add concurrency only for the latency-critical paths.

Frequently Asked Questions

What is the typical compressed size of a GDAL/rasterio Docker image before optimization?

Unoptimized images that install GDAL, PROJ, rasterio, and geopandas from a standard Ubuntu or Debian base typically reach 1.5–2.5 GB uncompressed. After multi-stage builds and binary stripping, the same stack fits in 350–500 MB uncompressed, or 150–220 MB compressed.

Can I use Alpine Linux for a GDAL container?

Yes, but Alpine uses musl libc rather than glibc. Pre-compiled GDAL wheels from PyPI are built against glibc and will segfault on Alpine. You must either compile from source against musl or use the osgeo/gdal:alpine-* images as your base. See Building Minimal Docker Images with Alpine and GDAL for the full procedure.

Does stripping .so files break GDAL or PROJ at runtime?

Stripping debug symbols with --strip-unneeded is safe for production. Stripping with --strip-all can remove dynamic symbol tables needed by dlopen, breaking GDAL driver registration. Always validate with ldd and a smoke-test import after stripping.

Should I use uv instead of pip for building wheels in Docker?

Yes, uv resolves geospatial dependency trees significantly faster than pip and produces deterministic lockfiles. Replace pip wheel with uv pip compile + uv pip install in the builder stage. The wheel output format is identical; the CI build time savings are typically 40–70% for deep dependency graphs like rasterio + geopandas.

Related

• Building Minimal Docker Images with Alpine and GDAL
• Python Layer Management and Size Reduction
• Native Library Compilation for Serverless
• Cold Start Mapping for Python GDAL
• Ephemeral Storage Limits in AWS Lambda

Back to Packaging & Dependency Management for Serverless GIS