Cold Start Tuning for GDAL Container Images on Lambda
Lambda never pulls a container image the way docker run does: it flattens the image once at deploy time into small content-addressed blocks, caches them per availability zone, and faults in only the blocks an execution environment actually reads. That makes cold-start time a function of which image layer changed, not of image size — so put the GDAL install early and the handler last, pin the base image by digest, and never rewrite the filesystem in a late layer. On a 1,240 MB GDAL image, a deploy that touches only the handler cold-starts in 1.94 s against 3.18 s for one that invalidates the GDAL layer.
Context
The Docker container optimization discipline gets a GDAL image from 1.8 GB down to a few hundred megabytes, and the reason it works is real: fewer bytes are fewer bytes. But shrinking is a blunt instrument, and past a point the remaining size is the drivers you need. What is left to tune is when the bytes are paid for, and that is where a container-image function stops resembling a zip one.
A zip-packaged function has its archive fetched and unpacked into /var/task before the runtime starts — a single serial cost proportional to the package, which is what the cold start budget for Python GDAL measures. A container-image function has no archive to unpack. Lambda converts the image at deploy time into a flattened block store, keeps those blocks in a cache shared across the availability zone, and lets the microVM page them in on demand. Blocks are content-addressed, so identical bytes across image versions — and across functions built on the same base — resolve to the same cache entry. Redeploying an image whose GDAL layer is unchanged fetches almost nothing.
The failure mode this page prevents is subtle: a Dockerfile that is perfectly correct, produces a small image, and still cold-starts badly on every deploy because a late RUN rewrites files the earlier layers created, invalidating the blocks that held the entire GDAL tree.
Where the time actually goes
Break a cold container start into the phases that can be tuned separately. The numbers below are a 1,240 MB GDAL image on a 1,769 MB x86_64 function, measured end to end from the client rather than from the log.
Only the first segment is specific to container packaging, and it is also the only one that moves when you reorder the Dockerfile. The other four are the same shared-library resolution and driver-registration work a zip function does, and they respond to the same levers — a smaller driver set, lazy imports, provisioned concurrency. Optimising them is covered by the parent guide and by reducing Python GDAL cold starts with provisioned concurrency; optimising the first segment is what the rest of this page is about.
How the layer-chunk cache works
Think of the deployed image not as a stack of tarballs but as a flat array of small blocks, each identified by the hash of its contents. Three consequences follow, and all three are actionable.
First, blocks that are byte-identical are stored and fetched once. Two functions built on the same base image digest share the base’s blocks; two versions of one function share every block the change did not touch. Second, a block is only fetched if something reads it — the ~40% of a GDAL image holding drivers your handler never registers costs storage and deploy time but not cold-start latency. Third, and this is where Dockerfiles go wrong, a file rewritten in a later layer produces entirely new blocks for that file. A RUN chown -R appuser /usr/local after the GDAL install does not “add a small layer”; it re-materialises every file under /usr/local with new content hashes, and the cached GDAL blocks become dead weight.
That last point is why the usual build-cache ordering advice pays a second time here. Ordering layers so heavy, stable content sits early is standard practice for keeping docker build fast — the same ordering keeps the runtime block cache warm across deploys, which is worth far more.
Prerequisites
- Runtime: Python 3.11 on
x86_64,PackageType: Image, memory 1,769 MB, timeout 300 s (ceiling 15 min),/tmpat the 512 MB default unless the handler needs more of the 10,240 MB available - Base image:
public.ecr.aws/lambda/python:3.11, pinned by digest so every function in the account resolves to identical base blocks - ECR repository in the same region as the function, with
ecr:BatchGetImageandecr:GetDownloadUrlForLayergranted to thelambda.amazonaws.comservice principal in the repository policy - BuildKit with attestations disabled:
docker buildx build --provenance=false --platform linux/amd64, because the OCI image index buildx emits by default is rejected byCreateFunctionwith an unsupported-manifest error - Pinned versions: GDAL 3.9.0, PROJ 9.4.0, GEOS 3.12.1,
rasterio==1.4.3,pyproj==3.7.0 - IAM for measurement:
lambda:UpdateFunctionConfigurationandlambda:InvokeFunction, plusxray:PutTraceSegmentsif you want the initialisation subsegment - Runtime environment variables baked into the image so they are part of a stable layer rather than a mutable function configuration:
GDAL_DATA=/usr/local/share/gdal PROJ_LIB=/usr/local/share/proj LD_LIBRARY_PATH=/usr/local/lib GDAL_PAM_ENABLED=NO
Implementation
The ordering below is the whole technique. Heavy and stable first, volatile last, and nothing after the GDAL copy that touches a file the GDAL copy created.
# syntax=docker/dockerfile:1.7
# Build: docker buildx build --provenance=false --platform linux/amd64 -t gdal-lambda:3.9.0 .
# ---- builder: compiled here, never shipped -------------------------------
FROM public.ecr.aws/lambda/python:3.11@sha256:9f2c1a0b3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f60718293a4b5c6d7e8 AS builder
RUN dnf install -y gcc gcc-c++ cmake make binutils tar gzip \
sqlite-devel libtiff-devel libcurl-devel zlib-devel >/dev/null
COPY build_gdal.sh /build_gdal.sh
RUN bash /build_gdal.sh # installs GDAL 3.9.0 / PROJ 9.4.0 / GEOS 3.12.1 into /usr/local
# ---- runtime -------------------------------------------------------------
# Pinned by digest, not by tag: every function in the account that pins the same
# digest resolves to the same cached blocks instead of a private copy.
FROM public.ecr.aws/lambda/python:3.11@sha256:9f2c1a0b3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f60718293a4b5c6d7e8
# LAYER 1 — runtime shared libraries. Changes a few times a year.
RUN dnf install -y libcurl-minimal expat libtiff sqlite-libs \
&& dnf clean all && rm -rf /var/cache/dnf
# LAYER 2 — the heavy, stable one: ~310 MB of GDAL, PROJ, GEOS and their data.
# Nothing below this line may rewrite anything under /usr/local, or every
# block holding this layer is invalidated and refetched on the next cold start.
COPY --from=builder /usr/local/lib /usr/local/lib
COPY --from=builder /usr/local/share/gdal /usr/local/share/gdal
COPY --from=builder /usr/local/share/proj /usr/local/share/proj
# LAYER 3 — metadata only, a few hundred bytes. Explicit, never left to the
# function configuration, so the values travel with the image they match.
ENV GDAL_DATA=/usr/local/share/gdal \
PROJ_LIB=/usr/local/share/proj \
LD_LIBRARY_PATH=/usr/local/lib \
GDAL_PAM_ENABLED=NO \
PYTHONDONTWRITEBYTECODE=1
# LAYER 4 — Python dependencies. Changes when requirements.txt changes.
COPY requirements.txt ${LAMBDA_TASK_ROOT}/
RUN pip install --no-cache-dir -r ${LAMBDA_TASK_ROOT}/requirements.txt \
-t ${LAMBDA_TASK_ROOT}
# LAYER 5 — the handler. Changes on every commit, and is the only layer that
# should: ~40 KB of new blocks per deploy instead of ~310 MB.
COPY app.py ${LAMBDA_TASK_ROOT}/
CMD ["app.lambda_handler"]
Three instructions are deliberately absent. There is no RUN chown -R or COPY --chown over /usr/local — Lambda already runs the container as a non-root user and the recursive rewrite would cost the entire GDAL layer. There is no --squash, which collapses the image into one layer and destroys every opportunity for block sharing between versions. And there is no late strip pass: symbol stripping belongs in the builder stage, as the multi-stage Cloud Run recipe does it, precisely so the shipped bytes are written exactly once.
Verification
The measurement has to be end to end, because the phase you are tuning is invisible in the log line most people read.
# measure_container_cold_start.py — forces cold environments and times them.
import base64
import json
import re
import statistics
import time
import boto3
lam = boto3.client("lambda")
FN = "gdal-image-fn"
def force_cold():
# Changing any environment variable replaces the execution environment, so
# the next invocation is guaranteed cold and must fetch its own blocks.
lam.update_function_configuration(
FunctionName=FN,
Environment={"Variables": {"COLD_NONCE": str(time.time_ns()),
"GDAL_DATA": "/usr/local/share/gdal",
"PROJ_LIB": "/usr/local/share/proj",
"LD_LIBRARY_PATH": "/usr/local/lib",
"GDAL_PAM_ENABLED": "NO"}},
)
waiter = lam.get_waiter("function_updated_v2")
waiter.wait(FunctionName=FN)
def one_cold_start():
force_cold()
t0 = time.perf_counter()
r = lam.invoke(FunctionName=FN, LogType="Tail", Payload=b"{}")
wall_ms = (time.perf_counter() - t0) * 1000
tail = base64.b64decode(r["LogResult"]).decode()
# Init Duration covers runtime + handler init only. The block-fetch phase
# happens before it, so the gap between wall clock and Init Duration is the
# number this page is about.
m = re.search(r"Init Duration: ([\d.]+) ms", tail)
return wall_ms, float(m.group(1)) if m else 0.0
samples = [one_cold_start() for _ in range(10)]
wall = statistics.median(s[0] for s in samples)
init = statistics.median(s[1] for s in samples)
print(json.dumps({
"end_to_end_p50_ms": round(wall, 1),
"report_init_duration_p50_ms": round(init, 1),
"block_fetch_p50_ms": round(wall - init, 1),
}, indent=2))
Run it twice: once after a deploy that rebuilt the GDAL layer, once after a deploy that changed only app.py.
{ "deploy": "GDAL layer rebuilt",
"end_to_end_p50_ms": 3180.4, "report_init_duration_p50_ms": 1392.6, "block_fetch_p50_ms": 1787.8 }
{ "deploy": "app.py only",
"end_to_end_p50_ms": 1943.1, "report_init_duration_p50_ms": 1388.2, "block_fetch_p50_ms": 554.9 }
Init Duration is flat to within 5 ms across both. Everything the reordering bought — 1.24 s of it — sits entirely in the difference between wall clock and the reported figure, which is why a team watching only the REPORT line concludes the ordering made no difference.
What SnapStart does and does not cover
SnapStart is the natural next question, because a snapshot of a fully initialised GDAL process would skip driver registration and proj.db entirely. For container images the answer is short.
Adopting SnapStart therefore means abandoning the container image, going back to a zip package, and going back under 250 MB unzipped across the function and all five layers — the ceiling that produced the image in the first place, as set out in comparing layers, container images, and EFS for GDAL. For a slim GDAL build that trade can be worth it. For a build carrying NetCDF, HDF5, and GRIB it is not available at any price, and block-cache ordering plus provisioned concurrency is the whole toolkit.
Gotchas and Edge Cases
docker buildxrejects your image before Lambda ever sees a cold start. Modern BuildKit attaches provenance and SBOM attestations by default, producing an OCI image index rather than a plain manifest.CreateFunctionfails with an unsupported manifest or media type error that says nothing about attestations. Build with--provenance=false --sbom=falseand a single explicit--platform.- A tag is not a digest, and the difference shows up as a cold-start regression.
FROM public.ecr.aws/lambda/python:3.11resolves to whatever AWS published this week. When that moves, every block in your image shifts and the first deploy afterwards refetches the whole thing. Pin the digest and bump it deliberately, on the same cadence as the version pins in your CI dependency sync. - Cross-region ECR is a cold-start tax, not just a data-transfer bill. The block store is populated per region and per availability zone. A function in
eu-west-1pointed at a repository inus-east-1is not merely slower to deploy; every uncached block crosses the Atlantic during initialisation. Replicate the repository rather than sharing one. - Provisioned concurrency hides this, and hides it expensively. Pre-initialised environments have their blocks already resident, so the phase you are tuning disappears from the latency you observe while continuing to cost money on every scale-out beyond the provisioned count. Order the layers first, then provision, and keep measuring the unprovisioned path — that is what a traffic spike will actually experience.
Frequently Asked Questions
Does Lambda download the whole container image before the first invocation?
No. The image is converted once at deploy time into small content-addressed blocks, and an execution environment fetches only the blocks it reads. A 1,240 MB GDAL image whose handler registers a modest driver set may touch well under half of itself, which is why total image size predicts cold-start time so poorly. What predicts it is how much of the image changed since the blocks were last cached in that availability zone.
Why does Init Duration stay flat when my container cold start clearly got slower?
Because Init Duration starts once the execution environment already has a filesystem to run in. The block-fetch phase that builds that filesystem happens earlier and is not part of the figure. A deploy that invalidates the GDAL image layer can add well over a second of end-to-end latency while Init Duration moves by single-digit milliseconds. Time the invocation at the client, or read the Initialization subsegment in an X-Ray trace.
Can I use SnapStart with a GDAL container image?
No. SnapStart covers zip-packaged functions on x86_64 only, and is mutually exclusive with container images, arm64, provisioned concurrency, an EFS mount, and ephemeral storage above 512 MB. Choosing it means returning to a zip package under the 250 MB unzipped ceiling, which a full-driver GDAL build cannot meet — so for those functions the levers are layer ordering, a pinned base digest, and provisioned concurrency.
Related
- Docker Container Optimization for GIS — the size work that comes before any ordering work
- Multi-Stage Dockerfile for GDAL on Cloud Run — the builder stage this Dockerfile copies from, and where stripping belongs
- Comparing Layers, Container Images, and EFS for GDAL — whether the image was the right mechanism at all
- Cold Start Mapping for Python GDAL — the four phases that behave identically in zip and image packaging
- Reducing Python GDAL Cold Starts with Provisioned Concurrency — the lever to reach for once ordering is exhausted