Cross-Compiling GEOS for arm64 Graviton Lambda

Pull public.ecr.aws/lambda/python:3.11 with --platform linux/arm64, compile GEOS 3.12.1, PROJ 9.4.0, and GDAL 3.9.0 into /opt/lib as libgeos_c.so.1, libproj.so.25, and libgdal.so.35, then install the Python layer with pip --platform manylinux_2_28_aarch64 --abi cp311 --only-binary :all: and publish it with --compatible-architectures arm64. Nothing from an x86_64 layer transfers: every .so is a different ELF machine type, and the aarch64 loader rejects the wrong one before your handler is reached. On a 1,769 MB tiling function the migration takes a million tile invocations from $124.00 to $92.20.

Context

The native library compilation discipline is built around one constraint: the build environment must match the runtime’s C ABI, which on modern Lambda means Amazon Linux 2023 and glibc 2.34. Architecture is the stricter half of that constraint. A glibc mismatch sometimes surfaces as a single missing symbol you can work around; an architecture mismatch cannot be worked around at all. The ELF header of an x86_64 object carries Machine: Advanced Micro Devices X86-64, and /lib/ld-linux-aarch64.so.1 will not map it — the function fails at import with wrong ELF class or cannot open shared object file, exactly the class of failure the Amazon Linux 2023 layer recipe exists to prevent on x86_64.

That makes an arm64 migration a full second build chain rather than a flag on the existing one. The chain below is the same shape as the x86_64 chain, but every stage has to be told the target explicitly — the container platform, the CMake toolchain, the wheel platform tag, and the layer’s compatible-architectures metadata. Miss any one of them and the build still succeeds; it simply produces an artifact that fails at the first cold start.

The arm64 Lambda build chain, stage by stageFive numbered stages of an arm64 GEOS build. First, register the aarch64 binfmt handler on an x86_64 host. Second, pull the Amazon Linux 2023 Python 3.11 image with platform linux arm64. Third, compile GEOS 3.12.1, PROJ 9.4.0 and GDAL 3.9.0 with CMake into slash layer. Fourth, install shapely and pyproj wheels with the manylinux 2 28 aarch64 platform tag. Fifth, publish the layer version with compatible-architectures arm64. Each stage carries the exact flag that names the target.Five places the target architecture has to be stated, and what happens if one is missed1Register the aarch64 handlerOnly on an x86_64 build host; once per bootbinfmt --install arm642Pull the AL2023 aarch64 imageglibc 2.34, loader /lib/ld-linux-aarch64.so.1--platform linux/arm643Compile GEOS, PROJ, GDALDependency order; installs into /layer/lib and /layer/share3.12.1 / 9.4.0 / 3.9.04Install the wheels that already existshapely and pyproj ship Graviton binaries — download, never buildmanylinux_2_28_aarch645Publish as a separate layerOwn layer name, own CI cache key, own ARN--compatible-architectures arm64
Every one of these stages succeeds if you forget the flag — it just produces an x86_64 artifact. The first thing that notices is the aarch64 loader, at the first cold start.

What you download and what you must rebuild

The useful question is not “how do I compile GEOS for aarch64” but “what actually has no aarch64 wheel”. Since the manylinux_2_28_aarch64 policy stabilised, the mainstream Python geospatial stack publishes Graviton wheels: shapely vendors libgeos_c inside shapely.libs, rasterio vendors a full GDAL inside rasterio.libs, and pyproj vendors PROJ plus its proj.db. For a handler that only calls those three, the correct arm64 build compiles nothing at all — it is the same pip invocation with one tag changed.

You reach for the compiler when something has to link against a GEOS you control rather than a GEOS hidden inside another package’s .libs directory. That is the case for a source-built GDAL, for PDAL, for MapServer or tippecanoe-style binaries, and for the slimmed single-libgdal layout described in GDAL wheel size reduction with pip --no-binary, where the Python bindings link against a system libgdal instead of shipping their own.

Packages that ship aarch64 wheels versus packages that must be rebuiltTwo panels. The left panel lists what pip downloads for aarch64: shapely 2.0.6 with vendored libgeos_c, pyproj 3.7.0 with vendored PROJ and proj.db, rasterio 1.4.3 with a vendored GDAL, numpy and fiona. The right panel lists what has no aarch64 wheel and must be compiled: a standalone libgeos_c.so.1 shared across the layer, a source-built libgdal linked against your own GEOS and PROJ, PDAL and MapServer style binaries, and any GDAL driver plugin. The verdict line states that a handler using only shapely, pyproj and rasterio compiles nothing at all.Which half of the stack actually needs a compiler on aarch64Downloaded as manylinux_2_28_aarch64shapely 2.0.6 — vendors libgeos_c in shapely.libspyproj 3.7.0 — vendors libproj and proj.dbrasterio 1.4.3 — vendors a full GDAL in rasterio.libsnumpy, fiona, affine, certifiCost: one changed --platform tagRebuilt from source on aarch64A standalone libgeos_c.so.1 shared across the layerlibgdal.so.35 linked against your GEOS and PROJPDAL, MapServer, tippecanoe-class binariesAny out-of-tree GDAL driver pluginCost: 4 min native, 25 min under emulationA handler that calls only shapely, pyproj and rasterio compiles nothing — the arm64 build is the x86_64 build with one tagchanged.
The compiler only appears once something outside a wheel has to link against GEOS. Check this list before writing a build script — for many handlers the correct arm64 build is one changed platform tag.

Prerequisites

  • Runtime: Python 3.11 on arm64 (Graviton2 on all current Lambda regions, Graviton3 where available)
  • Build image: public.ecr.aws/lambda/python:3.11 pulled with --platform linux/arm64 — Amazon Linux 2023, glibc 2.34, loader /lib/ld-linux-aarch64.so.1
  • Emulation, on an x86_64 host only: docker run --privileged --rm tonistiigi/binfmt --install arm64 to register the qemu-aarch64 handler with the kernel
  • Pinned source versions: GEOS 3.12.1, PROJ 9.4.0, GDAL 3.9.0, SQLite 3.45.3 — the same set the CI cache key is built from, with arm64 added to that key
  • Pinned wheels: shapely==2.0.6, pyproj==3.7.0, rasterio==1.4.3, all of which publish cp311-cp311-manylinux_2_28_aarch64 artifacts
  • System tools inside the container: gcc, gcc-c++, cmake, make, binutils, sqlite-devel, libtiff-devel, libcurl-devel, tar
  • Runtime environment variables (set on the function, not the build):
    code 
    GDAL_DATA=/opt/share/gdal
PROJ_LIB=/opt/share/proj
LD_LIBRARY_PATH=/opt/lib
  • IAM for the publish step: lambda:PublishLayerVersion on the arm64 layer ARN and s3:PutObject on the staging bucket, since a GDAL layer zip clears the 50 MB direct-upload threshold

Implementation

The script below runs on the host and drives the aarch64 container. It builds the three libraries in dependency order, downloads rather than compiles anything that already ships an aarch64 wheel, and lays out the /opt-shaped directories the layer will mount over.

bash 
#!/usr/bin/env bash
# build_arm64_geo_layer.sh — GEOS + PROJ + GDAL for Graviton Lambda.
set -euo pipefail

GEOS_VERSION="3.12.1"
PROJ_VERSION="9.4.0"
GDAL_VERSION="3.9.0"
LAYER_DIR="$(pwd)/layer-arm64"
SITE="python/lib/python3.11/site-packages"

rm -rf "${LAYER_DIR}" && mkdir -p "${LAYER_DIR}"

# On an x86_64 host, register the aarch64 binfmt handler once per boot.
docker run --privileged --rm tonistiigi/binfmt --install arm64 >/dev/null

docker run --rm \
  --platform linux/arm64 \
  -v "${LAYER_DIR}:/layer" \
  public.ecr.aws/lambda/python:3.11 \
  bash -c "
    set -euo pipefail
    dnf install -y gcc gcc-c++ cmake make binutils tar gzip \
      sqlite-devel libtiff-devel libcurl-devel zlib-devel >/dev/null

    # Inside a --platform arm64 container the whole userland is aarch64 and
    # binfmt runs it, so CMake never enters cross-compiling mode and every
    # compile-and-run feature probe executes for real. That is why no
    # CMAKE_CROSSCOMPILING_EMULATOR or toolchain file appears below.
    export PREFIX=/layer
    export CFLAGS='-O2 -fPIC'
    export CXXFLAGS='-O2 -fPIC'
    export PKG_CONFIG_PATH=\"\${PREFIX}/lib/pkgconfig\"
    export LD_LIBRARY_PATH=\"\${PREFIX}/lib\"

    cd /tmp
    curl -sL https://download.osgeo.org/geos/geos-${GEOS_VERSION}.tar.bz2 | tar xj
    curl -sL https://download.osgeo.org/proj/proj-${PROJ_VERSION}.tar.gz   | tar xz
    curl -sL https://github.com/OSGeo/gdal/releases/download/v${GDAL_VERSION}/gdal-${GDAL_VERSION}.tar.gz | tar xz

    # GEOS first: PROJ does not need it, but GDAL links against libgeos_c.
    # BUILD_GEOSOP=OFF drops a CLI binary that would be dead weight in /opt.
    cmake -S geos-${GEOS_VERSION} -B /tmp/b-geos \
      -DCMAKE_INSTALL_PREFIX=\"\${PREFIX}\" \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
      -DBUILD_SHARED_LIBS=ON -DBUILD_TESTING=OFF -DBUILD_GEOSOP=OFF
    cmake --build /tmp/b-geos --parallel \"\$(nproc)\" && cmake --install /tmp/b-geos

    # PROJ generates proj.db by executing sqlite3 during the build. Point it at
    # the container's own sqlite3 — under emulation that binary is aarch64 too,
    # which is why this step works here and breaks in a true cross-toolchain.
    cmake -S proj-${PROJ_VERSION} -B /tmp/b-proj \
      -DCMAKE_INSTALL_PREFIX=\"\${PREFIX}\" \
      -DCMAKE_BUILD_TYPE=Release \
      -DEXE_SQLITE3=/usr/bin/sqlite3 \
      -DBUILD_SHARED_LIBS=ON -DBUILD_TESTING=OFF -DBUILD_PROJSYNC=OFF
    cmake --build /tmp/b-proj --parallel \"\$(nproc)\" && cmake --install /tmp/b-proj

    cmake -S gdal-${GDAL_VERSION} -B /tmp/b-gdal \
      -DCMAKE_INSTALL_PREFIX=\"\${PREFIX}\" \
      -DCMAKE_BUILD_TYPE=Release \
      -DPROJ_ROOT=\"\${PREFIX}\" -DGEOS_ROOT=\"\${PREFIX}\" \
      -DBUILD_SHARED_LIBS=ON -DBUILD_TESTING=OFF -DBUILD_PYTHON_BINDINGS=OFF
    cmake --build /tmp/b-gdal --parallel \"\$(nproc)\" && cmake --install /tmp/b-gdal

    # Wheels that already ship aarch64 binaries: download, never compile.
    # --platform forces --only-binary, which is exactly the guarantee we want.
    pip install \
      --only-binary :all: \
      --platform manylinux_2_28_aarch64 \
      --python-version 3.11 --implementation cp --abi cp311 \
      --target /layer/${SITE} \
      shapely==2.0.6 pyproj==3.7.0

    find \"\${PREFIX}/lib\" -name '*.so*' -exec strip --strip-unneeded {} + 2>/dev/null || true
    find /layer/${SITE} -type d -name '__pycache__' -prune -exec rm -rf {} +
  "

cd "${LAYER_DIR}" && zip -qr9 ../geo-arm64-layer.zip . && cd ..
aws s3 cp geo-arm64-layer.zip s3://my-lambda-artifacts/layers/geo-arm64.zip

# --compatible-architectures is metadata for the attach-time check; the bytes
# inside the zip are what actually has to be aarch64.
aws lambda publish-layer-version \
  --layer-name "geo-3-9-py311-arm64" \
  --description "GEOS ${GEOS_VERSION} / PROJ ${PROJ_VERSION} / GDAL ${GDAL_VERSION}, aarch64" \
  --content "S3Bucket=my-lambda-artifacts,S3Key=layers/geo-arm64.zip" \
  --compatible-runtimes python3.11 \
  --compatible-architectures arm64

Verification

Read the ELF header before you read a version string. A version string proves the file parses; the header proves it will load on Graviton at all.

bash 
docker run --rm \
  --platform linux/arm64 \
  -v "$(pwd)/layer-arm64:/opt:ro" \
  -e GDAL_DATA=/opt/share/gdal \
  -e PROJ_LIB=/opt/share/proj \
  -e LD_LIBRARY_PATH=/opt/lib \
  -e PYTHONPATH=/opt/python/lib/python3.11/site-packages \
  public.ecr.aws/lambda/python:3.11 \
  bash -c '
    readelf -h /opt/lib/libgeos_c.so.1 | grep -E "Class|Machine"
    readelf -h /opt/lib/libgdal.so.35  | grep -E "Machine"
    python3 -c "
import shapely, pyproj
from shapely.geometry import Point
print(f\"shapely {shapely.__version__} geos={shapely.geos_version_string}\")
print(f\"pyproj  {pyproj.__version__} proj={pyproj.proj_version_str}\")
print(f\"buffer  area={Point(0, 0).buffer(1.0).area:.3f}\")
print(\"ARM64 LAYER VERIFIED\")
"'

Expected output:

code 
  Class:                             ELF64
  Machine:                           AArch64
  Machine:                           AArch64
shapely 2.0.6 geos=3.12.1-CAPI-1.18.1
pyproj  3.7.0 proj=9.4.0
buffer  area=3.137
ARM64 LAYER VERIFIED

Two things matter in that output. Machine: AArch64 on both the hand-built libgeos_c.so.1 and the hand-built libgdal.so.35 confirms the compiler emitted Graviton code rather than falling back to the host. And geos=3.12.1 reported through shapely.geos_version_string confirms which GEOS actually loaded — if the wheel’s vendored copy shadowed yours, this line reports the wheel’s version, not the one you just compiled, and the rest of the layer is dead weight.

Price and performance on a tiling workload

Graviton’s list price is roughly 20% below x86_64 per GB-second, and that discount is the floor, not the whole story. A tiling pass of the kind described in serverless NDVI tiling from Sentinel-2 is dominated by libtiff decode, resampling, and re-encode — integer and memory-bandwidth work where Graviton2 holds its own and Graviton3 pulls ahead. Measured across a 10,980 × 10,980 granule cut into 512 × 512 tiles, a 1,769 MB function averaged 4.20 s per tile invocation on x86_64 and 3.90 s on arm64, so the duration saving compounds with the rate saving.

Graviton versus x86_64 cost per million tile invocations at two memory tiersFour bars showing the cost in US dollars of one million tile invocations. At 1,769 MB, x86_64 costs 124 dollars and arm64 costs 92 dollars 20. At 3,008 MB, x86_64 costs 127 dollars 50 and arm64 costs 96 dollars 20. The arm64 bars are roughly a quarter shorter than their x86_64 counterparts at both memory tiers.Cost of one million 512 × 512 tile invocations, measured on a 10,980 × 10,980 granulex86_64 — 1,769 MB, 4.20 s per tile$124.00arm64 — 1,769 MB, 3.90 s per tile$92.20x86_64 — 3,008 MB, 2.60 s per tile$127.50arm64 — 3,008 MB, 2.45 s per tile$96.200USD per 1,000,000 tile invocationsRates of $0.0000166667 and $0.0000133334 per GB-second plus $0.20 per million requests; durations measured over a full granule, nota single warm tile.
The gap is about 26% and it survives the memory change, because the 20% rate discount compounds with a 4.20 s to 3.90 s duration saving on decode-and-resample work.

Raising memory does not change the ranking. At 3,008 MB both architectures finish faster in wall-clock terms but bill slightly more per invocation, because the extra vCPU share buys less than the extra GB costs on this workload — the same trade the memory and CPU allocation model for raster workloads makes explicit. The 26% gap between the architectures survives the memory change, which is what makes it worth a second build chain.

Gotchas and Edge Cases

  • Emulation is correct but slow, and the slowness lands in CI. Inside a --platform linux/arm64 container on an x86_64 host, every compiler invocation runs through qemu-aarch64. The output is genuine aarch64 machine code, but the GEOS-plus-PROJ-plus-GDAL build stretches from roughly four minutes native to twenty-five or more. Use emulation to get the recipe right, then move the recurring build to a native arm64 runner or a Graviton CodeBuild project and keep the emulated path as the fallback.
  • PROJ needs a runnable sqlite3 at build time. proj.db is generated by executing SQL against a real SQLite binary during the build, so a genuine cross-toolchain — aarch64-linux-gnu-gcc running on an x86_64 host — has to be handed a host sqlite3 through -DEXE_SQLITE3. Under container emulation the container’s own aarch64 sqlite3 runs fine, which is the main practical reason to prefer emulation over a cross-toolchain here.
  • --compatible-architectures is a label, not a check on the bytes. Publishing an x86_64 zip under an arm64 layer name succeeds, and the failure only appears when a function attaches it and cold-starts. Keep x86_64 and arm64 as separate layer names, put the architecture in the CI cache key, and never let the two zips share a staging path.
  • Not every geospatial workload wins, and one platform has no arm64 at all. Code paths that lean on hand-tuned AVX2 kernels — some scipy interpolation routines, a few resampling implementations — have no Graviton2 equivalent and can lose 10–15% of the duration saving. Benchmark the specific pass before migrating. Azure Functions on the Consumption plan is x86_64 only (10 min, 1,536 MB), so a portable pipeline needs the x86_64 layer maintained regardless.

Frequently Asked Questions

Does an x86_64 rasterio wheel run on a Graviton Lambda?

No. A manylinux2014_x86_64 or manylinux_2_28_x86_64 wheel contains ELF objects marked EM_X86_64, and the aarch64 dynamic linker refuses them during import with wrong ELF class or cannot open shared object file, before the handler runs. Architecture is not negotiable at load time the way a glibc minor version sometimes is — every wheel and every bundled .so in an arm64 layer must carry the aarch64 tag, which is why readelf -h belongs in the verification step and not just in the debugging session.

Do I need to compile GEOS at all if I only use shapely?

No. Shapely 2.x publishes cp311-cp311-manylinux_2_28_aarch64 wheels that vendor libgeos_c inside shapely.libs, so pip downloads a working GEOS for Graviton with nothing more than the platform tag changed. You compile GEOS yourself only when something outside shapely must link against it — a source-built GDAL, PDAL, or a standalone libgeos_c.so.1 shared across several packages in one layer so it is not duplicated the way NumPy gets duplicated across layers.

Is QEMU emulation good enough for a production arm64 GDAL layer?

The artifact is production-grade: the compiler itself runs as an aarch64 binary and emits real aarch64 code, and nothing about emulation changes the instructions written to the object files. What it changes is build time, by a factor of six to eight. That is tolerable for a monthly layer rebuild and painful for a pipeline that rebuilds on every merge, so treat emulation as the bring-up path and a native arm64 runner as the steady state.

Back to Native Library Compilation for Serverless