Serving a Static STAC Catalog from S3 and CloudFront

A static STAC catalogue is a tree of JSON on S3, but a browser will not walk it until CORS is configured in two places — the bucket rules and a CloudFront response-headers policy — and until Origin is part of the cache key. Set max-age=60 on Catalog and Collection nodes, max-age=300 on Items and max-age=31536000, immutable on the COG assets, then invalidate two or three tree paths per republish rather than one per Item. The 1,000 free invalidation paths a month are enough for a catalogue republished hourly, and nowhere near enough for one that invalidates per tile.

Context

STAC cataloging and metadata publishing argues that a static tree is the right default, and writing STAC Items from a Lambda tiling job produces the Items that fill it. This page is the serving layer between them: the configuration that turns a bucket full of correct JSON into something a STAC browser, a pystac-client script or a Leaflet map can actually consume.

The whole difficulty is that the consumer is a browser, and browsers enforce rules that curl does not. A catalogue that answers every request with a 200 and correct JSON can still be completely unusable from a web client, and the failure surfaces as an opaque JavaScript error with no useful detail. Almost every “my STAC catalogue does not work” report reduces to one of three things: a missing Access-Control-Allow-Origin, a cached response carrying the wrong one, or a relative link resolved across a redirect.

The path a single browse takes is worth having in mind before touching any configuration.

How a STAC browser resolves a catalogue node through CloudFrontSequence diagram with three participants: the STAC browser, the CloudFront edge and the S3 origin. The browser sends a CORS preflight OPTIONS for catalog.json; CloudFront forwards it to S3, which answers 204 with the allow-origin header; CloudFront returns the preflight to the browser. The browser then sends the real GET with its Origin header; CloudFront performs a cache lookup that misses on the first request after an invalidation, fetches the object from S3, and returns 200 with the CORS headers attached by the response-headers policy.One catalogue node, opened from a browser at a different originSTAC browserCloudFront edgeS3 originOPTIONS /stac/catalog.jsonOrigin + Access-Control-Request-Methodforward the preflightOPTIONS must be an allowed method204 + bucket CORS rulesAccess-Control-Allow-Origin204, cached 3,000 sAccessControlMaxAgeSecGET /stac/catalog.jsonOrigin travels on the real request toocache lookupkey must include Origin, or CORS is a coin flipMISS: fetch the objectfirst request after an invalidation200 + response-headers policythe edge attaches CORS on a hit, S3 neversees it
The cache lookup in the middle is where CORS goes wrong. If Origin is not part of the cache key, one visitor's allow-origin value is cached and served to everyone else.

Two round trips happen before any catalogue data moves. The preflight is not optional and it is not cheap on a cold cache: a browser issues one OPTIONS per distinct URL path pattern it has not seen recently, and a catalogue walk touching four nodes can therefore cost four preflights on top of four GETs. AccessControlMaxAgeSec is what collapses that — set it to 3,000 seconds and the browser stops asking for the rest of the session.

The critical detail is the cache lookup in the middle. If Origin is not part of the cache key, CloudFront can serve a response cached for one origin to a request from another, and the browser rejects a response that the origin server would have allowed.

Prerequisites

  • An S3 bucket holding the catalogue tree, with no public ACLs. Access is granted to CloudFront through Origin Access Control, not by making the bucket public.
  • A CloudFront distribution with that bucket as its origin, and a custom domain — because the Items written by the tiling job carry absolute self hrefs, and those hrefs must be the domain consumers actually use.
  • Absolute links in the published tree. pystac’s CatalogType.ABSOLUTE_PUBLISHED writes them; the relative alternative breaks the moment a request is redirected, since the browser then resolves children against the redirected location.
  • Correct content types on the objects. application/geo+json for Items, application/json for Catalog and Collection nodes, image/tiff; application=geotiff; profile=cloud-optimized for the COGs. S3 infers binary/octet-stream otherwise, and some clients branch on the type.
  • Permission to invalidate — cloudfront:CreateInvalidation on the distribution, scoped to the publishing role only, in line with IAM security boundaries for cloud GIS.

Implementation

One script configures the bucket, the response-headers policy and the cache behaviours, then republishes and invalidates.

python 
# serve_catalog.py — CORS, cache headers and a narrow invalidation
import json
import time

import boto3

s3 = boto3.client("s3")
cf = boto3.client("cloudfront")

BUCKET = "example-stac-catalog"
DISTRIBUTION_ID = "E1EXAMPLEDIST"

# --- 1. Bucket CORS. Governs responses served FROM THE ORIGIN only. ---------
# Expose the range headers: a browser reading a COG with a range request
# cannot see Content-Range or Accept-Ranges unless they are exposed.
s3.put_bucket_cors(
    Bucket=BUCKET,
    CORSConfiguration={
        "CORSRules": [{
            "AllowedOrigins": ["https://radiantearth.github.io", "https://maps.example.com"],
            "AllowedMethods": ["GET", "HEAD"],
            "AllowedHeaders": ["*"],
            "ExposeHeaders": ["Content-Length", "Content-Range",
                              "Accept-Ranges", "ETag", "Content-Type"],
            "MaxAgeSeconds": 3000,
        }]
    },
)

# --- 2. Cache-Control per object class, set at write time -------------------
CACHE = {
    "catalog.json":    "public, max-age=60",                  # tree nodes move
    "collection.json": "public, max-age=60",
    ".json":           "public, max-age=300",                 # Items
    ".tif":            "public, max-age=31536000, immutable",  # content-addressed
}


def cache_control_for(key: str) -> str:
    for suffix, value in CACHE.items():
        if key.endswith(suffix):
            return value
    return "public, max-age=300"


def upload(key: str, body: bytes, content_type: str) -> None:
    s3.put_object(
        Bucket=BUCKET, Key=key, Body=body,
        ContentType=content_type,
        CacheControl=cache_control_for(key),
    )


# --- 3. Invalidate the tree nodes ONLY. Items expire on their own. ---------
def republish_invalidate(changed_collections: list[str]) -> str:
    # One path per changed collection plus the root: two or three paths for a
    # normal run. Never one path per Item — 1,000 paths a month are free and
    # a per-tile invalidation exhausts that in a single scene.
    paths = ["/stac/catalog.json"]
    paths += [f"/stac/{cid}/collection.json" for cid in changed_collections]

    resp = cf.create_invalidation(
        DistributionId=DISTRIBUTION_ID,
        InvalidationBatch={
            "Paths": {"Quantity": len(paths), "Items": paths},
            "CallerReference": f"republish-{int(time.time())}",
        },
    )
    return resp["Invalidation"]["Id"]

The bucket CORS rules above are only half the configuration. They travel with responses S3 produces, and a response served from the CloudFront edge cache never reaches S3 — so on a cache hit those headers come from whatever was cached, or from nothing at all. The edge needs its own policy:

jsonc 
// response-headers-policy.json — attach to the distribution's default behaviour
{
  "Name": "stac-cors",
  "CorsConfig": {
    "AccessControlAllowOrigins": {"Quantity": 1, "Items": ["*"]},
    "AccessControlAllowMethods": {"Quantity": 2, "Items": ["GET", "HEAD"]},
    "AccessControlAllowHeaders": {"Quantity": 1, "Items": ["*"]},
    "AccessControlExposeHeaders": {
      "Quantity": 4,
      "Items": ["Content-Length", "Content-Range", "Accept-Ranges", "ETag"]
    },
    "AccessControlMaxAgeSec": 3000,
    "AccessControlAllowCredentials": false,
    "OriginOverride": true
  }
}

OriginOverride: true in that policy is deliberate. It tells CloudFront to replace whatever CORS headers S3 attached with the policy’s own, which means the edge and the origin can never disagree — and disagreement between the two is the single hardest CORS bug to diagnose, because it presents as a failure that appears and disappears depending on which edge location a user happens to hit. One authority for the headers, applied on every response, is worth more than a marginally tighter allow-list.

And the cache policy must include Origin in the cache key — HeadersConfig: {"HeaderBehavior": "whitelist", "Headers": ["Origin", "Access-Control-Request-Method", "Access-Control-Request-Headers"]}. Without it, one visitor’s allowed origin gets cached and served to everyone.

What each class of object wants

The three object classes in a catalogue have genuinely different lifecycles, and treating them uniformly means either a stale tree or an uncacheable archive.

Cache and CORS settings per class of catalogue objectComparison grid of the three classes of object in a static STAC catalogue: the Catalog and Collection tree nodes, the Item GeoJSON documents, and the Cloud-Optimized GeoTIFF assets. Rows compare Cache-Control, how often the object changes on a republish, whether it needs an invalidation, which CORS response headers must be exposed, edge compression, and typical object size.Three object classes, three lifecyclesCatalog / CollectionItem GeoJSONCOG assetsCache-Controlpublic, max-age=60public, max-age=300max-age=31536000immutableChanges on a republishevery runchild links moveretried tiles onlyneverkey is derived from the tileNeeds invalidationyes2-3 paths per runnothe 300 s TTL expires itnoCORS expose headersETag, Content-TypeETag, Content-Type+ Content-RangeAccept-Ranges, Content-LengthEdge compressiongzip / brgzip / broffalready DEFLATE; breaksrangesTypical object size2 KB - 420 KB3.6 KB~40 MBThe Content-Range row is what makes browser-side COG reads possible at all: a range response the client cannot inspect is a range responseit cannot use.
Only the first column needs invalidating, and only the third can be cached for a year. Treating all three uniformly gives you either a stale tree or an archive that re-downloads 40 MB assets on every browse.

The immutable on the assets is the load-bearing one. A COG written under a deterministic key — the same scheme the Item writer uses for its Items — never changes content without changing key, so it can be cached for a year. That is what makes browser-side range reads over a COG practical, and it is the serving-side mirror of the read tuning in tuning HTTP range requests for COG reads on S3.

What a republish actually costs in wall clock

Where the wall clock of one catalogue republish goesA ninety-second republish split into five segments: writing 2,700 Item JSON objects takes 22 seconds, rebuilding the twelve month Catalog nodes takes 3 seconds, syncing the tree to the bucket takes 9 seconds, the CreateInvalidation call is accepted in about 1 second, and CloudFront edge propagation accounts for the remaining 55 seconds.90 seconds from finished tiles to a fresh catalogue at the edgeWrite 2,700 Item JSONsSync thetree to S3Edge propagation22 s9 s55 sRebuild 12 month nodes — 3 sCreateInvalidation accepted — 1 sThe API returns after the fourth segment, not the fifth. A deploy that asserts freshness immediately after the call reads the old tree.
Edge propagation is the majority of the budget and no configuration removes it — which is exactly why the Item TTL is short and the invalidation list is two or three paths rather than one per tile.

Edge propagation dominates, and no configuration removes it — which is precisely why the Item TTL is short and the invalidation list is short. Invalidating per Item would multiply the slowest segment by the tile count while adding nothing a 300-second TTL does not already deliver.

Verification

The check that matters is the one a browser performs: a preflight, then a cross-origin GET, then a range request against an asset.

bash 
# verify_cors.sh — exactly what the browser will do, in the same order
BASE=https://catalog.example.com/stac
ORIGIN=https://maps.example.com

echo "--- preflight ---"
curl -s -o /dev/null -D - -X OPTIONS "$BASE/catalog.json" \
  -H "Origin: $ORIGIN" \
  -H "Access-Control-Request-Method: GET" \
  | grep -i -E 'HTTP/|access-control-allow-(origin|methods)'

echo "--- cross-origin GET ---"
curl -s -o /dev/null -D - "$BASE/catalog.json" -H "Origin: $ORIGIN" \
  | grep -i -E 'HTTP/|access-control-allow-origin|cache-control|x-cache'

echo "--- range read on an asset ---"
curl -s -o /dev/null -D - -r 0-16383 \
  "$BASE/ndvi-10m-v1/2026/07/14/S2B_33UUP_20260714_v1_r04096_c02048.tif" \
  -H "Origin: $ORIGIN" \
  | grep -i -E 'HTTP/|content-range|accept-ranges|cache-control'

Expected output:

code 
--- preflight ---
HTTP/2 204
access-control-allow-origin: *
access-control-allow-methods: GET, HEAD
--- cross-origin GET ---
HTTP/2 200
access-control-allow-origin: *
cache-control: public, max-age=60
x-cache: Hit from cloudfront
--- range read on an asset ---
HTTP/2 206
content-range: bytes 0-16383/41883648
accept-ranges: bytes
cache-control: public, max-age=31536000, immutable

Three things to read out of that. The preflight returns 204, not 403 — a 403 means OPTIONS is missing from the distribution’s allowed methods. The asset returns 206, not 200 — a 200 means the range header was dropped somewhere and the client just downloaded 40 MB to read a header. And x-cache distinguishes a hit from a miss, which is how you confirm an invalidation actually landed. To keep this check honest, run it once with an Origin you have not allowed and confirm the allow-origin header is absent; a CORS test that passes against every origin is testing nothing.

Gotchas and Edge Cases

  • The Origin header must be in the cache key, or CORS is a coin flip. CloudFront caches the whole response including Access-Control-Allow-Origin. If the header is not part of the key, the first requester’s value is served to everyone, and the failure is intermittent and origin-dependent — the worst possible signature to debug. Whitelist Origin, Access-Control-Request-Method and Access-Control-Request-Headers in the cache policy.
  • index.html-style default root objects break catalogue paths. A distribution configured with a default root object rewrites /stac/2026/07/ in ways that produce a redirect, and a redirect is exactly what breaks relative link resolution in a STAC browser. Publish with absolute self hrefs and leave the default root object unset for the catalogue behaviour.
  • Do not compress the COGs at the edge. A COG’s internal blocks are already DEFLATE- or LZW-compressed; asking CloudFront to gzip them burns CPU for a percent or two and, worse, a transformed body invalidates byte ranges. Restrict compression to the JSON path patterns, where it genuinely halves the transfer of a 420 KB day node.
  • Origin Access Control, not a public bucket. It is tempting to make the catalogue bucket public and skip the signing setup, but that leaves a second, uncached entry point to every object — one that bypasses your cache headers, your compression rules and your CORS policy entirely. Lock the bucket to the distribution’s OAC principal and let the only reachable path be the one you configured. Consumers that find the S3 URL and use it directly will also generate origin request charges you never see in the CDN metrics.
  • Invalidation is asynchronous and the API returns immediately. create_invalidation returns as soon as the batch is accepted, typically 55 seconds before the edges have actually dropped the object. A deploy pipeline that asserts freshness right after the call will see the old content; poll get_invalidation until the status reads Completed, or simply do not assert on the tree node until the TTL alone would have expired it.

Frequently Asked Questions

Why does a STAC browser fail when curl succeeds?

Because curl does not enforce CORS and a browser does. The request succeeds at the network level and arrives with a 200, but without an Access-Control-Allow-Origin header the browser refuses to hand the body to JavaScript and reports an opaque failure. CORS has to be configured in two places when CloudFront fronts S3: the bucket rules govern responses served from the origin, and a CloudFront response-headers policy governs responses served from the edge cache — where most of your traffic will be answered.

How many CloudFront invalidation paths does republishing a catalogue need?

Two or three, not one per Item. Invalidate the root catalog.json and the collection.json paths that actually changed, and let the Items expire on their own short TTL. AWS gives 1,000 invalidation paths per month free and charges per path afterwards, so a pipeline that invalidates every republished Item turns a free operation into a per-tile cost. Wildcards count as one path each, which makes a root path plus one wildcard per changed collection the practical shape.

Should COG assets be cached differently from the Item JSON?

Yes, and much more aggressively. An Item’s JSON changes whenever a tile is reprocessed, so it wants a short TTL of a few minutes. A COG written under a deterministic key never changes content without changing key, so it can carry max-age=31536000, immutable. That asymmetry matters because the assets are where the bytes are — caching a 40 MB COG for a year at the edge is what makes range reads from a browser viable at all.

Back to STAC Cataloging and Metadata Publishing