Choosing Block Size and Overview Levels for COGs

Write full-resolution satellite bands with gdal_translate -of COG -co BLOCKSIZE=512, then build five power-of-two overview levels with gdaladdo -r average scene.tif 2 4 8 16 32, and set --config GDAL_TIFF_OVR_BLOCKSIZE 512 because GDAL otherwise writes overview blocks at 128 pixels. On a Sentinel-2 L2A 10 m band of 10,980 × 10,980 pixels, that combination yields 484 full-resolution tiles, a tile index of roughly 10 KB, and a pyramid that terminates at 344 pixels — one block. The same band written at 256 produces 1,849 tiles and a tile index that, once the pyramid is appended, no longer fits in a single 32 KB opening request.

Context

Block size and overview levels are decided once, at write time, and every read for the life of the object pays for them. That makes this the highest-leverage decision in chunked I/O for large satellite imagery: the alignment rules and the GDAL tuning knobs downstream are both attempts to read a layout you already committed to. A worker cannot fetch part of an internal tile — the compression codec operates per tile — so the block size sets the granularity of every transfer, and the overview ladder sets whether a low-zoom consumer reads 64 tiles or one.

Three quantities move together when you change the block size, and only one of them improves in each direction. Tile count scales inversely with the square of the block edge. The tile index — the TileOffsets and TileByteCounts arrays that make a COG seekable — scales with the tile count, and it must be fetched before any pixel can be located. Per-tile transfer volume scales with the square of the block edge in the other direction, so a 512 block moves 512 KB of uint16 data uncompressed where a 256 block moves 128 KB.

The tile-index row is the one that decides it in practice. Tuning HTTP range requests for COG reads on S3 recommends GDAL_INGESTED_BYTES_AT_OPEN=32768 so the header and the full tile index arrive in the opening GET. At 512 blocks with a complete pyramid the index is around 10 KB and comfortably inside that budget. At 256 blocks the full-resolution index alone is near 30 KB and the pyramid pushes it past 32 KB, so every open costs a second round trip before the first pixel is located — and you would not notice, because nothing errors.

Block size is not free choice either way, though. A 512 block is the wrong answer when consumers routinely want regions smaller than 512 pixels: a point-sampling service that reads a 64 × 64 patch per request transfers 512 KB to return 8 KB. That is the case where 256 wins, and it wins for the same reason it loses elsewhere.

Prerequisites

• GDAL 3.6 or later with the COG driver. BLOCKSIZE is a COG-driver creation option; the GTiff driver spells the same thing BLOCKXSIZE and BLOCKYSIZE, and mixing them up silently gets you the driver default.
• Dependencies: rasterio>=1.3.9 and numpy>=1.26 for the verification step; rio-cogeo if you want an independent structural validator.
• Source rasters that are not already tiled. Sentinel-2 JP2 assets and legacy strip-layout GeoTIFFs both need a full rewrite; there is no in-place retiling.
• Disk or memory for the rewrite. A single 10 m band at uint16 is 10,980 × 10,980 × 2 bytes ≈ 230 MB, and gdal_translate needs room for the source, the target and the pyramid. On AWS Lambda that means budgeting against the 10,240 MB /tmp ceiling — 512 MB unless you provision more — as managing /tmp storage limits for GeoTIFF extraction covers. Conversion is a build-time job, not a per-request one.
• A decided read pattern. Write down the two consumers that matter — typically one analytic worker reading full-resolution windows and one tile server reading low zooms — before choosing anything. Every number below is downstream of that.

Sizing the Overview Ladder

The stopping rule is mechanical: add power-of-two levels until the smallest overview fits inside a single internal block. For a 10,980-pixel band at 512, the ladder is 5,490, 2,745, 1,373, 687, 344 — factors 2, 4, 8, 16 and 32, ending at 344 pixels, which is one block. A sixth level would produce a 172-pixel image that no renderer requests and that still occupies a full block. At 256-pixel blocks the same rule demands a sixth level, factor 64, because 344 does not fit in a 256 block.

The storage cost of a complete pyramid is fixed by geometry, not by content. Each level holds a quarter of the pixels of the one above, and the series 1/4 + 1/16 + 1/64 + … converges on one third. A pyramid therefore adds about 33 % to the object, and no choice of level count changes that materially — the first level alone is 25 of those 33 points.

Resampling is the choice that actually needs thought, because it is data-dependent and irreversible. Use -r average for continuous reflectance: it is what makes a zoomed-out NDVI composite look like the scene rather than like a sparse sample of it. Use -r nearest or -r mode for the Scene Classification Layer and any other categorical band — averaging class code 4 (vegetation) against class code 6 (water) produces code 5 (not-vegetated), a value that describes neither input. GDAL will do it without complaint, and the resulting overview is a plausible-looking lie. -r gauss is worth knowing about for imagery destined for visual inspection, where it suppresses the aliasing that average leaves on high-contrast edges.

Implementation

Two commands, both explicit about every option that has a default worth overriding:

#!/usr/bin/env bash
set -euo pipefail

SRC="T32TNM_20260712_B04_10m.jp2"      # Sentinel-2 L2A red band, 10,980 x 10,980
DST="T32TNM_20260712_B04_10m.tif"

# BLOCKSIZE is a COG-driver option and takes a single value (square blocks only).
# OVERVIEW_COUNT=5 stops the ladder at 344 px — one block — rather than letting
# the driver pick. PREDICTOR=2 is horizontal differencing for integer data;
# use 3 only for float32/float64 or the output is silently wrong on some builds.
gdal_translate "$SRC" "$DST" \
  -of COG \
  -co BLOCKSIZE=512 \
  -co COMPRESS=DEFLATE \
  -co PREDICTOR=2 \
  -co OVERVIEWS=AUTO \
  -co OVERVIEW_RESAMPLING=AVERAGE \
  -co OVERVIEW_COUNT=5 \
  -co BIGTIFF=IF_SAFER \
  -co NUM_THREADS=ALL_CPUS

# For a GTiff written without a pyramid, add one after the fact. The two --config
# settings are the ones people forget: without GDAL_TIFF_OVR_BLOCKSIZE the
# overview blocks are written at 128 px, which multiplies the overview tile count
# by sixteen and pushes the tile index past a single 32 KB header read. Without
# COMPRESS_OVERVIEW the pyramid is stored uncompressed and the 33% overhead
# becomes closer to 80%.
gdaladdo -r average \
  --config GDAL_TIFF_OVR_BLOCKSIZE 512 \
  --config COMPRESS_OVERVIEW DEFLATE \
  --config PREDICTOR_OVERVIEW 2 \
  "$DST" 2 4 8 16 32

# Categorical bands take a different resampler. Averaging class codes invents
# classes: SCL 4 (vegetation) averaged with SCL 6 (water) yields SCL 5.
gdaladdo -r mode \
  --config GDAL_TIFF_OVR_BLOCKSIZE 512 \
  --config COMPRESS_OVERVIEW DEFLATE \
  "T32TNM_20260712_SCL_20m.tif" 2 4 8 16

gdaladdo on a plain GeoTIFF writes the pyramid internally by default; on a read-only source it writes an external .ovr sidecar instead, which is the layout you do not want on object storage — it costs a second object, a second set of range requests, and it is exactly what CPL_VSIL_CURL_ALLOWED_EXTENSIONS=.tif suppresses probing for. If you already hold overviews in a sidecar, fold them in with gdal_translate -co COPY_SRC_OVERVIEWS=YES.

The payoff is on the read side, and it is large enough to be worth quantifying before anyone argues about the 33 % storage overhead.

A tile server rendering a 256-pixel web tile that covers 4,096 source pixels has to touch an 8 × 8 patch of 512-blocked full-resolution tiles when no pyramid exists: 64 range requests and 16.7 million decompressed pixels to produce 65,536. With a factor-8 overview present, the same region is a 512 × 512 read at the overview level — one to four blocks. The analytic path described in optimizing chunked I/O for multi-band Sentinel-2 processing never reads an overview at all, which is why the pyramid is invisible in its benchmarks and decisive in the tile server’s.

Verification

Check the layout, the block shape and the decimation list in one pass. Never trust the creation options you passed — trust what the file reports:

import rasterio

with rasterio.open("T32TNM_20260712_B04_10m.tif") as src:
    print("layout    :", src.tags(ns="IMAGE_STRUCTURE").get("LAYOUT"))
    print("size      :", src.width, "x", src.height)
    print("blocks    :", src.block_shapes)          # one tuple per band
    print("overviews :", src.overviews(1))          # decimation factors
    print("ovr sizes :", [(src.width // f, src.height // f) for f in src.overviews(1)])

Expected output for a correctly written 10 m band — LAYOUT must read COG, the block shape must be square 512, and the last overview must be smaller than one block:

layout    : COG
size      : 10980 x 10980
blocks    : [(512, 512)]
overviews : [2, 4, 8, 16, 32]
ovr sizes : [(5490, 5490), (2745, 2745), (1372, 1372), (686, 686), (343, 343)]

An empty overviews list means gdaladdo wrote a sidecar rather than internal levels, or that OVERVIEWS=IGNORE_EXISTING discarded a pyramid you thought you had built. A block_shapes entry of (512, 128) or (10980, 1) means the file is strip-organised, not tiled, and no amount of read tuning will rescue it.

Gotchas and Edge Cases

• GDAL_TIFF_OVR_BLOCKSIZE defaults to 128, not to the full-resolution block size. Every gdaladdo invocation that omits it writes overview tiles at 128 pixels. The first overview of a 10,980-pixel band then holds 43 × 43 = 1,849 tiles instead of 121, and the extra offsets go straight into the tile index that you were trying to keep under 32 KB.
• OVERVIEWS=IGNORE_EXISTING throws away the pyramid you already have. It is the right setting when rebuilding from a source whose overviews are stale or use the wrong resampler, and precisely the wrong one when you have just spent an hour building a mode-resampled pyramid for a classification band. AUTO reuses what exists.
• Block size interacts with the concurrency quota, not just with bandwidth. Halving the block edge to 256 turns a 484-window dispatch into a 1,849-window dispatch against the same 1,000 regional concurrency slots. The read of each window gets cheaper and the fan-out gets four times wider; whether that is an improvement depends entirely on which of the two you were short of.
• A dispatcher must read block_shapes, never assume 512. Archive vintage varies, and a scene rewritten by an upstream provider at a different block size will silently hand every worker a misaligned window. Reading the block shape from the open dataset costs one header request you are already paying for.
• BIGTIFF=IF_SAFER is not the same as BIGTIFF=YES. Classic TIFF caps at 4 GB, and a 12-band uncompressed stack passes that easily. IF_SAFER decides from the estimated output size, which is correct for compressed output and can be wrong at the boundary; force YES for any multi-band mosaic you intend to keep.

Frequently Asked Questions

Should a COG use 256 or 512 pixel internal blocks?

Use 512 for full-resolution satellite bands read by analytic workers, and 256 only when consumers routinely read regions smaller than 512 pixels. On a 10,980 × 10,980 Sentinel-2 band, 512-pixel blocks give 484 tiles and 256-pixel blocks give 1,849 — roughly four times the tile index and four times the range requests to cover the same area. The compensation is real but narrow: a 256 block transfers 128 KB uncompressed against 512 KB, so a service that only ever wants a small patch wastes less on every read.

How many overview levels should a COG have?

Add power-of-two levels until the smallest overview fits inside one internal block. A 10,980-pixel band with 512 blocks needs factors 2, 4, 8, 16 and 32, ending at 343 pixels. Stopping earlier leaves low-zoom consumers reading full-resolution tiles; going further adds levels no renderer will request. The pyramid costs about a third of the full-resolution band regardless, because a power-of-two series converges on one third.

Does gdaladdo change how many HTTP range requests a read costs?

Substantially, for anything reading below native resolution. Rendering a 256-pixel web tile covering 4,096 source pixels from a 512-blocked COG with no pyramid touches an 8 × 8 patch of full-resolution tiles — 64 range requests and 16.7 million decompressed pixels for 65,536 output pixels. With a factor-8 overview present, the same region is a single 512 × 512 read at the overview level and the count falls to roughly four.

Does block size affect compression ratio?

Slightly, and in favour of larger blocks. DEFLATE resets its dictionary at every tile boundary, so 1,849 small tiles compress marginally worse than 484 large ones on the same pixels — typically a few per cent on Sentinel-2 reflectance. It is not a reason to choose 512 on its own, but it does mean the storage argument and the request-count argument point the same way.

Related

• Chunked I/O for Large Satellite Imagery — the windowed read pattern whose alignment depends on the block size chosen here
• Tuning HTTP Range Requests for COG Reads on S3 — the read-side settings, including the 32 KB header budget the tile index has to fit inside
• Optimizing Chunked I/O for Multi-Band Sentinel-2 Processing — the analytic consumer that reads full resolution and never touches the pyramid
• Managing /tmp Storage Limits for GeoTIFF Extraction — why the conversion itself is a build step rather than a per-request one
• Computing NDVI per Tile with Rasterio and NumPy — a per-window workload that reads exactly one block per request when the block size is right

Back to Chunked I/O for Large Satellite Imagery