Reducing GCP Cloud Functions 2nd Gen Cold Starts with Minimum Instances

Set --min-instances=2 (or higher) on a GCP Cloud Functions 2nd gen deployment to hold that many Cloud Run containers past their GDAL initialization, eliminating the 3-6 second import rasterio cold start on the warmed instances. Because 2nd gen functions run on Cloud Run under the hood, this is the min_instance_count field on the service, and it bills per idle instance-second — a 4Gi/2 vCPU function held at two warm instances costs roughly a few dollars a day whether or not any request arrives.

Minimum instances warm pool versus cold scale-out An incoming request splits two ways. Requests within baseline demand hit warm min-instances where GDAL is already registered, returning fast. Requests above the warm pool trigger a cold Cloud Run container that must pull the image and register GDAL, adding 3 to 6 seconds. HTTP requests tile endpoint Warm pool (min-instances) GDAL already registered proj.db resident fast response < 100 ms overhead Cold scale-out image pull + container boot GDAL driver registration slow response +3 to 6 s
Traffic inside the warm pool skips GDAL registration; traffic above it pays the full cold-start cost.

Context

This page drills into the GCP row of the cold start comparison across AWS, GCP, and Azure. The reason minimum instances matter specifically for a GDAL function is that the cold start is not dominated by Cloud Run’s own boot — it is dominated by the Python GDAL initialization sequence: resolving libgdal, libproj, and libgeos, then running the driver registration sweep. A generic Node function scales from zero in under a second; a rasterio function needs three to six seconds before it can answer a single tile request, which is unacceptable behind an interactive map viewer.

Minimum instances solve this by holding a baseline number of Cloud Run containers permanently warm, so requests up to that concurrency level are served by an instance that has already paid the registration cost. It is the GCP analogue of AWS provisioned concurrency, but it lives on the Cloud Run service revision rather than on a Lambda version, and it is priced per idle instance-second. Keep the container image lean — the multi-stage Dockerfile for GDAL on Cloud Run approach keeps image-pull time low so even the cold scale-out beyond the warm pool stays bounded.

There is a second, GCP-specific subtlety that trips up teams coming from Lambda: Cloud Run’s per-instance concurrency. A single warm 2nd gen container can handle multiple concurrent requests (the default is up to 80), unlike a Lambda environment that serves exactly one invocation at a time. That means a modest --min-instances can cover more baseline traffic than the equivalent provisioned-concurrency number on AWS — but only if your GDAL handler is actually concurrency-safe. GDAL’s global driver registry and the PROJ context are not fully thread-safe under every configuration, so if you raise per-instance concurrency you must confirm each request uses its own dataset handles and that no two threads share a mutable gdal.Dataset. When in doubt, set --concurrency=1 and scale warmth with --min-instances instead; you trade some density for correctness. This interplay between minimum instances and per-instance concurrency is the main reason the GCP tuning story differs from the AWS one described in the cold-start comparison.

Prerequisites

  • Runtime: Cloud Functions 2nd gen, python311, deployed to a region (e.g. us-central1)
  • Container image: a lean GDAL/rasterio image in Artifact Registry with GDAL data under /usr/share/gdal and PROJ data under /usr/share/proj
  • IAM: the deploying principal needs roles/cloudfunctions.developer and roles/run.admin (min-instances is enforced on the Cloud Run service)
  • CPU allocation: enable CPU-always-allocated so idle warm instances keep proj.db memory-resident, not just the process alive
  • Environment variables (set at deploy, read at import):
    code 
    GEO_DATA_BASE=/usr/share
GDAL_DATA=/usr/share/gdal
PROJ_LIB=/usr/share/proj
  • Dependency versions pinned: rasterio==1.3.10, pyproj==3.6.1, shapely==2.0.5 in the image

Implementation

The key is that warm capacity is only useful if GDAL is initialized at module load, not lazily in the handler. Do the expensive registration once at import so every warm instance is genuinely ready.

python 
# main.py — Cloud Functions 2nd gen entry point (functions-framework)
import os
import functions_framework

# --- Module scope: runs once per container start, covered by min-instances ---
os.environ.setdefault("GDAL_DATA", "/usr/share/gdal")
os.environ.setdefault("PROJ_LIB", "/usr/share/proj")

from osgeo import gdal                 # shared-library resolution at container boot
gdal.UseExceptions()
gdal.AllRegister()                     # driver registration paid once, kept warm
_PROJ_WARM = __import__("pyproj").CRS.from_epsg(4326)  # force proj.db into memory


@functions_framework.http
def tile(request):
    """Serve a single reprojected tile. Warm instances skip all init above."""
    payload = request.get_json(silent=True) or {}
    src = payload.get("cog_url")
    if not src:
        return ({"error": "cog_url required"}, 400)

    # /vsicurl streams the COG over HTTP range requests — no full download
    ds = gdal.Open(f"/vsicurl/{src}")
    if ds is None:
        return ({"error": "unreadable source"}, 422)
    return {
        "bands": ds.RasterCount,
        "size": [ds.RasterXSize, ds.RasterYSize],
        "gdal": gdal.VersionInfo("RELEASE_NAME"),
    }

Deploy with a non-zero minimum instance count and explicit CPU/memory so the warm container is fully sized:

bash 
gcloud functions deploy geo-tiler \
  --gen2 \
  --runtime=python311 \
  --region=us-central1 \
  --source=. \
  --entry-point=tile \
  --trigger-http \
  --memory=4Gi \
  --cpu=2 \
  --min-instances=2 \
  --max-instances=50 \
  --set-env-vars=GEO_DATA_BASE=/usr/share,GDAL_DATA=/usr/share/gdal,PROJ_LIB=/usr/share/proj

Pin the setting in Terraform so a redeploy cannot silently drop the warm pool back to zero. On 2nd gen, min_instance_count lives in service_config:

hcl 
resource "google_cloudfunctions2_function" "geo_tiler" {
  name     = "geo-tiler"
  location = "us-central1"

  build_config {
    runtime     = "python311"
    entry_point = "tile"
  }

  service_config {
    available_memory   = "4Gi"
    available_cpu      = "2"
    min_instance_count = 2   # warm pool — bills per idle instance-second
    max_instance_count = 50
    # Keep vCPU allocated to idle instances so proj.db stays resident
    environment_variables = {
      GDAL_DATA = "/usr/share/gdal"
      PROJ_LIB  = "/usr/share/proj"
    }
  }
}

Verification

Confirm that requests served by the warm pool skip GDAL registration. Fire a burst below the min-instance count and inspect the Cloud Run startup latency, which should be effectively zero on warmed instances:

bash 
# Warm-pool request: startup latency ~0 because the container is pre-initialized
gcloud logging read \
  'resource.type="cloud_run_revision" AND
   resource.labels.service_name="geo-tiler" AND
   httpRequest.requestUrl:"tile"' \
  --limit=5 --format="value(jsonPayload.startupLatency, httpRequest.latency)"

Expected output — the warmed requests show negligible startup latency, while any scale-out beyond min_instance_count shows the multi-second GDAL init:

code 
        0.412s      # warm instance, no GDAL registration on the request path
        0.388s      # warm instance
4.9s    0.451s      # cold scale-out: 4.9s startup includes image pull + GDAL registration

Gotchas and Edge Cases

  • Lazy imports defeat the warm pool. If from osgeo import gdal and AllRegister() run inside the handler instead of at module scope, each request re-pays part of the cost and min-instances only saves the container boot. Always initialize GDAL at import time.

  • CPU-throttled idle instances let proj.db fall cold. Without CPU-always-allocated, GCP throttles vCPU to near zero between requests; the process survives but cached PROJ state can be evicted, so the first transform after idle is slow. Enable always-allocated CPU for coordinate-transform-heavy endpoints.

  • min-instances is a floor, not a reservation against bursts. A traffic spike above the warm count still triggers cold scale-out. Pair a realistic min_instance_count with a lean image so those overflow starts stay short; do not set the minimum to your peak just to avoid every cold start.

  • Redeploys create a new revision. Traffic shifts to the new revision, which must warm its own min-instances; expect a brief window of cold starts during a rollout unless you stage traffic gradually.

  • The idle bill scales with memory, so oversizing warm instances is expensive twice over. A --memory=8Gi warm pool held at three instances reserves 24 GiB around the clock. Because per-instance-second pricing is proportional to allocated memory, doubling the tier to give GDAL headroom also doubles the standing cost of every idle instance. Size the memory to what a single tile actually needs — profile with the memory and CPU allocation model — rather than defaulting high “to be safe,” because on a warm pool that safety margin is billed continuously, not per request.

Frequently Asked Questions

Does --min-instances keep GDAL fully initialized or just the container running?

It keeps the container process alive, but GDAL is only initialized if your module-level code runs the import and driver registration at startup rather than lazily inside the handler. Move from osgeo import gdal and gdal.AllRegister() to module scope so the warm instance has already paid the registration cost.

How is min-instances different from AWS provisioned concurrency?

Both hold warm capacity, but AWS provisioned concurrency is billed per GB-second of reserved concurrency and attaches to a specific function version, while GCP min-instances is a Cloud Run setting billed per idle instance-second and attaches to the service revision. GCP also keeps CPU allocated to idle min-instances only if you enable CPU-always-allocated, which matters for keeping proj.db warm.

Why do idle minimum instances still cost money with no traffic?

A warm instance holds reserved memory and, when CPU-always-allocated is on, reserved vCPU. GCP bills that reserved capacity per second regardless of request volume, which is the price of eliminating the 3-6 second GDAL cold start.

Back to Cold Start Comparison: AWS vs GCP vs Azure for Python GDAL