Migrating PostGIS SQL to BigQuery GIS functions

This page ports a PostGIS dbt project’s spatial SQL to BigQuery GIS: which calls translate one-to-one, which change units or semantics while keeping their name, and which have no equivalent and need the model restructured instead.

When to use this approach

  • You are moving an existing dbt project’s execution engine to BigQuery and want a checklist rather than a discovery process.
  • You maintain one project that must run on both and need to know which calls to put behind a dispatched macro — the mechanism is in cross-engine UDF portability.
  • A ported model runs but returns different numbers. The unit and semantic tables below are where that difference usually lives.

Prerequisites

  • A working BigQuery target — profile setup is in running dbt spatial models on BigQuery GIS.
  • An inventory of the spatial functions your project actually uses, produced in step 1 rather than from memory.
  • Source data already in WGS84, since BigQuery cannot reproject.
  • A parity dataset — the same rows on both engines — for the comparison in step 5.

Step-by-step instructions

1. Inventory every spatial call in the project

bash
grep -rhoiE "st_[a-z_]+" models/ macros/ | tr 'A-Z' 'a-z' | sort | uniq -c | sort -rn
text
     41 st_intersects
     28 st_transform
     22 st_isvalid
     19 st_dwithin
     14 st_area
     11 st_makevalid
      9 st_subdivide
      7 st_simplify
      5 st_setsrid

That list is the migration plan. Sort it by count and work top down; the long tail is usually a handful of calls in one model.

2. Translate the direct equivalents

Most of the list ports without thought. The names match, the arguments match, and the results agree.

PostGIS BigQuery GIS Note
ST_Intersects(a, b) ST_INTERSECTS(a, b) Same semantics on geography
ST_Contains(a, b) ST_CONTAINS(a, b) Same
ST_Union(g) ST_UNION_AGG(g) Aggregate form is renamed
ST_Centroid(g) ST_CENTROID(g) Geodesic centroid, so it can differ slightly
ST_Simplify(g, tol) ST_SIMPLIFY(g, tol_metres) Tolerance is metres, not degrees
ST_AsGeoJSON(g) ST_ASGEOJSON(g) Same
ST_Buffer(g, d) ST_BUFFER(g, d_metres) Metres; segments argument differs
ST_X(p) / ST_Y(p) ST_X(p) / ST_Y(p) Longitude and latitude respectively

3. Rewrite the calls that changed units

These compile, run, and return wrong numbers if ported literally. They are the reason a “successful” migration can ship a broken mart.

sql
-- PostGIS, geometry in EPSG:4326 — tolerance and radius are DEGREES
select
    st_simplify(geom, 0.0001)                as simplified,
    st_dwithin(a.geom, b.geom, 0.0045)       as nearby,
    st_area(geom)                            as area_sq_degrees
from ...

-- BigQuery GIS — tolerance and radius are METRES, area is square metres
select
    st_simplify(geog, 10)                    as simplified,
    st_dwithin(a.geog, b.geog, 500)          as nearby,
    st_area(geog)                            as area_sqm
from ...

Verify each converted literal against a known quantity rather than a conversion factor from memory:

sql
select st_distance(
  st_geogpoint(13.4050, 52.5200),
  st_geogpoint(13.4050, 52.5290)
) as metres;
-- Expect ≈ 1001 m — one hundredth of a degree of latitude
The three outcomes of porting a spatial function, and which needs a test Three bands. The top band, direct equivalents, ports by renaming and is verified by the build succeeding. The middle band, unit changes, compiles and runs while returning wrong numbers, and is caught only by a value test. The bottom band, no equivalent, fails at compile time and forces the model to be restructured. The middle band is highlighted as the dangerous one. direct equivalent ST_INTERSECTS, ST_CONTAINS, ST_ASGEOJSON the build proves it rename and move on same name, different unit ST_SIMPLIFY, ST_DWITHIN, ST_AREA, ST_BUFFER nothing fails — values change only a value test catches it no equivalent ST_TRANSFORM, ST_SETSRID, ST_SUBDIVIDE, ST_MAKEVALID compile error restructure the model Migrations fail in the middle band, because that is the only one the compiler cannot see.

4. Restructure around the functions that do not exist

Four PostGIS staples have no BigQuery counterpart, and each needs a different answer.

sql
-- ST_Transform / ST_SetSRID: no equivalent — there is no SRID.
--   → Reproject during ingestion, load WGS84, and delete the call.

-- ST_MakeValid: no equivalent — repair happens at construction.
--   → SAFE.ST_GEOGFROMTEXT(wkt, make_valid => true)

-- ST_IsValid: no equivalent — invalid geography cannot exist in a column.
--   → Test for NULL after the safe constructor instead:
select count(*) as unparsed from {{ ref('stg_zones') }} where zone_geog is null;

-- ST_Subdivide: no equivalent, and no need — there is no index to tighten.
--   → Replace with an S2 or H3 grid key join, per the grid macros topic.

The ST_Subdivide line is the one that surprises people. Subdivision exists to tighten bounding boxes for an index; BigQuery has no user-managed index, so the equivalent optimisation is a grid-key prefilter, described in discrete global grid macros.

Verify that no unported calls remain:

bash
dbt compile --target bq_dev 2>&1 | grep -i "function not found" || echo "no missing functions"

5. Run the parity comparison before switching consumers

sql
-- models/intermediate/int_port_parity.sql
select
    z.zone_id,
    z.area_sqm                                  as bq_area,
    p.area_sqm                                  as pg_area,
    abs(z.area_sqm - p.area_sqm) / nullif(p.area_sqm, 0) as area_diff,
    z.ping_count                                as bq_pings,
    p.ping_count                                as pg_pings,
    z.ping_count - p.ping_count                 as ping_delta
from {{ ref('mart_zone_daily_activity') }} as z
join {{ source('parity', 'zone_daily_activity_postgis') }} as p
  using (zone_id, activity_date)
yaml
models:
  - name: int_port_parity
    tests:
      - dbt_utils.expression_is_true:
          expression: "abs(ping_delta) <= 2"
      - dbt_utils.accepted_range:
          column_name: area_diff
          max_value: 0.01

A ping delta of one or two rows per zone per day is the expected consequence of geodesic edges reassigning a handful of boundary points. A delta of hundreds means a predicate changed meaning — go back to step 3.

Migration sequence with the parity gate placed before consumers are switched over A left-to-right sequence: inventory the spatial calls, translate direct equivalents, rewrite unit-changed calls, restructure functions with no equivalent, then run the parity comparison. Only after the parity gate passes does traffic move from the PostGIS marts to the BigQuery marts, shown as a switch at the right with both engines running in parallel until then. 1 · inventory grep the models 2 · rename direct equivalents 3 · re-unit degrees → metres 4 · restructure no-equivalent calls 5 · parity gate areas and counts within tolerance cut over consumers move PostGIS keeps serving production for the whole of this period — both engines build nightly The gate is what makes the cut-over reversible: until it passes, nothing downstream has moved.

Configuration reference

PostGIS construct BigQuery approach Migration note
geometry(Polygon, 4326) column GEOGRAPHY column No type modifier or SRID; enforce shape with tests
CREATE INDEX … GIST cluster_by in model config Declared at write time, not added afterwards
ST_Transform(geom, 3035) reproject before load No in-warehouse equivalent
ST_Subdivide(geom, 256) grid-key prefilter Different optimisation for a different engine
ST_MakeValid(geom) make_valid => true on the constructor Repair moves to parse time
ANALYZE post-hook nothing BigQuery maintains its own metadata
ST_DWithin(g, g, 0.0045) ST_DWITHIN(g, g, 500) Degrees to metres — convert every literal deliberately

Gotchas & edge cases

  • A ported ST_Simplify tolerance of 0.0001 becomes a 0.0001-metre tolerance — effectively no simplification, so payload sizes silently stay large. Check the output vertex counts, not just that the query ran.
  • ST_Union is ST_UNION_AGG in aggregate position and ST_UNION for two arguments; porting the wrong one produces a type error at compile time, which is the good case.
  • BigQuery has no ST_SetSRID, so the common PostGIS idiom of stamping an SRID onto SRID-0 geometry has no target. Fix the source instead.
  • Antimeridian-crossing geometry behaves differently. BigQuery takes the shorter path around the globe; planar engines do not. Datasets spanning the Pacific need explicit testing. Why a Pacific-spanning geometry ports differently between planar and geodesic engines A flattened world strip with the antimeridian marked at the right edge. A shape defined between longitude 170 east and 170 west is drawn twice: the planar engine connects the two points across the entire map through zero longitude, producing a very wide polygon, while the geodesic engine takes the short 20-degree path across the antimeridian. The two interpretations differ by nearly the whole width of the world. 0° longitude 180°W 180°E planar reading · spans the whole map geodesic reading · the short 20° hop across the antimeridian Same two coordinates, two polygons that differ by almost the width of the world.
  • SAFE. prefixes only work on some functions. SAFE.ST_GEOGFROMTEXT is supported; wrapping arbitrary spatial calls in SAFE. is not a general strategy.

FAQ

Is there an automated way to port the SQL?

Not reliably, and the reason is step 3: the dangerous cases are calls whose names are identical and whose meaning changed. A tool that renames functions will silently pass those through. The grep inventory plus a parity test is faster in practice than debugging an automated port.

How do I keep both engines working during a long migration?

Put every spatial call behind a dispatched macro from the start, so one model file compiles for both targets, and run both builds nightly with the parity model comparing them. That way the migration is a series of small verified steps rather than one cut-over weekend.

What tolerance is realistic for the parity test?

For metro-scale polygons, one per cent on area and a couple of rows on point-in-polygon counts. For regional or national polygons with long east–west edges, geodesic and planar areas can differ by several per cent legitimately — densify the edges before comparing, or raise the threshold knowingly rather than by trial and error.

Do I need to change my dbt tests as well as my models?

The spatial ones, yes — an ST_IsValid test has no meaning on BigQuery and should become a null-after-parse test. Generic tests carry over untouched, which is a good argument for expressing as many invariants as possible with unique, not_null, relationships and accepted_range rather than with spatial SQL.

Up: Part of Warehouse-Native GIS Adapters.