Tagging and selecting spatial models in dbt runs
This page sets up the tag vocabulary and YAML selectors that let a spatial project run the slice it needs: a fast attribute refresh every hour, the expensive geometry joins nightly, the CI fixtures never in production, and the index rebuilds only when a full refresh has actually dropped them.
When to use this approach
- Your build is one job that does everything. Spatial DAGs have wildly uneven cost, so an all-or-nothing build is either too slow or too stale.
- Something is running that should not be. Fixture models in production and tile pyramids on every hourly run are the two classics.
- You need selection to be reviewable. Selectors defined in YAML are versioned and diffable; the same logic buried in an orchestrator’s shell command is not. The DAG structure this operates on is described in spatial model dependency graphs.
Prerequisites
- dbt 1.5+ for
selectors.ymland the--selectorflag. - A DAG that already builds correctly end to end; selection makes a working build cheaper, it does not fix a broken one.
- Agreement on what each tag means, written down. A tag vocabulary nobody agreed on becomes noise within a quarter.
Step-by-step instructions
1. Define a small tag vocabulary
Resist the temptation to tag by team, domain and layer at once. Four tags describe most spatial projects, and each answers a scheduling question rather than a descriptive one.
# models/schema.yml (excerpt)
models:
- name: int_pings_zoned
config:
tags: ['geometry_heavy']
- name: mart_zone_labels
config:
tags: ['attribute_only']
- name: seed_broken_geometry_fixtures
config:
tags: ['ci_only']
enabled: "{{ target.name == 'ci' }}"
- name: serve_zone_tiles
config:
tags: ['serving', 'geometry_heavy']
# dbt_project.yml — tag whole directories rather than every model
models:
dbt_geospatial:
intermediate:
+tags: ['geometry_heavy']
serving:
+tags: ['serving']
Verify the tags landed where you think:
dbt ls --select tag:geometry_heavy --output name
dbt ls --select tag:ci_only --target prod --output name
# The second should print nothing — the fixtures are disabled outside CI
2. Write the selectors in YAML, not in the scheduler
# selectors.yml
selectors:
- name: hourly_attributes
description: Cheap attribute refresh; touches no geometry.
definition:
union:
- method: tag
value: attribute_only
- method: fqn
value: marts.mart_zone_labels
- name: nightly_geometry
description: Expensive spatial work and everything downstream of it.
definition:
union:
- method: tag
value: geometry_heavy
children: true
exclude:
- method: tag
value: ci_only
- name: pr_check
description: Modified nodes plus the negative fixtures that prove the gate fires.
definition:
union:
- method: state
value: modified
children: true
- method: tag
value: ci_only
dbt build --selector nightly_geometry --target prod
dbt build --selector hourly_attributes --target prod
dbt build --selector pr_check --target ci --state ./prod-manifest
Verify each selector resolves to the set you expect before wiring it into a schedule:
dbt ls --selector nightly_geometry --output path | wc -l
dbt ls --selector hourly_attributes --output path
3. Keep index rebuilds out of the hourly path
Post-hooks that create indexes are cheap when the index exists (IF NOT EXISTS) and expensive when it does not. That asymmetry matters because a full refresh drops the table and the index with it.
{{ config(
materialized = 'incremental',
tags = ['geometry_heavy'],
post_hook = [
"{% if flags.FULL_REFRESH %}CREATE INDEX IF NOT EXISTS {{ this.name }}_geom_idx ON {{ this }} USING GIST (geom){% endif %}",
"{% if flags.FULL_REFRESH %}ANALYZE {{ this }}{% endif %}"
]
) }}
Verify the index survives an incremental run and is rebuilt by a full one:
select indexname from pg_indexes where tablename = 'int_pings_zoned';
-- Present after both; the hook simply does nothing on incremental runs
4. Make the exclusions explicit and tested
The dangerous selection is the one that quietly includes something. Assert the negative case in CI rather than trusting the tag.
-- tests/assert_no_ci_fixtures_in_prod.sql
select table_name
from information_schema.tables
where table_schema = '{{ target.schema }}'
and table_name like '%fixture%'
{% if target.name != 'ci' %}
-- in any non-CI target, finding one at all is the failure
{% else %}
and false
{% endif %}
Verify by running the production build and confirming the fixtures were never created:
dbt build --selector nightly_geometry --target prod
dbt test --select assert_no_ci_fixtures_in_prod --target prod
Configuration reference
| Selector method | Example | Use |
|---|---|---|
tag |
tag:geometry_heavy |
The main axis for scheduling |
tag with children |
tag:geometry_heavy+ |
Include everything downstream of the expensive work |
state |
state:modified+ |
PR builds; needs a stored production manifest |
fqn |
marts.mart_zone_labels |
Pin one model into a slice without tagging it |
config.materialized |
config.materialized:incremental |
Useful for a full-refresh-only schedule |
exclude |
exclude: tag:ci_only |
The clause that keeps fixtures out of production |
Gotchas & edge cases
- A tag is not a guard. Tagging a fixture
ci_onlydoes not stopdbt buildwith no selector from building it; pair the tag withenabled: "{{ target.name == 'ci' }}". +is directional and easy to reverse.tag:x+selects descendants,+tag:xselects ancestors. Selecting ancestors when you meant descendants runs the expensive upstream models you were trying to skip.state:modifiedneeds a manifest from the right environment. Comparing against a stale manifest silently rebuilds far more than expected — or far less.- Tests inherit tags from their models, but singular tests do not. Tag singular tests explicitly, or a slice will run without the tests that protect it.
- An hourly slice that depends on a nightly model reads yesterday’s data. That is usually fine and occasionally a serious bug; state it in the model description so consumers know which freshness they are getting.
FAQ
How many tags should a project have?
As few as the schedules require — typically three to five. Tags that describe rather than schedule (domain:mobility, owner:platform) are better expressed as meta fields, which are queryable in the manifest without cluttering the selection namespace.
Should selectors live in the repository or the orchestrator?
In the repository, in selectors.yml. They are then versioned with the models they select, reviewed in the same pull request, and identical whether a human or a scheduler invokes them. The orchestrator should name a selector, not compose one.
What is the right slice for a pull-request build?
state:modified+ plus the CI fixtures, run against a small dataset. That builds only what changed and everything downstream of it, which is the set a reviewer cares about; the full DAG on production data belongs to the nightly schedule, not to a PR.
How do I stop the expensive slice from being run by accident?
Give it a selector and no default. A bare dbt build on a production target should be either impossible in your orchestration or explicitly safe; relying on people remembering the right flags is how a full tile-pyramid rebuild happens on a Friday afternoon.
Related
- Spatial Model Dependency Graphs — the DAG this selects from.
- Running Spatial Tests in GitHub Actions — where the PR selector is used.
- Monitoring Spatial Model Run Times in dbt — measuring what each slice costs.
Up: Part of Spatial Model Dependency Graphs.