Tagging and selecting spatial models in dbt runs

This page sets up the tag vocabulary and YAML selectors that let a spatial project run the slice it needs: a fast attribute refresh every hour, the expensive geometry joins nightly, the CI fixtures never in production, and the index rebuilds only when a full refresh has actually dropped them.

When to use this approach

  • Your build is one job that does everything. Spatial DAGs have wildly uneven cost, so an all-or-nothing build is either too slow or too stale.
  • Something is running that should not be. Fixture models in production and tile pyramids on every hourly run are the two classics.
  • You need selection to be reviewable. Selectors defined in YAML are versioned and diffable; the same logic buried in an orchestrator’s shell command is not. The DAG structure this operates on is described in spatial model dependency graphs.

Prerequisites

  • dbt 1.5+ for selectors.yml and the --selector flag.
  • A DAG that already builds correctly end to end; selection makes a working build cheaper, it does not fix a broken one.
  • Agreement on what each tag means, written down. A tag vocabulary nobody agreed on becomes noise within a quarter.

Step-by-step instructions

1. Define a small tag vocabulary

Resist the temptation to tag by team, domain and layer at once. Four tags describe most spatial projects, and each answers a scheduling question rather than a descriptive one.

yaml
# models/schema.yml (excerpt)
models:
  - name: int_pings_zoned
    config:
      tags: ['geometry_heavy']
  - name: mart_zone_labels
    config:
      tags: ['attribute_only']
  - name: seed_broken_geometry_fixtures
    config:
      tags: ['ci_only']
      enabled: "{{ target.name == 'ci' }}"
  - name: serve_zone_tiles
    config:
      tags: ['serving', 'geometry_heavy']
yaml
# dbt_project.yml — tag whole directories rather than every model
models:
  dbt_geospatial:
    intermediate:
      +tags: ['geometry_heavy']
    serving:
      +tags: ['serving']

Verify the tags landed where you think:

bash
dbt ls --select tag:geometry_heavy --output name
dbt ls --select tag:ci_only --target prod --output name
# The second should print nothing — the fixtures are disabled outside CI
Four schedules over one DAG, each selecting a different slice by tag One dependency graph is shown four times at reduced size, each with a different subset highlighted. The hourly schedule selects attribute-only models. The nightly schedule selects geometry-heavy models and everything downstream. The release schedule selects serving models. The pull-request schedule selects modified models plus CI-only fixtures. A note records that all four run against the same project with no branching logic in the models. hourly · attribute_only 2 models · 40 seconds nightly · geometry_heavy+ 5 models · 34 minutes release · serving tiles and features only PR · state:modified+ changed nodes + fixtures One project, one DAG, four schedules — and no branching logic inside any model.

2. Write the selectors in YAML, not in the scheduler

yaml
# selectors.yml
selectors:
  - name: hourly_attributes
    description: Cheap attribute refresh; touches no geometry.
    definition:
      union:
        - method: tag
          value: attribute_only
        - method: fqn
          value: marts.mart_zone_labels

  - name: nightly_geometry
    description: Expensive spatial work and everything downstream of it.
    definition:
      union:
        - method: tag
          value: geometry_heavy
          children: true
      exclude:
        - method: tag
          value: ci_only

  - name: pr_check
    description: Modified nodes plus the negative fixtures that prove the gate fires.
    definition:
      union:
        - method: state
          value: modified
          children: true
        - method: tag
          value: ci_only
bash
dbt build --selector nightly_geometry --target prod
dbt build --selector hourly_attributes --target prod
dbt build --selector pr_check --target ci --state ./prod-manifest

Verify each selector resolves to the set you expect before wiring it into a schedule:

bash
dbt ls --selector nightly_geometry --output path | wc -l
dbt ls --selector hourly_attributes --output path

3. Keep index rebuilds out of the hourly path

Post-hooks that create indexes are cheap when the index exists (IF NOT EXISTS) and expensive when it does not. That asymmetry matters because a full refresh drops the table and the index with it.

sql
{{ config(
    materialized = 'incremental',
    tags = ['geometry_heavy'],
    post_hook = [
      "{% if flags.FULL_REFRESH %}CREATE INDEX IF NOT EXISTS {{ this.name }}_geom_idx ON {{ this }} USING GIST (geom){% endif %}",
      "{% if flags.FULL_REFRESH %}ANALYZE {{ this }}{% endif %}"
    ]
) }}

Verify the index survives an incremental run and is rebuilt by a full one:

sql
select indexname from pg_indexes where tablename = 'int_pings_zoned';
-- Present after both; the hook simply does nothing on incremental runs

4. Make the exclusions explicit and tested

The dangerous selection is the one that quietly includes something. Assert the negative case in CI rather than trusting the tag.

sql
-- tests/assert_no_ci_fixtures_in_prod.sql
select table_name
from information_schema.tables
where table_schema = '{{ target.schema }}'
  and table_name like '%fixture%'
  {% if target.name != 'ci' %}
  -- in any non-CI target, finding one at all is the failure
  {% else %}
  and false
  {% endif %}

Verify by running the production build and confirming the fixtures were never created:

bash
dbt build --selector nightly_geometry --target prod
dbt test --select assert_no_ci_fixtures_in_prod --target prod
Build cost before and after splitting the schedule by tag Before, a single build runs every model every hour, so twenty-four expensive geometry builds happen per day. After, the geometry-heavy slice runs once nightly and the attribute slice runs hourly, cutting daily compute by roughly ninety percent while making attributes fresher than before. Two bars compare daily minutes, and a note records that no model changed. one build, hourly 24 × 34 min = 816 min/day split by tag 1 × 34 min nightly 24 × 40 s hourly ≈ 50 min/day in total attributes are now fresher, not staler — they run every hour instead of waiting Not one model changed. Only what runs, and when. The direction of the plus operator, and what each reading runs A small chain of five models with the tagged model in the middle. Selecting the tag with a trailing plus highlights the tagged model and the two downstream of it. Selecting with a leading plus highlights the tagged model and the two upstream of it, which are the expensive staging models the schedule was trying to skip. A caption notes that reversing the operator inverts the intent. tag:geometry_heavy+ — the tagged model and its descendants tagged 3 models +tag:geometry_heavy — the tagged model and its ancestors the expensive staging you meant to skip

Configuration reference

Selector method Example Use
tag tag:geometry_heavy The main axis for scheduling
tag with children tag:geometry_heavy+ Include everything downstream of the expensive work
state state:modified+ PR builds; needs a stored production manifest
fqn marts.mart_zone_labels Pin one model into a slice without tagging it
config.materialized config.materialized:incremental Useful for a full-refresh-only schedule
exclude exclude: tag:ci_only The clause that keeps fixtures out of production

Gotchas & edge cases

  • A tag is not a guard. Tagging a fixture ci_only does not stop dbt build with no selector from building it; pair the tag with enabled: "{{ target.name == 'ci' }}".
  • + is directional and easy to reverse. tag:x+ selects descendants, +tag:x selects ancestors. Selecting ancestors when you meant descendants runs the expensive upstream models you were trying to skip.
  • state:modified needs a manifest from the right environment. Comparing against a stale manifest silently rebuilds far more than expected — or far less.
  • Tests inherit tags from their models, but singular tests do not. Tag singular tests explicitly, or a slice will run without the tests that protect it.
  • An hourly slice that depends on a nightly model reads yesterday’s data. That is usually fine and occasionally a serious bug; state it in the model description so consumers know which freshness they are getting.

FAQ

How many tags should a project have?

As few as the schedules require — typically three to five. Tags that describe rather than schedule (domain:mobility, owner:platform) are better expressed as meta fields, which are queryable in the manifest without cluttering the selection namespace.

Should selectors live in the repository or the orchestrator?

In the repository, in selectors.yml. They are then versioned with the models they select, reviewed in the same pull request, and identical whether a human or a scheduler invokes them. The orchestrator should name a selector, not compose one.

What is the right slice for a pull-request build?

state:modified+ plus the CI fixtures, run against a small dataset. That builds only what changed and everything downstream of it, which is the set a reviewer cares about; the full DAG on production data belongs to the nightly schedule, not to a PR.

How do I stop the expensive slice from being run by accident?

Give it a selector and no default. A bare dbt build on a production target should be either impossible in your orchestration or explicitly safe; relying on people remembering the right flags is how a full tile-pyramid rebuild happens on a Friday afternoon.

Up: Part of Spatial Model Dependency Graphs.