Running this package as a CI check#

You can use this package as a CI check to ensure code changes don't introduce new misalignments. To fail on any violation of the whole project, first address the current misalignments (either by fixing them or accepting them as exceptions); to fail only on the violations a change introduces, select the modified resources.

Since the rules are native dbt checks, there is nothing to build in your warehouse: the checks run when the project is parsed, and a check configured with severity: error makes the command fail when it finds violations.

1. Make the checks fail in CI#

By default, the checks in this package are configured with "warn" severity: they report the violations but don't make dbt check or dbt build fail. You can set the severity for the whole package, for a category of checks or for an individual check in your dbt_project.yml:

checks:
  dbt_project_evaluator:
    +severity: error              # every check
    documentation:
      +severity: warn             # but only warn for a whole category
      fct_undocumented_models:
        +severity: error          # except for this one

To only make the checks fail in CI, you can use an environment variable, set to "error" in the CI environment and left unset everywhere else:

checks:
  dbt_project_evaluator:
    +severity: "{{ env_var('DBT_PROJECT_EVALUATOR_SEVERITY', 'warn') }}"

2. Run the checks for each pull request#

The checks are executed at the beginning of dbt build, so a CI job running dbt build already evaluates your project. When a check with severity: error finds violations, dbt build fails before building any model.

You can also run only the checks, without building anything, with

dbt check

The command exits with a non-zero code when a check configured with severity: error returns rows.

Only reporting the violations on the modified resources#

Each check returns the resource to fix in the column unique_id, which allows --select to restrict the violations reported to the selected resources. Combined with state:modified, which compares your project with the artifacts of your main branch, this reports the violations introduced by the pull request and ignores the existing ones:

dbt check --select state:modified --state path/to/main/target   # GitHub Actions, local...
dbt check --select state:modified                               # dbt platform CI job

--state is the folder with the artifacts of your main branch. In a CI job on the dbt platform, the state of the production environment is provided to the job through deferral, so you don't pass --state. Anywhere else, you have to produce the artifacts of your main branch and pass their location, as in the example below.

For example, after modifying the model stg_orders, the checks that look at individual resources (fct_undocumented_models, fct_missing_primary_key_tests...) only report stg_orders.

Example with GitHub Actions#

The following workflow parses the base branch of the pull request to get a manifest to compare to, then runs the checks on the modified resources:

name: dbt project evaluator

on: pull_request

jobs:
  checks:
    runs-on: ubuntu-latest
    env:
      DBT_PROJECT_EVALUATOR_SEVERITY: error
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - run: pip install "dbt>=2.0.0,<3.0.0"

      - name: Parse the base branch to get the state to compare to
        run: |
          git worktree add ../base "origin/${{ github.base_ref }}"
          cd ../base
          dbt deps
          dbt parse

      - run: dbt deps

      - name: Check the modified resources
        run: dbt check --select state:modified --state ../base/target