driftpipev0.9.4docs for Sign in

GitLab CI

NAME

gitlab - a job on the published image, with the pipefail trap

Running from the container image removes the download step entirely, at the cost of one shell subtlety that silently defeats the check if it is left out.

JOB

.gitlab-ci.yml
schema-drift:
  stage: test
  image:
    name: reg.driftpipe.sh/driftpipe:0.9.4
    entrypoint: [""]
  script:
    - set -o pipefail
    - driftpipe check --input fixtures/orders.ndjson --format json | tee drift.json
  artifacts:
    when: always
    expire_in: 1 week
    paths: [drift.json]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_PIPELINE_SOURCE == "schedule"

THE PIPEFAIL TRAP

Without set -o pipefail, the status of driftpipe ... | tee drift.json is the status of tee, which is zero, and the job goes green over genuine drift. This is the single most common way a schema check is deployed broken, and it is not specific to this tool.

The entrypoint: [""] line clears the image default, which is the binary itself, so that the script block gets a shell to run in. Keep the entrypoint instead and you drop the script block and pass arguments through the job command.

WHAT IT CATCHES

The JSON artifact is retained on both pass and fail, so a drift that fails the pipeline leaves drift.json attached for inspection without re-running anything. The rules block scopes the job to merge requests and scheduled runs, keeping it off unrelated branch pushes.

GOTCHAS

  • Pin the image to 0.9.4, never latest: a classification that changes under you is indistinguishable from data that changed.
  • The artifact is handy, but the exit status is the signal; do not gate the pipeline on parsing the JSON when the code already tells you.